Distributed and parallel computing without DCS

Is there a way to perform distributed computing across multiple computers without licensing Distributed Computing Server (DCS)? It is implied that Sun Grid Engine may be used without DCS, but my attempts to integrate torque+maui produce licensing errors. Which schedulers/resource managers are possible without the quite expensive (relative term, of course) DCS?

[Chuck]

Has no one integrated Sun Grid Engine and MATLAB? Can anyone help with this?

Answer 1

[Daniel Shub]

If you have the MATLAB compiler you can compile your code and then run it with the Sun Grid Engine without MATLAB or the DCS toolbox.

[Thomas]

A good suggestion... Compiler might be more cost effective than the DCS licensing (which depends on the number of DCS nodes)

[Daniel Shub]

It limits what you can do. Everything I do is embarrassingly parallel, so I can get away with it. There are advantages of the PCT and DCS. With the PCT you can use parfor, with the compiler you need to roll your own. I shudder to think about trying to deal with distributed arrays and the such.

[Ah So]

Hi all If i want to use MATLAB compiler to compile the matlab program to Java or C++ program and then deploy with grid technology such as Datasynapse without using DCS, is it viable? If yes, is there any document / URL could provide very details. Many thx for sharing the info

[Daniel Shub]

You need to provide a lot more details to get a useful answer. The comments of an answer is probably not the best place to ask this question. You should ask a new question (it might be converting my MATLAB code to Java/C, or making an executable from MATLAB code, or modifying MATLAB code so that it will work in a distributed environment, or deploying a set of executables on a Datasynapse grid). The last not really being a MATLAB question.

Answer 2

[Thomas]

Hi Chuck,

In order to perform distributed computing across multiple computers you need the DCS license. We have DCS running with the Sun Grid Engine on our Cheaha cluster at UAB. The implication that SGE may be used with out the DCS is not correct. What I think was implied was that - You can use the SGE scheduler instead of the scheduler provided by the MATHWORKS (MATHWORKS job manager) along with the DCS.

Matlab page http://www.mathworks.com/products/distriben/supported/sched/ clearly states:

 Grid Engine Family:
 Grid Engine
 Sun N1 Grid Engine   

 Sample integration scripts are available with the
 MATLAB Distributed Computing Server installation.

In short- you cannot use the distributed computing without the DCS.

You can use the parallel computing toolbox on your local multicore machine to use upto 12 workers to run in parallel with R2011b.

If you need help integrating DCS with SGE there are sample scripts available or you can contact MATHWORKS support.

Answer 3

[Jason Ross]

To add a little additional information: If you want to use Torque/PBS, you can do so with the direct integration from the Parallel menu, or using scripts similar to the ones for SGE if your environment won't allow the use of the direct integration.

Thomas and Anthony have already covered the licensing issues fairly thoroughly.

[Ah So]

Hi Jason, per your above additional info, does it means it is technically possible to use the the Parallel menu of PCT to directly integrate with some scheduler such as SGE or grid technology without DCS? If so, is there any document / URL could provide very details. Many thx for sharing the info

Answer 4

[Konrad Malkowski]

Starting with MATLAB R2010b MATLAB Compiler allows you to compile MATLAB code containing PARFOR and SPMD running on local scheduler.

The same limitations as with PCT local scheduler apply to compiled MATLAB applications.

[Martijn]

This is not correct. The feature was added in release R2011a. See the MATLAB Compiler release notes:

http://www.mathworks.com/help/compiler/release-notes.html

Answer 5

[Chuck]

Thanks for this information. We do have the compiler licensed, so that may be a good solution for us.