Compiled MATLAB with TORQUE

I need to process a lot of data in parallel, and then plot the result of that processing. I have access to a cluster (~100 cores), and currently run the parallel data processing using compiled matlab code (mcc -mv), and then using qsub. What would be optimal, however, is to be able to call qsub from within a matlab script, have it submit the jobs to the cluster, wait until they are all completed, and then continue plotting, processing, in serial, in matlab. Is such a thing possible? If so, any help would be great.

Thanks!

Answer 1

[Jason Ross]

If you have Parallel Computing Toolbox and MATLAB Distributed Computing Engine installed/configured on the cluster, you can set up a direct integration using the Parallel menu (this assumes a number of things -- you have qsub on the path, and a common file system). If you don't have those things, there are examples in matlabroot/toolbox/distcomp/examples that detail how to set up the integration between MATLAB and your cluster for a variety of scenarios.

Once the integration is set up, you can submit the jobs to the cluster using various commands and techniques (batch, job/task, spmd, parfor) from within MATLAB that are detailed in the Parallel Computing Toolbox documentation.

[Danny]

My school does not provide neither the parallel computing tool box or the distributed computing toolbox to clusters. Thank you for the help, though.

Answer 2

[Muthu Annamalai]

You may alternatively consider writing a "monitor" thread or app using the MATLAB timer() functionality.

See Time object