How to use a supercomputer
35 views (last 30 days)
I'm working on a code which has several loops. I need to run this code for very small increments, however, when I estimated the time for this, it gave me roughly 512 days. This has been done on my computer which has only 2 cores. I have now access to a 64 and higher core supercomputer. But when I use parallel computing using parfor and matlabpool, I cannot open more than 8 cores. It doesn't really change a lot the simulations' time. I was wondering if anyone can help me use up all the 64 or higher cores for my program. Please let me know if there is anyway to activate all the possible cores to run my program in a parallel manner.
Walter Roberson on 18 Mar 2013
A later version of MATLAB would permit 12 cores.
Beyond that you need to use the Distributed Computing Toolbox.
Jan on 19 Mar 2013
Edited: Jan on 19 Mar 2013
I suggest to post the relevant part of the code at first. When increasing the number of cores from 2 to 8 does not decrease the runtime significantly, expanding to 64 cores will not help much more. But I haven't seen a code, which could not profit from the massive knowhow of the forum members. While improvements of the code can increase the program by a factor of 1000, 64 cores can be 32 times faster only in the absolute not realistic best case.
Jason Ross on 12 Apr 2013
Basic outline of how to connect to a MDCS server using PCT:
- If you have a recent enough MATLAB, use "Discover Clusters" under the Parallel Menu and see if your cluster shows up. Validate the profile when prompted to do so to make sure the cluster is indeed configured correctly.
- Ask your admin staff if they have a cluster profile they used to validate that the cluster was working properly. They should be able to save/export it and then you can import it.
- If you don't have Discover Clusters, make a new MDCS profile using the Profile Manager, filling in the required blanks and then validating it at the end.
When the cluster validates, you should be good to go. The easiest way to try out your code on the cluster is to set your profile as the default and just run your m-code. If you've got it working with the local profile already, it's highly likely that it will "just work" on the cluster. The only thing you may need to do is add file or folder dependencies in the cluster profile if you have external dependencies you need on the cluster.