Does SPMD always use all cores?

So far as I know, parfor will not use all available cores if Matlab considers the task as not sufficiently heavy.

But does SPMD always use all available cores even if the task is small? It seems to me YES as far as I have tried.

I am working on the shared memory system. Due to the memory restriction, I cannot increase the size of each sub-task in order to increase the computational load. If SPMD can use all the cores regardless of the computational load, then it will be probably more suitable in my case, as SPMD will then not waste the computational ability of the hardwares. But if I finally resort to SPMD, then I will have to make a certain amount of changes to my code.

Any opinions are appreciated. Thank you.

[Adam]

I've not really noticed parfor using anything less than the number of cores available within the opened Matlab pool, unless, of course, there aren't as many indices in the parfor loop as there are cores. I haven't studied it closely though.

[Hanzhi Diao]

I had the same idea with you and expected that parfor could take full use of the computational ability. But the observation has denied my expectation.

I have no idea about the system you work on. The system that I use consists of 80 cores sharing 2TB memory. A typical computation will last around one hour and the entire task is usually divided into about one thousand sub-tasks independent of each other. The computation of each sub-task takes several minutes.

Using 32 cores and 64 cores, the difference in computation time is no more than 300 seconds shown by tic/toc. I believe that my code itself can still be to some extent optimised. Besides that I suspect that some cores cannot be fully engaged and therefore stay idle in the most time of the computation using parfor. This is also observed with the top command in linux environment.

Answer 1

[Edric Ellis]

spmd blocks by default use all the workers in your parallel pool. You can make an spmd block use fewer workers by specifying the optional argument, like so

parpool('local', 4);
spmd(2)
  assert(numlabs == 2);
end

[Hanzhi Diao]

Thanks a lot. I am now on the work changing the parfor code to SPDM.