Utilizing all cores with SPMD

I have the following function.

It performs two optimizations in parallel. However, to my understanding each optimization is done on only one WORKER.

I want to know if it is possible to run both optimizations in parallel and that each will utilize more than one worker (hopefully both optimizations will utilize all 32 workers):

function [XX,XXX]=spmd_test
c = parcluster;
c.NumWorkers=32;
poolobj=parpool(c);
D=6;
X0=10;
X1=100;
fh=@(X)myfun(X,D);
spmd
    switch labindex
        case 1
            XX = fminsearch(fh,X0,optimset('UseParallel','always','TolFun',1e-8,'TolX',1e-8,'MaxFunEvals',1000000,'MaxIter',1000000));
        case 2
            XXX = fminsearch(fh,X1,optimset('UseParallel','always','TolFun',1e-8,'TolX',1e-8,'MaxFunEvals',1000000,'MaxIter',1000000));
    end
end
XX=XX{1};
XXX=XXX{2};
end

[Walter Roberson]

https://www.mathworks.com/matlabcentral/answers/452964-is-it-possible-to-determine-the-cpu-capacity-of-each-parfor-worker#comment_687146

[Rony Reuveni]

Hi Walter

I read the link and the links in the link

It seems that no matter what I try to do each optimization seems to use only one thred of one CPU.

My PC has two "Intel® Xeon® Processor E5-2687W". Each has 8 cores and 16 threads. So a total of 32 threads.

The optimization in myfun contains a loop of 100,000 iterations for calculations and on one thread it takes 4.5 seconds.

When adding more fminsearches with additional labindex cases each one takes 4.5 seconds. What I want is to speedup each of the fminsearches. I want each fminsearch to use more than one thread.

I tried changing the for loop in the fminsearch to a parfor loop but this only caused the time to increase.

In the cluster profile manager I couldn't find any parameter controlling the number of threads per worker as suggested here:

https://www.mathworks.com/matlabcentral/answers/383925-the-assignment-of-workers-to-cores#comment_540082

I did however check and the N = maxNumCompThreads returns a value of 16.

I also tried to change in the cluster profile manager the value of "Range of number of workers to run job NumWorkersRange" from [1 inf] to [4 inf] so hopfully it will force each worker to use a minimum of 4 threads however I saw no change in performance.

So I still haven't succedded to force the fminsearch to use more than 1 thread.

Hope you can help.

Thanks,

[Walter Roberson]

Sorry, I do not have a dual-CPU machine to experiment with.

[Rony Reuveni]

Hi Walter

I am not sure a dual-CPU PC is needed. Isn't a PC with 1 CPY & many cores also enough?

Thanks for your help !

[Walter Roberson]

No. The behavior you are observing is not present on my single CPU multiple core machine.

[Rony Reuveni]

Do you maybe have an idea why in the cluster profile manager I couldn't find any parameter controlling the number of threads per worker as suggested here:

https://www.mathworks.com/matlabcentral/answers/383925-the-assignment-of-workers-to-cores#comment_540082

[Walter Roberson]

I just noticed that you are using R2016a. If I recall correctly that parameter was added to the cluster profile in a later release. Which release exactly is not coming to mind at the moment.

If I recall correctly you might be able to make the call to change the maximum threads in each worker, perhaps with parfevalOnAll. You would need to take some care not to ask for too many workers.

Answer 1

[Rony Reuveni]

Hopefully, could someone help on this subject?

Thanks,