Problem with parfor loop
38 vues (au cours des 30 derniers jours)
Afficher commentaires plus anciens
Hello,
I need help, I have this problem when I use parfor loop for this code, where x a square matrix of order N = 75692 of size approximately equal to 13 Gb and t a row vector of ![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/537451/image.png)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/537451/image.png)
N=length(t);
alpha= [2.4e-7, 2.5e-7, 2.5e-7, 2.3e-6];
beta = [0.5, 0.8, 0.6, 0.5 ];
YT=zeros(N,4);
YD=zeros(N,4);
parfor ima1:4
[YT(:,ima),YD(:,ima)]=f(t,x,alpha(ima),beta(ima));
end
Error using distcomp.remoteparfor/getCompleteIntervals (line 133)
2 commentaires
Réponses (1)
Edric Ellis
le 4 Mar 2021
That error basically means a worker crashed while trying to run the parfor loop. You mention that x is large. If you are using the 'local' cluster, then please be aware that x must be copied to each of the worker processes. This can cause a large amount of memory usage, and it's possible (probable?) that this is causing the workers to shut down.
If you're using a recent version of MATLAB, you might be able to use parpool('threads'), which uses multiple computational threads in a single process, and can avoid some memory duplication. (But not all MATLAB functions can operate in this environment).
Otherwise, you are going to be constrained by the memory on your system. Transferring x to the workers from the client incurs additional duplication while the messages are in transit, so if you're only just exceeding the memory, and you can build x directly on the workers by executing a function, then the following pattern might help:
xC = parallel.pool.Constant(@myFunctionThatBuildsX); % build 'x' directly on the worker
parfor ...
[YT(:,ima),YD(:,ima)]=f(t,xC.Value,alpha(ima),beta(ima));
end
5 commentaires
Edric Ellis
le 5 Mar 2021
Right, but I was wondering if with parpool(1) the single worker still crashed.
Voir également
Catégories
En savoir plus sur Parallel Computing Fundamentals dans Help Center et File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!