Effacer les filtres
Effacer les filtres

Parallel for loop problem

1 vue (au cours des 30 derniers jours)
Simone Fiumi
Simone Fiumi le 12 Août 2022
Commenté : Simone Fiumi le 12 Août 2022
I'm running the following code in parallel. The variables of interes are w_b and alpha_b. The problem is that if I pause at a random time and then quit debugging the variables computed up to now are not saved. If i do that without the parallel cicle but with a simple for loop the variables are saved. For example, let's say I stop after 1 hour and I have 5 variables saved using a standard for loop, while with a parfor loop there are no saved variables (they are still equal to 0 as preallocated).
parfor k=1:Nsim
for j=1:size(M,1)
[LL, ~, P, obsmat, nrIterations]=dhmm_em(seq, prior, P_start_b, E,N_btstrp,Y_btstrp);
[w_b(k),alpha_b(k)]=parameters(P,ne,nn,N_btstrp,Y_btstrp,1); % VARIABLES OF INTEREST
  2 commentaires
Jeffrey Clark
Jeffrey Clark le 12 Août 2022
@Simone Fiumi, there is no guarenteed order of processing of the parfor indexs. Are you sure you didn't get 5 or more values somewhere (but the same somewhere) in each of w_b and alpha_b vectors? If Nsim is large the run might have filled in end-(0:4:20) and you might not have noticed.
Simone Fiumi
Simone Fiumi le 12 Août 2022
I don't know if that happened or not. Right now it is running and since I don't want to risk I'll wait for it to end completely so that I won't encounter this problem. Thanks.

Connectez-vous pour commenter.

Réponses (2)

Raymond Norris
Raymond Norris le 12 Août 2022
@Simone Fiumi think of the code running in the parfor as running on some entirely different machine. There is no "debugging" per se. So pausing/stopping the parfor terminates the work entiring including the ability to checkpoint variables.
What you could consider doing is creating a data queue that sends data back to the MATLAB client. The client (outside of the parfor) could view the data queue. For instance:
function sf
D = parallel.pool.DataQueue;
parfor idx = 1:10
[wb, ab] = parameters(P,ne,nn,N_btstrp,Y_btstrp,1);
% Send data back to client
send(D,[wb ab])
w_b(k) = wb;
alpha_b(k) = ab;
function logdata(D)
% Evaluate a snapshot of wb, ab
Not a perfect solution, but might help.
  1 commentaire
Simone Fiumi
Simone Fiumi le 12 Août 2022
Yes it could be helpful but unfortunatly I don't how to use and write such functions. Thanks.

Connectez-vous pour commenter.

Benjamin Thompson
Benjamin Thompson le 12 Août 2022
You may want to review this thread and use parfeval if the partial results option is something you want.


En savoir plus sur Parallel for-Loops (parfor) dans Help Center et File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by