Effacer les filtres
Effacer les filtres

All workers aborted during execution of parfor loop

33 vues (au cours des 30 derniers jours)
Birjit
Birjit le 12 Juin 2023
" Error using distcomp.remoteparfor/rebuildParforController
All workers aborted during execution of the parfor loop.
Error in distcomp.remoteparfor/handleIntervalErrorResult (line 259)
obj.rebuildParforController();
Error in distcomp.remoteparfor/getCompleteIntervals (line 396)
[r, err] = obj.handleIntervalErrorResult(r);
Error in MTD_Finalinv (line 655)
parfor i=16:numberofplaintexts
The client lost connection to worker 1. This might be due to network problems,
or the interactive communicating job might have errored. "
This is the error when I use 1 lakh csv files for a parallel computation involving code. But the same code runs properly for 50000 csv files without any error. Anyone knows the why it is so???

Réponses (1)

Raymond Norris
Raymond Norris le 15 Juin 2023
When I see
The client lost connection to worker 1. This might be due to network problems, or the interactive communicating job might have errored
I'm inclined to think it's because one of the workers crashed (out of memory). One suggestion I have is to use ticBytes/tocBytes to see how much data is being passed in/out of the workers. However, this won't tell you how much is being consumed in the parfor loop. To get a sense of how much is being consumed, try the following.
Refactor your parfor so that is calls a single function, as such
function birjit
ticBytes(gcp)
parfor idx = 16:numberofplaintexts
unit_of_work()
end
tocBytes(gcp)
end
function unit_of_work
A = rand(10);
whos
end
Because of transparency rules, you need to call whos within a function, not directly in parfor.
Another consideration is using parforOptions
function birjit2
opts = parforOptions(parcluster,'RangePartitionMethod','fixed','SubrangeSize',N);
parfor (idx = 16:numberofplaintexts,opts)
...
end
Here, were are running a job array of (numberofplaintexts-16)/N jobs. This way each worker might not run out of memory.

Catégories

En savoir plus sur MATLAB Parallel Server dans Help Center et File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by