bug parfor: analyzing and transferring files to worker: not done: blocked

Hello there,

I am a running a working script with parfor on matlab, after a while (~1200 iteration on parfor), matlab block without appareant reasons while producing the following message: "analyzing and transferring files to workers...". I am running the parfor on matlab 2023a, on local parfor only. I did validate the cluster of parfor before. The code is working correctly as the 1200 files were done. I tried with other very similar files and it did block also similarly. There is largely enough memory space on my PC. I tried to restart matlab, and also my PC, but this did not solve the problem. I also tried to find solution online, unsuccesfully. Anyone would have an idea ? many thanks for your attention.

[Ganesh]

Hi @student student,

In order for us to reproduce this issue, it would be a helpful if you share your code files.

I would also request you to share the stack trace of the error you are encountering.

[student student]

Thanks Ganesh,

The code is working, so there is no error message.

The code is very classic, and as following:

---------------------------------------------------------------------------------------------------

a=1:1:10000;
parfor amF=1:length(a)
    functAA(a,amF)
end

---------------------------------------------------------------------------------------------------

So, I guess it is more low level, matlab bug on parfor

thanks

Answer 1

[Edric Ellis]

The "analyzing and transferring files to workers" message occurs only when the workers encounter a failure to find some code that they need. If you're using the "local" cluster type, then you should basically never see this. You can diagnose what got transferred by using listAutoAttachedFiles after your parfor loop, like this:

parfor i = 1:n
    doStuff()
end
% Assuming "analyzing and transferring..." message has occurred
listAutoAttachedFiles(gcp())

[student student]

Many thanks for your help. I did what you recommended. The function, which works fine without using parfor, does not work anymore and throw me the error: " an undefined function error was thrown on the workers for "finChanH". this is because "finChanH" is not accesible for the workers. Which is very strange given that this variable is fully created in the function and not fet to the function. therefore the code listAutoAttachedFiles did not get me any thing yet. Any idea how a working function with classic for is not working with parfor ?

thanks for your help, i am really stucked here

[Steven Lord]

Without seeing the body of your functAA function, I suspect it's probably going to be difficult or impossible to offer any specific suggestions.

[student student]

thank you for your responses. what I suspect here is that the parfor display: "analyzing and transferring files to workers..." as it cannot find some unproduced data into the loop: so it should stop and report the problem. but instead it display this message. thanks

Answer 2

[Felix]

Ok. This is a bit late but hope it still helps. Also this may help people like me who encountered the same problem.

The behavior of stucking on 'analyzing and transferring files to worker ...' is most likely caused by error thrown by whatever functions called within the parfor loop somewhere in the middle of the parfor iterations.

When this happens, for reasons, instead of actually reporting the error, MATLAB will stuck on the message 'analyzing and transferring files to worker ...'.

The fix of this problem is obviously to resolve the error within the parfor loop. To see the error message, the workaround for MATLAB 2024a/2023b (I didn`t check other releases) is using thread based parpool, i.e., pool = parpool('Threads') before the parpool.

I hope this helps.