Parfor Loop Memory Issues

Hello,

My code runs on a parfor loop which runs with no issues for most cases.

The issue is, I am intending to execute this loop in some cases for vast amounts of data, the loop has to run for 7,805,662 iterations, to give an idea. The parfor loop then executes a significant number of calculations, which are all stored in arrays that are then post-processed after the parfor loop.

When executing this super large iterations, I am finding that my Matlab "freezes", and if I try to pause the run Matlab becomes unresponsive and has to be closed. No error is dropped whatsoever, which does not help me to try identify the problem. I have then done a few tests. The parfor loop runs perfectly fine when the number of iterations is not so large, and also I have found that if I run on a standard, serial "for" loop instead of parfor, it runs with no problems, but it takes way too much time and is unfeasible.

This is then narrowing the problem down to the parfor loop, and makes me think that the workers might be becoming "overwhelmed" and causes the run to stall. I am trying to find a way of partitioning the parfor loop so that this doesn't happen using the parforOptions, but I am struggling to use this effectively. Just to start, I would like to understand how Matlab partitions the loop by default, when no options are set. Is anyone able to answer that? What might be a reasonable number to partition the loop?

My machine has capacity for 8 Workers, 128GB Ram, 64-bit, and an Intel(R) Xeon W-2145 CPU @ 3.7GHz.

Many thanks for your help, really appreciate it.

Carlos

[Mohammad Sami]

Without specific details of what you are doing, it's a bit challenging to see how you might solve the problem.

One idea is perhaps use tall arrays if your data does not fit into memory. This will work if your calculations can be vectorised.

[Tim Johns]

Hi Carlos,

My guess is that it runs in serial ok because you have 1 worker with 128 - 6 (ish, for OS) = 122 GB RAM available. When you divide that up by 8 workers, you only have 15 GB. If your calculations are loading/generating a lot of data, you could be running out of RAM and so the OS has to start swapping data on and off the hard disk - this is very slow.

Try running in serial and record what the peak RAM usage is, round up, and then divide the total available RAM by that to get the number of workers.

If this doesn't work, please file a help ticket and someone will be able to work with you more closely.

Tim