How can I speed-up 3 functions without terminating workers?

I have four different functions in which each function runs three superimposed for-loops with 3D arrays, (3 are in dependent and 1 is dependent on those 3 independent functions)

for i=1:nx
    for j=1:ny
        for k=1:nz
            statement or my calculations 
        end
    end
end

All of these loops are just a part of two while loops: 1) inner while loop, and 2) outer while loop. The previous two loops are dependent loops, which means i can't use parallel computing functions for them.

What is the best way to run them all in parallel with the minimum computational cost as this process is a very time consuming and computationally expensive process. I'm wondering, if i can run each function on a worker without terminating the worker everytime i enter the next iteration for the inner while loop!! as terminating and executing MATLAB worker takes few seconds, like 30-60 seconds, which means every inner while loop might take more than 2 minutes to be calculated and it is not the case which i'm looking for. I'm just trying to speed-up the calculations which takes about 60-90 seconds for each inner loop, if you know what i mean.

Answer 1

[Thomas Falch]

If you use a parpool you can start up the MATLAB workers once, and then use them to execute e.g. parfor loops, or execute functions in parallel with parfeval, without having to pay for the cost of starting up MATLAB.

For example:

% This starts up 4 MATLAB workers and will take 10s of seconds
parpool('local', 4) 

% Run the function pause(1) on a worker, the worker is already running, this wil take 1 second
wait(parfeval(@() pause(1), 0));

The pool remains open (with the workers running idle in the background) until you close it.

In your case, you may also consider using parfor loops. These can often be used to parallelize for loops with parpools (with some restrictions)

[ahmed gamal]

Thank you. I will try your suggestions but may i ask what ate those restrictions which u mentioned later in ur answer?

[Thomas Falch]

Essentially, parfor loops require that the iterations are independent, that is, no iteration should depend on the result of another iteration (if it did, those two iterations could not be executed in parallel at the same time).

You can find more information here: https://uk.mathworks.com/help/parallel-computing/decide-when-to-use-parfor.html

If you have the Parallel Computing Toolbox installed, you can just replace for with parfor, if the iterations are not independent, it will error.