PARFOR in real applications

I've installed the Parallel Computing Toolbox for some experiments with my code. To my surprise none of the codes run faster with PARFOR compared to sequential FOR loops.

Examples:

1. https://www.mathworks.com/matlabcentral/answers/762196-how-can-i-efficiently-add-multiple-arrays-generated-in-a-loop
2. Another example was a simple loop calling the external lame.exe function:

lamebin  = 'C:\Program_\lame3.99.5\lame.exe';
switches = ' -m j -h -V 1 -q 2 --vbr-new --nohist';

WavFiles = dir(fullfile(Folder, '*.wav'));
parfor iWav = 1:nFile
    aFile          = fullfile(Folder, WavFiles(iWav).name);
    [aPath, aFile] = fileparts(aFile);
    aMP3           = fullfile(aPath, [aFile, '.mp3']);
    [s, w]         = dos([lamebin, switches, '"', aFile, '" "', aMP3, '"']);
 end

Both examples take about the double time than a FOR loop on my 2 core CPU, but there is no acceleration on the 4 core also. The RAM is not exhausted in both cases.

Questions:

• Are there obvious mistakes in my naive approachs?
• How do you use PARFOR in your applications to accelerates the processing on a pool with 2 or 4 local workers? I know the examples from the documentation, but I was not successful yet to implement it in my codes.

[Mario Malic]

Hi Jan,

At least on Windows, I have no problems with parfor when I run this. If your program creates some temporary files, it would be also good to consider WorkingDirectory within StartInfo so workers and programs do not cause some issues with each other.

exeProcess = System.Diagnostics.Process;
exeProcess.StartInfo.Arguments = '-example arguments';
exeProcess.StartInfo.FileName = lamebin % full file path
exeProcess.Start();

Does your program utilize all cores already with its single instance?

[Jan]

Thanks Mario. My test examples do not use mutliple cores. Otherwise the parallelization cannot be expected to be faster.

Using .NET or the equivalent Java method to call an external program seems to be a reliable solution without too much overhead:

runtime = java.lang.Runtime.getRuntime();
proc    = runtime.exec(cmd);
while true
     try
         exitCode = proc.exitValue();
     catch
         % not ready
         pause(0.2);
     end
end

But actually I'd expected PARFOR to work also efficiently.

[Mario Malic]

Approximately, how long does take for one run of your program?

[Edric Ellis]

I presume when you're trying with Java you are running multiple "lame" processes simultaneously, and seeing a sensible speed-up? (I was going to speculate that perhaps disk access was limiting performance, but if you're able to get expected performance running multiple processes a different way, then that would seem unlikely).

Answer 1

[Jan]

After trying many examples from many questions in the forum, I've found:

https://www.mathworks.com/matlabcentral/answers/1720175-perform-specific-operations-on-arrays-without-for-loop

This runs 25% with parfor on my weak i5 mobile with 2 cores.

[TO BE EXPANDED] I'm going to add further examples here...