Data extraction after parallel computing

2 views (last 30 days)
Giovanni Gardan
Giovanni Gardan on 6 Oct 2022
Edited: Chris on 6 Oct 2022
I have three functions (fun1, fun2, fun3) to run in parallel (parfor command). Each of this function has three array input (array1,array2, array3), and has three output arrays (Out1,Out2,Out3).
My question is: how can I extract each of the output array of each function after running the parfor command?
The starting code is the following, but I cannot extract the output data for each function.
funcList = {@fun1, @fun2, @fun3};
dataList = {array1,array2, array3; array1,array2, array3; array1,array2, array3}; %# or pass file names
parfor idx = 1 : length(funcList)
result{idx} = funcList{idx}(dataList{idx,:});
Thank you in advance

Answers (1)

Chris on 6 Oct 2022
Edited: Chris on 6 Oct 2022
To debug a parfor loop, first remove the par and try to run it as a normal loop.
It's possible something in your funcList is not capable of operating across three arrays at once.
If you are trying to apply each function to each array separately, you need 9 outputs for the code above. In that case, something like the following might work:
funcList = {@(x) sum(x,'all'), @(x) prod(x,'all'), @(x) std(x,[],'all')}
funcList = 1×3 cell array
{@(x)sum(x,'all')} {@(x)prod(x,'all')} {@(x)std(x,[],'all')}
dataList = {magic(3),magic(4),magic(5)}
dataList = 1×3 cell array
{3×3 double} {4×4 double} {5×5 double}
% Precalculate/preallocate things where it makes sense
nd = numel(funcList)*numel(dataList);
arrayidx = rem((1:nd)-1,numel(funcList))+1
arrayidx = 1×9
1 2 3 1 2 3 1 2 3
funidx = floor(((1:nd)-1)./numel(funcList))+1
funidx = 1×9
1 1 1 2 2 2 3 3 3
results = cell(numel(funcList),numel(dataList));
parfor idx = 1:nd
results{idx} = funcList{funidx(idx)}(dataList{arrayidx(idx)});
Starting parallel pool (parpool) using the 'Processes' profile ... Connected to the parallel pool (number of workers: 2).
results = 3×3 cell array
{[ 45]} {[ 362880]} {[2.7386]} {[136]} {[2.0923e+13]} {[4.7610]} {[325]} {[1.5511e+25]} {[7.3598]}
For more potentially useful information, read decide when to use parfor.
  1 Comment
Chris on 6 Oct 2022
I would caution you against accessing files on disk in a parfor loop, though. That may not be possible, as a hard drive can't read multiple locations at once.

Sign in to comment.


Find more on Loops and Conditional Statements in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by