BREAK in a PARFOR loop + processing time example to compare it with a FOR loop.

Hi,

BREAK usage is restricted in a PARFOR loop: in the PARFOR loop itself, it makes totally sense, just as CONTINUE. But in general, this is too exclusive. For instance, if inside my PARFOR loop I have a FOR loop that I am interested in breaking this inside loop at some point: Matlab "live debugger" is going to mark it in red complaining it is not allowed.

Toy example:

tic
Sset=randi(10000,1,10000);
matlabpool
parfor i=1:10000
    for j=1:length(Sset)
        if j==Sset(i)
            break
        end
    end
    Spos(i)=j;
end
matlabpool close
if sum(Sset==Spos)==10000
    disp('happy')
else
    disp('not happy')
end
toc

It will work, Matlab can compile it.

By the way, on my 2 processors machine, it takes in average about 40 sec to execute this simple toy example. While the usual double FOR loop program (below) is solved in one processor in about 0.3 sec !... :)

tic
Sset=randi(10000,1,10000);
%matlabpool
for i=1:10000
    for j=1:length(Sset)
        if j==Sset(i)
            break
        end
    end
    Spos(i)=j;
end
%matlabpool close
if sum(Sset==Spos)==10000
    disp('happy')
else
    disp('not happy')
end
toc

One can thus conclude that it takes quite some time for Matlab to connect the "labs" as they call them. Therefore it is not a good idea to close the MATLABPOOL just after the PARFOR loop if this loop is inside a loop for instance. Do you agree?

Even then, if I place the TIC TOC just around the PARFOR loop so that I don't take into account the time to create the "labs", the average processing time is 21 sec! Thus a PARFOR loop on 2 labs is 70 times longer than a FOR loop on 1 processor (for this toy example). Parallel computing is not magic :/

Answer 1

[Raymond Norris]

I wouldn't suggest starting and stopping a MATLAB Pool at random. Bring it up when you need it, bring it down when you're done or you need your compute resources for something else.

Regarding your benchmark, you're just not giving the parfor enough work to do. So you're being saturated by the communication overhead between labs. A parfor loop won't run an individual loop faster, it will run the entire set of iterations faster. So a good "toy" example would be to keep the time it takes to run the body of the code constant.

tic
parfor idx = 1:100
   pause(1)
end
toc

Answer 2

[Edric Ellis]

Further to what Raymond has pointed out, PARFOR doesn't support the concept of early return (other than through throwing an error). In the case where you're searching for some value, you might be better served using SPMD - where the labs operating in parallel can communicate to see if the stopping criterion has been reached.

spmd
  done = false;
  while ~done
    % each lab does some computation in parallel
    myDone = rand() > 0.99;
    % communicate to see if anyone has found the answer:
    % this ORs together the values of "myDone" from each lab
    done = gop(@or, myDone)
  end
end

EDIT: In R2013b, parfeval was introduced to Parallel Computing Toolbox. This was designed with early-return cases in mind. Something like this:

p = gcp();
for idx = 1:10
  f(idx) = parfeval(p,@rand,1);
end

for idx = 1:10
  % fetchNext blocks until next results are available.
  [completedIdx,value] = fetchNext(f);
  if value > 0.99
      break
  end
end
cancel(f); % Stop any other execution