Help with PARFOR in a very simple LOOP

parfor k = 1:n
  mpc = loadcase(filename); % Loads file case into varaible mpc
  mpc.gen(:,2) = ProbMatrix.Prob(:,k);
  PercentageOfLoadDistAndPF(:,3)=A(k).*PercentageOfLoadDistAndPF(:,1); 
  PercentageOfLoadDistAndPF(:,4) = PercentageOfLoadDistAndPF(:,3)./PercentageOfLoadDistAndPF(:,2); 
  PercentageOfLoadDistAndPF(:,5)=sqrt((PercentageOfLoadDistAndPF(:,4).^2)-(PercentageOfLoadDistAndPF(:,3).^2)); 
  mpc.bus(:,3) = PercentageOfLoadDistAndPF(:,3); 
  mpc.bus(:,4) = PercentageOfLoadDistAndPF(:,5); 
  results = runpf(mpc, mpopt);
  PFResults(k).Results = results;
end

Doing a probabilistic load flow problem, in where each load flow had a different generator dispatch and load demand. With parfor, I get the following error:

The variable PercentageOfLoadDistAndPF in a parfor cannot be classified.

Quick explanation:

• n equals usually to 2000+, taking 100+ seconds.
• mpc changes every loop from the same file (file NEVER changes).
• mpc.gen(:,2) is loaded with whatever is in ProbMatrix.Prob(:,k) (ProbMatrix.Prob(:,k) NEVER changes)
• PercentageOfLoadDistAndPF is originally a (167,2) matrix (this part never changes) and this loop adds 3 additional columns next to the original 2 with data that depends on the first 2 columns + a A(k) vector (this changes every iteration)
• mpc.bus(:,3) is then loaded with whatever came up in newly made column in PercentageOfLoadDistAndPF(:,3)
• same goes for mpc.bus(:,4) except it uses PercentageOfLoadDistAndPF(:,4) instead of the (:,3)
• mpc then goes into PF program (runpf) and whatever it spits out, it is saved within a struct.Final return is the 1x2000+ struct.

Every iteration of the loop seems to me to be independent of each other...

I apologize in advance for my archaic and non-efficient coding.

[Adam]

It's a while since I have done analysis of parfor loops and the various errors they give, but I'm pretty sure that having a variable that changes size in the parfor loop is not going to work (or even if it does it is not a good idea).

Try writing the extra columns to a new variable and then concatenate them back into the original after the loop.

Edric Ellis or some other expert on parallel computing can probably give a better answer for you though, that is just a quick observation for a start.

[Chilean]

The end results of that whole variable are of no importance at the end, so just like you stated I tried zeroing the extra columns before the loop in another new variable... but I still got the same error, except this time blaming the new variable.

[Adam]

I think also parfor needs the first index of an array to be the one varying in the loop, but I may be wrong on that.

Answer 1

[Walter Roberson]

When you modify a matrix created outside of the parfor, every time you index it the indices must be the same. Your code uses indices (:,1) through (:,4) for a variable that is input to the loop and being changed by the loop. That is not allowed.

Consider that in parfor loop (say) k=7, you might modify PercentageOfLoadDistAndPF(:,3) and then by chance the parfor loop running (say) k=56 might modify the same PercentageOfLoadDistAndPF(:,3) [because the variable is not local to the loop] and then in the next line of the k=7 when you read back PercentageOfLoadDistAndPF(:,3) you would get the k=56 version.

If the extra columns are intended to be independent of the other loop indices, then make the calculations into a matrix that is unused outside the loop.

PercentageOfLoadDistAndPF_1 = PercentageOfLoadDistAndPF(:,1);
PercentageOfLoadDistAndPF_2 = PercentageOfLoadDistAndPF(:,2);
parfor k = 1:n
  mpc = loadcase(filename); % Loads file case into varaible mpc
  mpc.gen(:,2) = ProbMatrix.Prob(:,k);
  PercentageOfLoadDistAndPF_3 = A(k).*PercentageOfLoadDistAndPF_1; 
  PercentageOfLoadDistAndPF_4 = PercentageOfLoadDistAndPF_3 ./ PercentageOfLoadDistAndPF_2; 
  PercentageOfLoadDistAndPF_5 = sqrt((PercentageOfLoadDistAndPF_4.^2) - (PercentageOfLoadDistAndPF_3.^2)); 
  mpc.bus(:,3) = PercentageOfLoadDistAndPF_3; 
  mpc.bus(:,4) = PercentageOfLoadDistAndPF_5; 
  results = runpf(mpc, mpopt);
  PFResults(k).Results = results;
end

You could make the PercentageOfLoadDistAndPF_3, PercentageOfLoadDistAndPF_4, PercentageOfLoadDistAndPF_5 into columns of the same array if you wanted, but there is not a lot of benefit of doing so in this situation.

[Chilean]

Wow, this did it. So many thanks.

Without parfor:

Total Iteration Count: 10089.00 Elapsed time is 110.117543 seconds.

With parfor (only 4 workers):

Total Iteration Count: 10089.00 Elapsed time is 49.094014 seconds.

I have to keep adding some stuff into this loop, hopefully I can do it in such a way as to keep the ability to make it parallelable.

So, to avoid my original mistake, shouldn't I be able to use my original PercentageOfLoadDistAndPF(:,1) and PercentageOfLoadDistAndPF(:,2) to create the newly inside-loop-initialized PercentageOfLoadDistAndPF_3, PercentageOfLoadDistAndPF_4, PercentageOfLoadDistAndPF_5?

EDIT: Yes. It works too.