Just a Beginner's 'parfor' confusion

I'm trying to execute this code and constantly getting the message that 'variable jVals is indexed but not sliced'. Can anyone kindly help

iVal=5:0.2:5.4;
jVal=2:0.5:2.5;
iLen=length(iVal)
jLen=length(jVal);
matrix=zeros(iLen,jLen);
 parfor i=1:iLen
     dummy=zeros(1,jLen);
     for j=1:jLen
         dummy(j)=jVal(j);  %/// This line is in ERROR
     end
     matrix(i,:)=dummy;
 end

Answer 1

[Matt J]

In your case, it will go away if you modify the loop as

   parfor i=1:iLen
       matrix(i,:)=jVal;
   end

It's not an error, but rather a warning that you might not have efficient code. MATLAB sees that you are indexing a matrix variable inside the loop using another variable j. This usually means the matrix variable is a large array. If it were a small array, you would usually just index it with known, fixed constants, like jVal(1). If it is a large array, you usually want to find a way to have parfor send to the worker only a slice of the array that the worker will use, instead of broadcasting the whole array to every worker.

[Matt J]

More relevant reading on this here

http://www.mathworks.com/help/distcomp/advanced-topics.html#bq_of7_-1

[Amnah]

Thanks Matt.. it works that way :) And you're right. jVal is a very large array. With two nested for loops it works fine but with parfor, one of the arrays gets sliced and the second index variable array gets broadcasted. Is it necessary to slice both arrays?

[Matt J]

If you're only using a small part of the broadcasted array on any given lab, it makes sense to try to slice it, so that you don't have so much data copying/broadcasting.

There are also ways to create build large data on the labs, rather than broadcasting them, and to make the data persistent there between parfor loops, see

http://www.mathworks.com/matlabcentral/fileexchange/31972-worker-object-wrapper

[Amnah]

pardon.. I need the whole broadcasted array for every slice of my sliced array. Let me share my code:

rvals = [0.0002 0.0006 0.0010 0.0014 0.0018 0.0022 0.0026 0.003]; 

R = length(rvals); 
A = zeros(n,m,R); 

[yIndex xIndex] = find(E); % E is an edge map of an image numEdg=length(xIndex);     % xIndex, yIndex are edge locations 

parfor cnt = 1:numEdg      % length(xIndex) is around 1800 

    yIn=yIndex(cnt);  
    xIn=xIndex(cnt);

   for r=1:R  

    rv=rvals(r); 

      for x0 = 1:m 

          y0 = round(yIn-rv*(xIn-x0)^2); 
          if y0 < n && y0 >= 1 
          tmp = zeros(n,m,R); 
          tmp(y0,x0,r) = 1; 
          A = A + tmp; 

      end 

    end 

   end 

end

Now here rvals is the broadcasted array. And I can see that workers have to go back to client again and again to retrieve every value of rvals, which is producing a huge communication overhead. Can you help with this.

[Matt J]

And I can see that workers have to go back to client again

What do you mean by you can "see" it? How are you viewing the communication activity of the workers?

Since rvals is only length-10, I can't imagine it's creating a lot of communication overhead, but you can see if it makes a difference to copy rvals to a temporary variable

 parfor cnt = 1:numEdg      % length(xIndex) is around 1800 

     ...

   tmp_rvals=rvals;

    for r=1:R  

     rv=tmp_rvals(r); 

      for x0 = 1:m 

    ...

[Amnah]

Hmm.. When I use for-loop, it takes around 70 seconds to give result. And when I use parfor, it takes around 35 minutes..

[Amnah]

Now it is not saying that rvals is not sliced.. but it is still taking a lot of time in parallel processing as compared to sequential processing.. probably my code is inefficient or I don't know.. Thanks anyway Matt :)

[Matt J]

Are you running inside a script or a function? Try both.

Answer 2

[Arslan Ahmad]

It is better not to use another index inside the "parfor" use only the index of parfor in your case it's better to use i instead of j for more information I would like to refer you to MATLAB parfor documentation for more information and so for your can try this code for solving your problem. I modified it for you, it tooks only 0.194221 seconds on two matlab workers. If you find it useful than please accept the answer.

 tic
iVal=5:0.2:5.4;
jVal=2:0.5:2.5;
iLen=length(iVal)
jLen=length(jVal);
matrix=zeros(iLen,jLen);
 parfor i=1:iLen
jVal=2:0.5:2.5;
jLen=length(jVal);
     dummy=zeros(1,jLen);
      j=1:jLen
         dummy(j)=jVal(j);  %/// This line is in ERROR
     matrix(i,:)=dummy;
 end