parfor question with 40 nodes

Hello! I have a script which runs in 20 hours

SNR=2:0.25:5

for i=1:length(SNR)
    ...
    for j=1:3000

          decodeFUNC(...)
      end
  end

The decodeFunc code is:

function[..]=decodeFunc(..)

for iters=1:100

      for ii=1:10000
          ...
      end
      ...
      for jj=1:5000
          ....
      end

end

I am user of a grid system with 40 nodes with 4 proccesors per node. Which is the most efficient way to use it? I am doing matlabpool 150 and i use parfor to this:

parfor j=1:3000

            decodeFUNC(...)
end

Is this correct? With createTask will i have better results? Thanks

Answer 1

[Edric Ellis]

If you manually divide your problem and submit many independent tasks, you will probably see a slight improvement in the amount of cluster time you use. PARFOR automatically tries to perform load-balancing, but inevitably there is some time wasted on the workers. Using lots of independent tasks, each worker will be returned to the cluster immediately once it has finished working on its portion of the problem.

[freebil]

Thanks for the answer. I have to do 150 tasks?

[Edric Ellis]

Each task uses only a single processor, so to get your job to run in the minimum amount of time, you need at least as many tasks as you have processors. Precisely how you divide the work depends on how much of the work is not in parallel (i.e. how much work is duplicated - the work that would be done ahead of the PARFOR loop).