Making sliced variables persist between multiple PARFOR loops

I have a sliced variable which is to be processed over a sequence of parfor loops, each time with the same slicing pattern. I am wondering if there is a way to avoid repeating the transmission to the workers and the data slicing process multiple times. Basically, I'm looking for something like parallel.pool.Constant, but where the constant data persists in sliced form.

In the past, I have only been able to accomplish this kind of thing with an spmd block and a for-drange loop. But these are supposedly less efficient than parfor, so there is a performance penalty. It seems like a natural kind of thing that one would want to do, however. Is there any parfor-efficient alternative?

Answer 1

[Edric Ellis]

Hm, interesting problem - unfortunately, there isn't really any sensible way to use parallel.pool.Constant here while still using parfor. You could imagine building a Constant with unique values on each worker - but you wouldn't then be able to arrange for the parfor loop iterations to cover the slices on each worker correctly.

In this case, I would be tempted to investigate spmd and for-drange. This main reason for-drange can end up less efficient than parfor is where you have an imbalance in execution times between the loop iterations ( parfor knows how to do dynamic load-balancing, for-drange does not). If your loop iterations are mostly uniform in execution time, then for-drange should be fine, and you can use Composite or even codistributed to set up the "sliced"-style data.

[Matt J]

OK, I'll try. Thanks.