Running anonymous functions in parallel avoiding comunication overhead

Hi,

I want to run a set of anonymous function with different input data.

My code strucutre looks like this:

    hugeData = loadData(); 
    nJobsW = 10;

    obj =Composite();
    for w = 1:numel(obj)
        objW = cell(nJobsW,1);
        for i = 1:nJobsW
            funData = importFunData(w,i);
            objW{i} = @(x) fun(x,workerData,hugeData);
        end
        obj{w} = objW;
    end

    result = Composite();

    condition = true;
    while condition

        spmd
            x = getX(result);
            result = cell(nJobsW,1);

            for i = 1:nJobsW
                result{k} = obj{i}(x);
            end

        end

        condition = testCondition(result);

    end

So as you can see, there is a large constant data, and data per worker-job. This code is consuming a lot of memory, much more than if executed sequentially.

I suspect that hugedata is being copied many times.

My questions are: If all workers are local (same machine), is hugeData being copied? Can I avoid copying hugeData? How do I check what is consuming memory? How do I minimize the overall data overhead?

Thanks

Answer 1

[Walter Roberson]

See http://www.mathworks.co.uk/matlabcentral/answers/21478-slow-execution-of-parfor-loops-due-to-communication-overhead-load-static-data-into-worker-workspace