How to input global variables in function that is in parloop?

Hi.

I want to call some data ( = matrix) in function in parloop.

First time, I used global variable, but it is not worked in parloop.

How can I implement the code below without global variable?

The size of 'data1' is so large, so individually calling the data in function takes so much computing resources.

load('...\data1')
data1 = data1;

global data1
x = nchoosek([1:10], 2);

parfor i = 1:10
    y(i) = myfunc(x(i, :));
end

% Function
function y = myfunc(x)
    global data1
    y = data1(x);
end

Answer 1

[Raymond Norris]

We can agree that if global variables were supported, you wouldn't modify data1 and have that reflected on the client -- that would violate being task independent.

Therefore, you ought to create/load the data on the worker. To avoid loading it each time, you could make the variable persistent, such as:

x = nchoosek(1:10, 2);
parfor i = 1:10
    y(i) = myfunc(x(i, :));
end

% Function
function y = myfunc(x)
persistent data1
if isempty(data1)
    data1 = load("data1");
end
y = data1(x);
end

Of course, the caveat is that wherever the workers are running, they need access to data1.mat If you think this workflow would work, I'm guessing if it's large, this is probably running on a cluster, correct? I can sketch out an example of how the workers can find the data on a cluster (it would be a requirement that MAT-file is on the cluster).

[Jungeon Kim]

Thanks a lot!

It works well!

For answer your question, 'no'. I'm using workstation, so single computer is running.

Thank you so much for the consideration.

Answer 2

[Edric Ellis]

Another option is to use parallel.pool.Constant to arrange for the data to be loaded and then accessed. This can be a bit simpler than writing your own function using persistent data (behind the scenes it is basically the same thing though). Using the "function handle" constructor avoids transferring the data, and gets each worker to call load for itself. Here's a simple example:

parpool("local");

Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 2).

% This instructs each worker to load variable X from the file durer.mat and
% retain the value.
c = parallel.pool.Constant(@() load("durer.mat", "X"));
parfor i = 1:100
    % Here's how you access the value - use "c.Value". The return from the
    % "load" function is a struct, so we need to get the field "X" out of
    % that struct
    data = c.Value.X;
    % Now we can operate on the data.
    y(i) = sum(data(:,i));
end