Call parfeval using global variables?

23 vues (au cours des 30 derniers jours)
Kurt
Kurt le 23 Jan 2023
Modifié(e) : Kurt le 9 Fév 2023
I am using parfeval() to do multithread logic. I have 16 cores. The documention mentions listing the calling parameters as arguments in parfeval, for example:
for ii = 1:partition
f(ii) = parfeval(@func, 1, a, b, c);
end
My three a b c arguments happen to be globals. Can I call parfeval this way instead, and pick up the globals in func? The return value is also a global.
for ii = 1:partition
f(ii) = parfeval(@func,0);
end
I suspect not, since my code isn't working.
Also, one of the three arguments is a pathname to a DOS batch file, which I execute with a System call. Is this possible across multiple cores?

Réponse acceptée

Kurt
Kurt le 3 Fév 2023
Using a combination of spmd and spmdIndex (NOT labIndex, which is obsolete), I was able to achieve what I wanted: setting the DOS path environment for each worker individually. You can also write messages from each worker to a shared log file:
parpool(4);
arraystuff = ["Users","Home","Lib","exe");
execution_path = 'C:/Users/kurt/matlab/batch_files';
run_path = strcat(execution_path, '/run_cases.bat');
logfile = 'C:/Users/kurt/logfile.txt';
spmd
switch spmdIndex
case 1
if(ispc) % running on a PC?
setenv('input_path', arrayStuff(1,spmdIndex)); % set the DOS environment
end
system(run_path, '-echo'); % do some work on this core
msg = fopen(logfile, 'a'); % open diagnostic log
fprintf(msg, '%s %s %s %s', datetime('now'), ' worker ', num2str(spmdIndex), ' completed');
fclose(msg);
case 2
if(ispc) % running on a PC?
setenv('input_path', arrayStuff(1,spmdIndex)); % set the DOS environment
end
system(run_path, '-echo'); % do some work on this core
msg = fopen(logfile, 'a'); % open diagnostic log
fprintf(msg, '%s %s %s %s', datetime('now'), ' worker ', num2str(spmdIndex), ' completed');
fclose(msg);
case 3
% etc
case 4
%etc
end
end
The 'system', 'fprintf' and 'setenv' functions are not necessary, but illustrate some capabilities for working with remote cores. Printing to a log file is one way to debug multithreaded code, since you can't set breakpoints. Note that system calls only work in Process pools, not Thread pools.
  1 commentaire
Kurt
Kurt le 9 Fév 2023
Modifié(e) : Kurt le 9 Fév 2023
Another useful technique:
If a worker process crashes, it is well-nigh impossible to return an error message to the main CPU unless you pass it to a log file as described above. You can encapsulate each worker code block in a try-catch-end block, and write the error message to your log file as follows:
case 14 % worker number 14
try
do_some_stuff(arg1, arg2...)
catch ME
msg = fopen(logFile, 'a');
fprintf(msg, '%s', ME.identifier);
fclose(msg);
end
case 15...

Connectez-vous pour commenter.

Plus de réponses (1)

Walter Roberson
Walter Roberson le 23 Jan 2023
When you pass a global as a parameter, what is received in the called routine is treated as a local. For example, assigning to that parameter does not change the global variable.
This is the case when using parfeval as well. The global a, b, c in your call would be received as local.
To emphasize: if you had
function result = func(a, b, c)
result = some_internal_function;
end
function result = some_internal_function
global a b c
result = a*25 + b*5 + c;
end
then the fact that you passed global variable a into func does not mean that a gets treated as global inside some_internal_function. You would have to code something like
function result = func(a_in, b_in, c_in)
global a b c
a = a_in; b = b_in; c = c_in;
result = some_internal_function;
end
and then a b c would become global within that worker for as long as the worker continues to live. But if some_internal_function modified (say) b then the change would not affect any other worker and would not affect the client.
  10 commentaires
Walter Roberson
Walter Roberson le 1 Fév 2023
What difficulty did you encounter with addAttachedFiles
Kurt
Kurt le 1 Fév 2023
Modifié(e) : Kurt le 1 Fév 2023
Walter, I did not pursue that path. I was able to get the code to work as described in the previous comment without having to worry about all the issues I had questions about.
However, I still have one step to complete: Currently all 16 cores are running the identical software and producing the identical results. What I want to happen is for each core to work on a different set of input and output files, simultaneously. I'm not sure how to do that. I need some kind of loop within the spmd block to modify the pathnames for each core, something like this:
spmd
for each core % not all simultaneously:
setenv('input_path', input_pathname);
setenv('output_path', output_pathname);
system(execute_pathname, '-echo');
end
end
(later): I saw your post on labindex at
Maybe that will work?

Connectez-vous pour commenter.

Catégories

En savoir plus sur Debugging and Analysis dans Help Center et File Exchange

Produits


Version

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by