Effacer les filtres
Effacer les filtres

MATLAB Parallel server: save output data of each worker by program independent job

8 vues (au cours des 30 derniers jours)
The program independent job can work on 10 workers like the following example.
It is assumed that all workers are diffrent nodes each other (node01~node10).
The following code is run on the worker of node01.
% % example.m
c = parcluster;
job1 = createJob(c);
num_worker = 10;
for cnt = 1:num_worker
input_var{cnt} = cnt;
end
createTask(job1,@test_func,1,input_var);
submit(job1);
wait(job1);
result = fetchOutputs(job1);
function output_var = test_func(input_var)
A = rand(100,100) + 1i*rand(100,100);
output_var = A*input_var;
I want to save the results from all workers.
If the system memory of computing the results is over the limit, we can't acquire and save these results.
So, I try to save each result inside the function, 'test_func.m', as follows.
function [] = test_func(input_var)
A = rand(100,100) + 1i*rand(100,100);
output_var = A*input_var;
save(sprintf('filename_%d.mat',input_var),'output_var');
If the function is changed as above, the result of each worker is saved in the following path,
'C:\ProgramData\MJS\Checkpoint\node(number)._worker01_mlworker_log\matlab\work'.
I want to save these files on the other path, not a default path,
like the path of node01, '\\node01\E:\folder_name\'.
The follows have error, not to find the path on the other worker except to headnode (node01).
function [] = test_func(input_var)
A = rand(100,100) + 1i*rand(100,100);
output_var = A*input_var;
save(sprintf('\\node01\E:\folder_name\filename_%d.mat',input_var),'output_var');
I wonder how to save output data of each worker on user-defined path of headnode.
Please, help me.
Thank you.

Réponse acceptée

Raymond Norris
Raymond Norris le 6 Mar 2023
Modifié(e) : Raymond Norris le 7 Mar 2023
Try the following
function [] = test_func(input_var)
% Store the local hostname for future runs
persistent HN
if isempty(HN)
[FAILED, HN] = system('hostname');
if FAILED
HN = getenv('COMPUTERNAME');
end
HN = strtrim(HN);
if isempty(HN)
error('Failed to get hostname')
end
end
A = rand(100,100) + 1i*rand(100,100);
output_var = A*input_var;
% Need to escape each of the backslashes.
% E:\ won't work. Use E$
save(sprintf('\\\\%s\\E$\\folder_name\\filename_%d.mat',HN,input_var),'output_var');
  6 commentaires
Raymond Norris
Raymond Norris le 7 Mar 2023
Just so that I understand, you have 10 workers running on 10 nodes. Each worker is to write a result file to the same machine (node01). This machine can be accessed via the E:\ drive. When you try the following
save(sprintf('\\node01\E:\folder_name\filename_%d.mat',input_var),'output_var');
MATLAB throws an error. What error does MATLAB throw? Does each worker throw the same error?
I don't believe this will work for several reasons.
  1. For starters, is "E" a mapped drive or a physical drive? If it's a physical drive, then \\node01\E: isn't proper syntax. I believe this should be \\node01\E$.
  2. If E is mapped drive, then you ought not need to specify \\node01 (since that's already apart of "E")
  3. But I don't think E:\folder_name\... will work because drive mappings are specific to a user, not a machine. Hence, you need to go back to the UNC path (\\node01\E$)
What happens when you try
save(sprintf('\\node01\E$\folder_name\filename_%d.mat',input_var),'output_var');

Connectez-vous pour commenter.

Plus de réponses (0)

Catégories

En savoir plus sur MATLAB Parallel Server dans Help Center et File Exchange

Produits


Version

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by