SLURM and tailoring walltime for different jobs

3 vues (au cours des 30 derniers jours)
Maria
Maria le 20 Déc 2021
Hi,
so finally, I have access to a big cluster that uses SLURM as scheduler for Matlab. So far so good.
Now, I would need to understand if I am planning the execution of my program properly. I have a Main file, with several batch jobs. At the moment, it looks something like this:
c = parcluster('Cluster name');
c.AdditionalProperties.AccountName = 'my account name';
c.AdditionalProperties.WallTime = '00:30:00';
N_big = 32;%c.NumWorkers;
N_small = 32;
LAMBDA_SCALING_FACTOR = 5e12;%100e9;
job_geometry_setup = batch(c,@rectangular_loop_geometria_setup,2,{},'Pool',N_small-1,'CurrentFolder','.','AutoAddClientPath',false);
wait(job_geometry_setup)
result_geometry_setup = fetchOutputs(job_geometry_setup);
Regioni = result_geometry_setup{1};
induttanze = result_geometry_setup{2};
clear result_geometry_setup
delete(job_geometry_setup)
job_mesh_sup = batch(c,@genera_mesh_sup_new,7,{Regioni, induttanze,time,LAMBDA_SCALING_FACTOR},'Pool',N_small-1,'CurrentFolder','.','AutoAddClientPath',false,'AttachedFiles',{'discr_psp_nono_3D_vol_sup_save_noglobal_mex.mexa64' 'genera_nodi_interni_rev.p' 'interpolating_vectors_rev.p'});
wait(job_mesh_sup)
result_mesh_sup = fetchOutputs(job_mesh_sup);
Regioni = result_mesh_sup{1};
induttanze = result_mesh_sup{2};
A_th = result_mesh_sup{3};
lambda = result_mesh_sup{4};
nodi = result_mesh_sup{5};
A = result_mesh_sup{6};
A_obj = result_mesh_sup{7};
clear result_mesh_sup
delete(job_mesh_sup)
So, I have a couple of questions:
  • in the way I am using now the batch, I continuously need to fetch the output from a batch job to the client, just to feed them to another batch job, as in the example above. I would not need to save all the variables. Is there a way to not fetch the output in the client, and to use what is directly available in the shared memory?
  • each job can have a different walltime. Shall I just redefine c.AdditionalProperties.WallTime = '00:30:00'; before each job to a value that I find reasonable?
Best

Réponses (0)

Catégories

En savoir plus sur Third-Party Cluster Configuration dans Help Center et File Exchange

Produits


Version

R2020a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by