Loop through files of remote server
4 vues (au cours des 30 derniers jours)
Afficher commentaires plus anciens
I'm using matlab to perform simulations on a cluster, where the jobs are submitted from a local machine through matlab to the cluster.
A sample submission script might look like this:
%%Get handle to the job scheduler
sched = findResource();
%%Create a job
job = createJob(sched, 'FileDependencies', {'Analysis.m'});
%%Create the tasks
filelist=dir('/dir1/dir2/')
for tidx = 1:length(filelist)
tasks(tidx) = createTask(job,@Analysis, 1, {tidx});
end
%%Submit the job
submit(job)
I'm now trying to obtain and loop through some files on the cluster and run a script on the files, say, Analysis.m
How would I do this to get a file list on the cluster and not on the local machine from which the job-scheduling, and then pass each file to the Analysis.m one at a time?
0 commentaires
Réponse acceptée
Edric Ellis
le 10 Oct 2011
Perhaps a PARFOR loop on the cluster would be simplest. Something like this:
function x = doStuff
% list the files on the cluster
d = dir( '/path/on/cluster/*.dat' );
% loop over the files - this will spread the work
% among the workers
parfor ii = 1:numel( d )
x(ii) = someFcn( d(ii).name );
end
Then, you need to submit 'doStuff' as a matlabpool job to the cluster, something like this:
job = createMatlabpoolJob( sched, 'MaximumNumberOfWorkers', 4 );
createTask( job, @doStuff, 1, ... );
job = batch( @doStuff, 1, {}, 'Matlabpool', 4 );
4 commentaires
Plus de réponses (1)
Voir également
Catégories
En savoir plus sur Startup and Shutdown dans Help Center et File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!