Correct fetchNext index after clearing FevalFuture object

3 vues (au cours des 30 derniers jours)
Nicola Dalla Pozza
Nicola Dalla Pozza le 14 Oct 2019
I am using parfeval to run code asynchronously on workers. The running time varies a lot between workers depending on some parameters that in the following code I represent with 'm'.
When I collect the results with fetchNext, I then process them with idx, in particular retrieving m(idx). At the same time, I need to clear the last retrieved FevalFuture object to save memory.
In Parfeval - Memory consumption piling up - Clear output data? it is suggest to simply assign [ ] to the FevalFuture object. However, it seems to me that doing this mix up the indices idx of the objects, which I need to further process the data. See for instance this mve
N = 10;
m = 1:10;
for idx = N:-1:1
f(idx) = parfeval(@(m,N) m*rand(N), 1, m(idx), N);
end
total = 0;
for idx = 1:N
[idx, data] = fetchNext(f);
idx
f(idx) = []; % This line here
% process data with idx, m(idx) ..
end
If you comment the line indicated the code will work fine, but if I clear f(idx) the just-used idx could be reused and I would process the data with the wrong m(idx).
How is it possible to get the correct index idx with which f(idx) was called in the fist loop?
Thanks in advantage

Réponses (1)

Edric Ellis
Edric Ellis le 15 Oct 2019
One way you can deal with this is by making the same modification to m as you proceed. Something like this perhaps:
N = 10;
m = 1:10;
for idx = N:-1:1
f(idx) = parfeval(@(m,N) m*rand(N), 1, m(idx), N);
end
for idx = 1:N
[fidx, data] = fetchNext(f);
% Get the value out of "m" corresponding to the completed future
mval = m(fidx);
% Delete corresponding elements of both "f" and "m"
f(fidx) = [];
m(fidx) = [];
% Work with the particular value of "m" and the result
disp([mval, sum(data(:))]);
end

Catégories

En savoir plus sur Asynchronous Parallel Programming dans Help Center et File Exchange

Produits


Version

R2018b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by