How to reduce the usage of "broadcast variables" when using parfor
63 vues (au cours des 30 derniers jours)
Afficher commentaires plus anciens
Hi, in a "parfor" loop, a same warning message is put on three variables which I named "el", "ds", and "vmin". The message is: "The entire array or structure 'el' (or 'ds', 'vmin') is a broadcast variable. This might result in unnecessary communication overhead."
This "parfor" loop requires four variables which were calculated before the loop: Variable "el" is 4-by-378 double, "NElem"=27, "ds" is 27-by-27 sparse double, and "vmin" is 1-by-378 double.
Part of this loop is as follows:
el_row1=el(1,:);
parfor e=1:NElem
k=el_row1-e==0;
ind=el(4,k);
ds_e=ds(:,ind);
vs_e=ds_e*vmin(k)';
...
end
Is there a way to avoid this "unnecessary communication overhead", so that this parfor loop can run faster please? I read on the various types of variables in parfor-loops in Matlab documentation, I'm guessing the reason these three variables get the warning message is the form of indexing: "k" or "ind" is not a loop variable, so "el", "ds" or "vmin" cannot be sliced. But I don't know how to turn the variables "el", "ds", and "vmin" into slice variables. Any help is appreciated.
0 commentaires
Réponse acceptée
Walter Roberson
le 10 Août 2022
"Is there a way to avoid this "unnecessary communication overhead", so that this parfor loop can run faster please?"
No. You are accessing the values in random order.
k=el_row1-e==0;
ind=el(4,k);
You can build el4=el(4,:); and index that at el_row1==e so that el as a whole does not need to be sent.
You could findgroups el_row1 and splitapply or similar ahead of time to create cell ind{e} so that the index information could be sliced.
But ds and vmin are going to have to be sliced. Unless, that is, it turns out that the information needed for any given e turns out to be consecutive and can be split ahead of time.
Your arrays are not large at all. I would be concerned about whether they are doing enough work to make it worth using parfor.
6 commentaires
Bruno Luong
le 10 Août 2022
el_row1=[1 2 3 2 1 1];
[~,~,J] = unique(el_row1);
i = accumarray(J,(1:length(J))',[],@(x){x});
i{:}
Plus de réponses (1)
Bruno Luong
le 10 Août 2022
Modifié(e) : Bruno Luong
le 10 Août 2022
You can use parallel constant to convert el, el_row1, ds, vmin to constant and reduce the data transfert, if they are not modified (the snipet of code you showed they are not).
0 commentaires
Voir également
Catégories
En savoir plus sur Parallel for-Loops (parfor) dans Help Center et File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!