Slow access of large composite element
1 vue (au cours des 30 derniers jours)
Afficher commentaires plus anciens
I am using spmd to run some parallel algorithms which periodically communicate. At the end, each process has a structure which I would like to pull back to the client for further analysis. Unfortunately, the size of the structure is large (but I don't think they are excessive), and trying to pull just one of the composite elements leads to a timeout of the cluster (30 min) and therefore fails. Is there any way to have the client as one of the spmd processes, or a more efficient way to transfer data from one or more of the workers back to the client in a more efficient way?
EDIT:
A clunky workaround is to have each process save their workspace to a directory. This seems to work pretty quickly, but it's obnoxious to have to import all the workspaces to do the diagnostics. I would appreciate more elegant solutions.
5 commentaires
Walter Roberson
le 9 Fév 2018
Looking more closely, it is for "communicating jobs", of which spmd is one kind.
Réponse acceptée
Plus de réponses (0)
Voir également
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!