Is there a technical limitation/design choice preventing dynamic allocation of parfor workload?

1 vue (au cours des 30 derniers jours)
Naivity might be to blame for my asking this, but I mostly use parallel computing for running series of simulations and it's not at all uncommon for one or a few of these to involve way more processing time than the rest, or for a worker to inexplicably slow down for one iteration (presumably due to some sort of demand on that logical processor by another program/the OS). This usually results in most of the workers finishing way ahead of one or a few, and often results in the last worker(s) having to run through several iterations at the end while the rest of the workers sit idle. I'm sure this could be handled by taking more direct control over the parallelization process, but it seems like putting idle workers to task on the work backlog of other workers would generally be preferable in a quick-'n-dirty parallel computation situation, which seems to be exactly the application suited to parfor. I can definitely think of reasons why it's more straightforward to code parfor as it is now (less logic involved; simply divvy up the work, send it off, wait for all workers to report back), but are there stronger reasons for this behavior as well?

Réponses (1)

Edric Ellis
Edric Ellis le 11 Jan 2019
The parfor implementation already attempts to take account of this sort of situation - behind the scenes, the loop is split up into multiple "intervals" or sub-divisions of the overall loop range. The interval sizes are chosen using a heuristic that attempts to get decent performance in most common situations. Each worker gets roughly 5 intervals (providing there are enough loop iterates). These intervals are not equally sized - we start with short intervals to get the workers busy as soon as possible, then some larger intervals to try keep the workers busy with decent amounts of work, and finishing with shorter intervals to try and overcome effects like you're seeing in cases where the iterations take differing amounts of time.
In particular, we don't send all the intervals out to the workers up-front - if one worker gets an iterate that takes a huge amount of time, it will not receive any further intervals from the client.
If you want more control, you should investigate parfeval as this lets you divide up the work as you see fit, at the expense of not being quite so convenient as parfor.

Catégories

En savoir plus sur Parallel for-Loops (parfor) dans Help Center et File Exchange

Produits


Version

R2017b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by