How to avoid repeatly loading parallel pool when running .exe matlab file in cmd

5 vues (au cours des 30 derniers jours)
I am using parfor in a matlab program.
Then I generate a excutable .exe file using "mcc -m " command on that program.
Later, when we I call the .exe file in cmd, the program load parallel pool everytime:
"Starting parallel pool (parpool) using the 'Processes' profile ..."
It takes so much time since I need to repeatly call it.
Do you know any way to avoid loading parpool every time when I call it? Can I preload it in cmd? Thank you.
  8 commentaires
Damian Pietrus
Damian Pietrus le 16 Oct 2024
As long as the code ran as expected on your local machine, you should be able to have it run separately on your colleague's machines as well. A thread pool is limited to one physical machine at a time, meaning that both you and your colleagues can separately run the code on your own systems. Where a thread pool would not be supported would be taking one instance of your application and having the pool open across multiple machines.
Mingkang
Mingkang le 16 Oct 2024
I see, thank you for your help.

Connectez-vous pour commenter.

Réponse acceptée

Damian Pietrus
Damian Pietrus le 16 Oct 2024
Moving from comments to an Answer:
When a processes parpool is launched, multiple separate MATLAB processes are initiated. The client then connects to these separate processes and the pool is established. Since each instance of your .exe is a separate client, you will always end up opening a pool every time you launch the file.
If the pool startup time is your main issue, there is one potential option. If the code in your parfor is thread safe, you can launch a thread pool instead of a process pool. Here, the workers run as a thread in the main MATLAB processes and share memory, with the added benefit of quicker startup. There's some more info here:
Note that while thread workers are supported in standalone applications created using Compiler, not all functions are supported in a thread pool. Also note that a single instance of your application can only run on your local machine -- a threadpool cannot be opened across multiple compute nodes

Plus de réponses (2)

Hitesh
Hitesh le 16 Oct 2024
The issue might occur if the code is creating new parpooleverytime without checking the existence of the existing “parpool”. To avoid creating new “parpool” and use the existing instance you need to check the existence of existing parpool. You can add the below condition to check for any active "parpool" before calling "parfor". Kindly refer to the below code for an example:
if isempty(gcp('nocreate'))
parpool('Processes');
end
Please find below the description of the functions used in above code example.
  • gcp('nocreate'): This command checks if a parallel pool is currently active. The "nocreate" option ensures that it only checks for an existing pool and does not create a new one if none exists.
  • isempty(...): This function checks if the result of "gcp('nocreate')" is empty, meaning no parallel pool is currently running.
  • parpool('Processes'): If no parallel pool is found, this line starts a new parallel pool using the 'Processes' profile.
For more information on "parpool" and "gcp('nocreate')", refer to the below MATLAB Documentation:
  1 commentaire
Mingkang
Mingkang le 16 Oct 2024
Thank you for your reply. I need to call an .exe file with parfor many independently in cmd. Everytime, it load and unload the parpool once. I hope to avoid it.

Connectez-vous pour commenter.


Walter Roberson
Walter Roberson le 16 Oct 2024
The official way to reduce the initiation time is to use the MATLAB Production Server product; it keeps pools "warm" so there is little time spent initializing pools.
If you do not use MATLAB Production Server, then there is no solution to your difficulty.
  1 commentaire
Mingkang
Mingkang le 16 Oct 2024
I see, thank you for you reply. Then I suppose the only way is convert the program into python.

Connectez-vous pour commenter.

Catégories

En savoir plus sur Parallel Computing Fundamentals dans Help Center et File Exchange

Tags

Produits


Version

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by