Using multiple GPUs in a parfor type of loop
4 views (last 30 days)
I am working on a machine learning problem where I am training my classifier using a GPU and the parallel computing toolbox. I want to do 10-fold cross-validation (basically train 10 different classifiers, completely independently of each other). I have access to a machine with two Tesla GPUs. Is there a straightforward way to to combine a parfor loop with two workers, each with their own GPU attached, to do this?
When I open the matlabpool can I just specify the two different GPU devices, one for each worker and then never have to bother with it again?
James Lebak on 5 Apr 2013
In R2012b and later, in a matlabpool on a single machine with two GPUs, each worker will use a different GPU by default. You can see this by executing the following code:
matlabpool local 2
parfor ix = 1:10
The value of d will show you the index of the GPU on which each iteration was executed.
In earlier releases the workers all use device 1 by default, and you would have to set the device manually using spmd outside the parfor loop or (less optimally) inside each loop iteration.