Accessing a single GPU from different workers of a parpool thread pool

It used to be conventional wisdom that parfor does not play well together with gpuArray on a single-GPU system because different parpool workers end up competing for the same CPU-GPU communication channel. Is that still true now that there is a distinction between process pools and thread pools? Can it be worthwhile to parfor-parallelize a loop containing gpuArray operations if I have only a single GPU?

Answer 1

[Isha]

Hello,

Yes, the old advice is still mostly true. On a single GPU, parfor + gpuArray usually gives little or no benefit, even with thread pools.

No real GPU parallelism:parfor distributes work across CPU workers; with one GPU, all workers still target the same device, so it doesn’t scale unless you have multiple GPUs. https://www.mathworks.com/help/parallel-computing/run-matlab-functions-on-multiple-gpus.html

Overheads still matter:Parallel pools introduce scheduling and data‑movement overhead, which can offset gains. https://www.mathworks.com/help/parallel-computing/compare-performance-multithreading-processpool.html

Thread pools only reduce CPU overhead:They offer lower data transfer and faster scheduling compared to process pools, but they do not change how many GPUs you have. https://www.mathworks.com/help/parallel-computing/choose-between-thread-based-and-process-based-environments.html

Hope this helps.

[Matt J]

all workers still target the same device, so it doesn’t scale unless you have multiple GPUs

It's not immediately clear to me why that follows. Aren't there some modern GPUs that can communicate with several CPU processes and run several kernels in parallel?