Which is the difference between 'multi-gpu' and 'parallel-gpu' in 'trainingOption' function of the DeepLearning Toolbox?

Hi everyone,

I have two NVIDIA RTX 3060 installed on my local computer and I want to train a neural network in parallel on both GPUs. I am worried about which is the best strategy between 'multi-gpu' and 'parallel-gpu'. Does anyone know how they work and which is the difference between 'multi-gpu' and 'parallel-gpu'?

Thank you.

Answer 1

[Matt J]

According to the doc, 'parallel-gpu' has the additional capability of being able to use remote GPUs. Since that doesn't apply to the hardware environment you describe, you can probably use either one.

[Enzo Losi]

Thank you very much for the quick response!

Answer 2

[Joss Knight]

The purpose of 'multi-gpu' is effectively to try to ensure you are using a local pool with numGpus workers, rather than needing to understand anything about configuring a cluster. So either can work, but multi-gpu will give you helpful errors if you're doing something you didn't intend.