A problem when using "multi-gpu" as "ExecutionEnvironment" for training a CNN
5 views (last 30 days)
Show older comments
Hyung Joon on 7 Aug 2022
Commented: Joss Knight on 8 Aug 2022
I am experiencing weird problems when I use the “multi-gpu” as the “ExecutionEnvironment” in the training option for training a CNN.
I am using the simple CNN example shown in the MATLAB’s website: https://www.mathworks.com/help/deeplearning/ug/create-simple-deep-learning-network-for-classification.html
I only added the “multi-gpu” as the “ExecutionEnvironment” and changed the “MaxEpochs” to 50 in the training option.
I am using two of NVIDA RTX 2080 Ti 11GB from the same manufacturer (Galax). The two graphic cards are the same models from the same manufacturer.
In MATLAB R2020a, the CNN example works fine with faster training speeds when I use the “multi-gpu” as the “ExecutionEnvironment” in the training option. The training progress plot is attached below.
But in MATLAB R2022a, the training progress works really weird when I run the same code. The validation data curve goes high while the training data curve fluctuates at the middle and never goes up. The screenshot showing this is attached below.
And the validation accuracy in the command window after the training is also bad (63.92%) while the validation accuracy only in the training progress plot is good.
This issue also happens in other MATLAB versions later than R2020a. When I tested with a single GPU separately and individually, the training works fine.
Can anyone please help me with this issue? I need to use my two GPUs for the "multi-gpu" training in MATLAB R2022a for my work so I first need to fix this weird problem in the example.
I would appreciate any help. Thank you very much.
Joss Knight on 7 Aug 2022
Edited: Joss Knight on 8 Aug 2022
Most likely this is this issue, which is fixed in the latest update to R2022a. You can also try downgrading your GPU drivers.
More Answers (0)
Find more on Parallel and Cloud in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!