difference between traingdm in patternnet vs sgdm in cnn

Are they the same method of training in differnet types of neural network or how are they different?

traingdm = gradienbt descent with momentum backpropagation (used in patternnet)

sgdm = stochastic gradient descent with momentum (used in convolutional neural network)

Answer 1

[Abhipsa]

Hi @San May, “traingdm” is a training function used in “patternnet” networks. Since “patternnet” supports only batch processing i.e. it uses the entire dataset to compute the gradient of the loss function once per iteration hence “traingdm” performs batch gradient descent with momentum. This can be computationally intensive for large datasets but provides a more stable convergence path.

On the other hand, “sgdm” is “Stochastic Gradient Descent with Momentum” which is used in CNNs. Being stochastic, it updates model parameters using a single data point (or a mini batch) at each iteration. This approach may introduce more noise in the updates but can lead to faster convergence.

You can refer to the below MATLAB answer for more details about training mechanism used in “patternnet”.

https://www.mathworks.com/matlabcentral/answers/464205-batch-or-online-training

For more details, you can also refer to the following MATLAB documentations.

• traingdm: https://www.mathworks.com/help/releases/R2024b/deeplearning/ref/traingdm.html
• sgdmupdate: https://www.mathworks.com/help/releases/R2024b/deeplearning/ref/sgdmupdate.html
• trainingoptions: https://www.mathworks.com/help/releases/R2024b/deeplearning/ref/trainingoptions.html
• patternnet: https://www.mathworks.com/help/releases/R2024b/deeplearning/ref/patternnet.html

I hope this helps you.