The gradient of mini batches
11 vues (au cours des 30 derniers jours)
Afficher commentaires plus anciens
MAHSA YOUSEFI
le 23 Nov 2020
Commenté : Mahesh Taparia
le 21 Déc 2020
Hi there.
I need your confimation or rejection for this question...
In following code, if the minibatch size is h,
[grad,loss] = dlfeval(@modelGradients,dlnet,dlX_miniBatch,Y_miniBatch);
the grad is the average of gradients of loss over h samples? Does it calculate dradients automatically and at the end with:
grad = 1/h * sum_i=1:h (\nabla loss(y_i,yHat_i)) ??
Following this question, for computing the total loss and geadient (for a full batch), does we should take avarage of losses and averages of gradients (averaging with the number of batches, say 1000 batches each with h size)??
0 commentaires
Réponse acceptée
Mahesh Taparia
le 14 Déc 2020
Hi
The function dlfeval evaluate the custom deep learning models. The loss are calculated based on what has been defined in modelGradients function. So if you are calculating the average loss in this function, then it will return the averaged one. For example, consider this modelGradient function, it is calculating the average cross entropy loss, so it will return the average loss. The gradients are calculated with respect to the loss function defined in for the network.
2 commentaires
Mahesh Taparia
le 21 Déc 2020
Yeah, crossentropy loss will be calculated between dlY and T. The documentation page will be updated.
Plus de réponses (0)
Voir également
Produits
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!