Why is the learning rate for the bias usually twice as large as the the LR for the weights?

Why is the learning rate for the bias usually twice as large as the the LR for the weights?

2 commentaires

Please post the proof of this assertion.
Greg
on the other hand if you check, for example alexnet layers you will find that net.Layers(N, 1).BiasLearnRateFactor=2 and net.Layers(N, 1).WeightLearnRateFactor = 1

Connectez-vous pour commenter.

Réponses (0)

Catégories

En savoir plus sur Deep Learning Toolbox dans Centre d'aide et File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by