MATLAB Answers

0

Why is the learning rate for the bias usually twice as large as the the LR for the weights?

Asked by As Has on 3 Jan 2018
Latest activity Edited by Image Analyst
on 5 Jan 2018
Why is the learning rate for the bias usually twice as large as the the LR for the weights?

  2 Comments

Please post the proof of this assertion.
Greg
on the other hand if you check, for example alexnet layers you will find that net.Layers(N, 1).BiasLearnRateFactor=2 and net.Layers(N, 1).WeightLearnRateFactor = 1

Sign in to comment.

0 Answers