MATLAB Answers

As Has

As Has (view profile)

0

Why is the learning rate for the bias usually twice as large as the the LR for the weights?

Asked by As Has

As Has (view profile)

on 3 Jan 2018
Latest activity Edited by Image Analyst

Image Analyst (view profile)

on 5 Jan 2018
Why is the learning rate for the bias usually twice as large as the the LR for the weights?

  2 Comments

Greg Heath

Greg Heath (view profile)

on 4 Jan 2018
Please post the proof of this assertion.
Greg
As Has

As Has (view profile)

on 5 Jan 2018
https://datascience.stackexchange.com/questions/23549/why-is-the-learning-rate-for-the-bias-usually-twice-as-large-as-the-the-lr-for-t
on the other hand if you check, for example alexnet layers you will find that net.Layers(N, 1).BiasLearnRateFactor=2 and net.Layers(N, 1).WeightLearnRateFactor = 1

Sign in to comment.

Tags

Products

0 Answers

Sign in to answer this question.