Why does training my network in batches give different results than training in a single step?

I would like to know why training my network in batches gives different results than training in a single step.

If I train my Neural Netwok 10 times for 10 epochs, I get a different result than if I had trained it once for a hundred epochs.

Answer 1

[MathWorks Support Team]

When training a neural network, each input vector is presented to the network at least once. Presenting all vectors once is called an epoch.

Most of the training algorithms have parameters that are adjusted adaptively during training. When you stop and restart training, these parameters revert to their initial values. This means that training the network 10 times for 10 epochs will give you a result similar to if you had trained the network just once for 10 epochs. This is why training a network 10 times for 10 epochs is not equivalent to training 1 time for 100 epochs.

[Greg Heath]

However, you can obtain the values of the parameters after the 10th epoch and explicitly initialize the net with those values at the beginning of the 11th epoch, etc.

On the example I ran, I only had to reinitialize mu. Your example may differ.

Hope this helps.

Greg

[Greg Heath]

According to the previous comment, only mu reverts to the initial value.

[Greg Heath]

foo???

My answer was based on comparing the results of a simple comparison test.

Try it.

Greg