MATLAB Answers

0

Avoiding Overfitting by Averaging Multiple Neural Network

Asked by Robert Henson on 29 Jan 2018
Latest activity Edited by Greg Heath
on 25 May 2018
I’m using a narnet neural network to model stock time series data so I can get step-ahead and multi-step predictions. I’m concerned about overfitting in this model and have been reading in Matlab about techniques to avoid overfitting. I would like to try averaging multiple networks to improve generalization and train multiple neural networks and average their outputs (as in the example provided). The example shows this technique on a feedforward net. So, my question is (and please forgive me is I ask stupid questions since I’m still a relatively new user of Matlab and neural networks) can I use the averaging technique on my narnet network? If so, would the same example work? If not and I need to use the feedforward net, how then can I use that output in my model to find step-ahead and multi-step predictions for my stock time series? I would appreciate any input and/or suggestions. Many thanks. Robert Henson

  0 Comments

Sign in to comment.

1 Answer

Answer by Greg Heath
on 25 May 2018
Edited by Greg Heath
on 25 May 2018

The best way to mitigate overtraining an overfit net is
MINIMIZE THE NUMBER OF HIDDEN NODES SUBJECT TO A MAXIMUM ALLOWED ERROR RATE.
The conventional I-H-O NN has I Input nodes, H Hidden nodes, O output nodes and Nw unknown weights where
Nw = (I+1)*H+(H+1)*O = (I+O+1)*H +1
With Ntrn training examples the total number of training equations is
Ntrneq = Ntrn*O
To prevent overfitting: No. eq >= No. unknowns:
Ntrneq >= Nw
or
H <= Hmax <= Hub = (Ntrn*O-1)/(I+O+1)
I have posted zillions of examples in BOTH
comp.soft-sys.matlab and ANSWERS.
Search on "greg" and one or more of the following Ntrneq, Hmax, Hub
Thank you for formally accepting my answer
Greg

  0 Comments

Sign in to comment.