a question on neural network

Hello Could you plz let me know what does Hub mean in NN trainig? I have seen that some of criteria are used for setting NN training parameters like Hub,Nw, Ntr,.. could you plz introduce some refernces like papers, books for all those who are interested to know more about these terms? best

Answer 1

[Greg Heath]

The only direct references are my posts in NEWSGROUP and ANSWERS. Search on those variable names to obtain further details.

An over-fit net is a net that contains more weights than are necessary to successfully approximate the desired I/O relation. The dangers of overtraining an over-fit are well documented in books and other references that are not difficult to find. I recall that the FAQ of comp.ai.neural-nets explains it well.

One solution to avoiding the problem of overtraining an over-fit net is to not over-fit the net.

If Ntrneq is the number of training equations and Nw is the number of unknown weights, Ntrneq >> Nw leads to locally optimum approximate solutions which are robust with respect to noise, interference and measurement error.

If the number of hidden nodes, H, is much less than Hub, then Ntrneq >> Nw.

Other solutions to the problem are

 Use a non-training validation set to stop training at the first signs of overtraining an overfit net.

 Use a regularized objective function (a mse option or the obsolete msereg ) that prevents the size of weights from being large enough to cause the net to exhibit overtraining characteristics.

Thank you for formally accepting my answer

Greg

[Mohamad]

Greg Thank you very much.