Time delay vs. NAR neural network and time series

Let's assume I am able gather high frequency sampling (second order) data from a (non-trivial) dynamical process. I end with a single and quite long time series y(t). This process has some short and long-term memory (possibly at different scales), but we don't really know, these are general assumptions. I aim at predicting the immediate upcomging value of y(t), namely y(t+1), knowing d past values of y(t), including y(t). I also assume that I am able to find an optimal value of d (yay!).

The goal is to build a regressive neural network (NN) that can predict y(t+1) from I=[y(t-d):y(t)] values. This NN will be used in a program and will continuously predict the output, knowing I. It will act like a sliding window.

I have three questions:

1. At first I was considering using a time-delay network, then now I am considering using a NAR network. Both architectures seem possible to me. Is there a best option?
2. As Greg suggested in a previous post, I am considering using 'divideblock' instead of 'dividerand' to keep track of autocorrelation that may exist. Any more discussion / guide I should read about this?
3. I am going to test for different initial conditions. I'm using the timedelaynet(.) and narnet(.) functions. As far as I read I should be using init(.) or configure(.) but I tested a 1 period training without using these two functions and it seems that the train(.) function does initialize the weights at random. Is this ok? I was not able to find any clues about this in the official matlab documentation.

Thank you!

Answer 1

[Greg Heath]

ONE:

Use NARNET :

y(t) = f( y( t-d : t-1 ) )

TIMEDELAYNET requires an applied input and does not use output feedback :

y(t) = f( x( t-d : t-0 ) )

Both are special cases of NARXNET

 y(t) = f( x( t-d1 : t-0 ) , y( t-d2 : t-1 ) )

NOTE: In general, MATLAB doc and/or help incorrectly treats time = 0

=============================================== TWO:

AFAIK I'm the only one that as mentioned it.

Plot abs(autocorr)) to make sure your delays make sense

=============================================== THREE:

TRAIN will randomly initialize ONLY if there are no existing weights.

If you use loops to optimize H and/or the initial weights:

After the first pass though the loop, the net will have weights. Therefore the net will not automatically initialize and you will just keep trying to improve one design!!!

Hope this helps.

Thank you for formally accepting my answer

Greg

[Gabe Hopper]

Thank you for such a fast reply (as you always do!).

I do not have any exogeneous inputs so I don't think I'll go for a NARXNET.

However, you don't adress any of the two questions 2 and 3, i really need more 'input' on these two :D

Thank you

[Gabe Hopper]

That's perfect! thanks!