Why can future values affect NARX time series output?

I am using A NARX network with one target vector and several input vectors. I am trying to predict the target value each day based on the inputs provided that day. If I feed all of the input and target values into the network up to and including the current day, I get one output result. But if I also feed in additional future input and target values, I get a different output result.

I have used " rng 'default' " to keep the same seed for the random number generator. I suspect there is some normalization or something going on that is using the future values, but I can't locate it. I am not doing that myself.

Obviously, any prediction routine that uses future data is invalid.

Thoughts?

I checked this as follows: I substituted a linear regression model instead of the NARX, and got completely reproducible and identical results whether the future data was present or not. The inputs and targets fed to this routine were identical to those fed to the NARX.

Answer 1

[Walter Roberson]

I am perhaps being naive on this, but the behaviour you describe for NARX is what I would expect.

Suppose you had a series of points and you were doing a polynomial fit on them and using it to estimate values:

rng(12346)
N = 25;
x = 1:25;
y = rand(1, N);
x20 = x(1:20);
y20 = y(1:20);
p20 = polyfit(x20, y20, 8);
yfit20 = polyval(p20, x20, y20);
plot(x, y, '.k');
hold on
plot(x20, yfit20, '-ob');
hold off

Now, would you expect the same prediction for the 20th location if you added more data in?

p_all = polyfit(x, y, 8);
yfit_all = polyval(p_all, x, y);
figure
plot(x, y, '.k');
hold on
plot(x20, yfit20, '-ob');
plot(x, yfit_all, ':+r');
hold off

Any system that uses any kind of aggregate information (mean, standard deviation, moments) is likely to produce a different output for a given location if you fed it more information.

[Kevin Johnson]

So you are saying that indeed, the NARX time series prediction uses future data?

In what sense then is it a prediction? It's more a regression then.

[Walter Roberson]

NARX is nonlinear autoregressive network with exogenous inputs -- so Yes, it includes an element of regression.

[Kevin Johnson]

The point is not the regression, it's the prediction. If future data is used for the regression, it is not prediction. Agree?

[Kevin Johnson]

[Kevin Johnson]

No mention of using values at t+1 etc.

[Walter Roberson]

But if I also feed in additional future input and target values, I get a different output result.

Perhaps I am not understanding you in what you mean by that.

If you are saying that given rng('default') that you expect that if you feed in only x(1:20) and y(1:20) that the prediction for x(21) should be the same as-if you had feed in all x(1:25) and y(1:25) and then asked to predict for x(21) -- if that is what you are saying, then I would remind you that when you have more input data then the random splits between training and test and validation are going to be different, and different number of iterations might be used to meet tolerance goals (training can stop for a number of different reasons, not just for having taken a fixed number of iterations).

[Kevin Johnson]

That is what I am saying, yes. I can see what you mean. I was under the impression that the algorithm marches along in time, predicting at the leading edge - the "frontier". I see I am likely mistaken in that. I think my error is that I am training on the whole set in both variant examples (partial data vs. complete), and during training, the algorithm does indeed look backward and forward in the sense of data divison etc as you say. So instead I need to train the network, then separately simulate the network going forward in tme. My results were very good when I input all the data, and I see now that was probably essentially an in-sample result, which is almost always better than an out-of-sample result. Sound right?