Time Series forecasting with 3 input data NARnet or NARXnet?

My Input Data is given in this form in MS-Excel spreadsheet:

Date               OilRate(BOPD) GasProduced(MscfD)                   WaterRate(BOPD)
7/27/2008          4108                4620                             2.08
7/28/2008          4557               5120                             5.54
7/29/2008          4128                4640                             5.03
7/30/2008          5722                6730                             9.25
7/31/2008          3321                3720                             7.11

For up to 2000 timesteps over 3 years;

My questions are:

1. I would like to predict future values of the above 3 for up to 2000 timesteps. (giving me a 3 input, 3 output)?
2. Do I import the data as a matrix or a cell array?
3. Would you recommend a NARnet or NARXnet, I have used NARNET with the ntstool
4. I tried adopting the code in this tutorial: http://www.mathworks.com/matlabcentral/newsreader/view_thread/338508#934732but plotting was giving me a problem as the sample data set was a 1x100 cell and my data is 3x1500 (utilized only 1500 timesteps)

Any useful suggestions are highly appreciated.

Thanks!

[Olumide Oladoyin]

I had issues with the dimensions of my input codes, so I decided to stick with using the NARnet and only the volume of Oil Produced as my input (1050x1 double).

NMSEs =

      0.0572

NMSEc =

      6.0875

My Questions are:

1. What do the NMSE values mean for my network?
2. The error autocorrelation and input-error autocorrelation look good? ( i guess!) what do they also mean for my network?
3. The open loop results look right but the closed loop network just looks awful, please help!

Thank You

Here is my code:

clc 
plt=0;
% Autoregression Time-Series Problem with a NAR Neural Network
% Created Sat May 16 23:01:03 WAT 2015
%
% This script assumes this variable is defined:
%
%   PInput - feedback time series. 1050x1double
T = tonndata(PInput,false,false);
N = length (T);
% Choose a Training Function
% 
trainFcn = 'trainlm';  % Levenberg-Marquardt

% Create a Nonlinear Autoregressive Network
feedbackDelays = 1:4;
hiddenLayerSize = 6;
net = narnet(feedbackDelays,hiddenLayerSize,'open',trainFcn);
rng ('default')
% Choose Feedback Pre/Post-Processing Functions
% Settings for feedback input are automatically applied to feedback output
% For a list of all processing functions type: help nnprocess
net.input.processFcns = {'removeconstantrows','mapminmax'};

% Prepare the Data for Training and Simulation
% The function PREPARETS prepares timeseries data for a particular network,
% shifting time by the minimum amount to fill input states and layer states.
% Using PREPARETS allows you to keep your original time series data unchanged, while
% easily customizing it for networks with differing numbers of delays, with
% open loop or closed loop feedback modes.
[Xs Xsi Asi Ts] = preparets(net,{},{},T);
ts1 = cell2mat( Ts );

plt = plt+1; figure(plt), hold on
plot( 5:N, ts1, 'LineWidth', 2 )
% Setup Division of Data for Training, Validation, Testing
% For a list of all data division functions type: help nndivide
net.divideFcn = 'divideblock';  % Divide data randomly
net.divideMode = 'time';  % Divide up every value
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;

% Choose a Performance Function
% For a list of all performance functions type: help nnperformance
net.performFcn = 'mse';  % Mean squared error

% Choose Plot Functions
% For a list of all plot functions type: help nnplot
net.plotFcns = {'plotperform','plottrainstate','plotresponse', ...
  'ploterrcorr', 'plotinerrcorr'};

% Train the Network
[net tr Ys Es Af Xf] = train(net,Xs,Ts,Xsi,Asi);
ys1=cell2mat(Ys);
plot(5:N, ys1, 'ro', 'LineWidth', 2 )
legend( 'TARGET', 'OUTPUT' )
title( 'OPENLOOP NARNET RESULTS' )
%
Es = gsubtract( Ts, Ys )
%view( net )
NMSEs = mse( Es ) /var( ts1,1 ) 
% Test the Network
y = net(Xs,Xsi,Asi);
e = gsubtract(Ts,y);
performance = perform(net,Ts,y);

% Recalculate Training, Validation and Test Performance
trainTargets = gmultiply(Ts,tr.trainMask);
valTargets = gmultiply(Ts,tr.valMask);
testTargets = gmultiply(Ts,tr.testMask);
trainPerformance = perform(net,trainTargets,y);
valPerformance = perform(net,valTargets,y);
testPerformance = perform(net,testTargets,y);

% Closed Loop Network
% For multi-step prediction.
% The function CLOSELOOP replaces the feedback input with a direct
% connection from the outout layer.
        %netc = closeloop(net);
        %[xc,xic,aic,tc] = preparets(netc,{},{},T);
        %yc = netc(xc,xic,aic);
        %perfc = perform(net,tc,yc);
 [ netc Xci Aci ] = closeloop(net,Xsi,Asi);
 %view(netc)
 [Xc,Xci,Aci,Tc] = preparets(netc,{},{},Ts);
 [ Yc Xcf Acf ] = netc(Xc,Xci,Aci);
 Ec = gsubtract(Tc,Yc);
 yc1 = cell2mat(Yc);
 tc = ts1;
 NMSEc = mse(Ec) /var(tc,1)
% Multi-step Prediction
Xc2 = cell(1,N);
  [ Yc2 Xcf2 Acf2 ] = netc( Xc2, Xcf, Acf ); 
  yc2 = cell2mat(Yc2);

plt = plt+1; figure(plt), hold on
plot( 5:N, tc, 'LineWidth', 2 )
plot( 9:N, yc1, 'ro', 'LineWidth', 2 )
plot( N+1:2*N, yc2, 'o', 'LineWidth', 2 )
plot( N+1:2*N, yc2, 'r', 'LineWidth', 2 )
%axis( [ 0 2*N+2 0 1.3 ] )
legend( 'TARGET', 'OUTPUT' , 'TARGETLESS PREDICTION')
title( 'CLOSED LOOP NARNET RESULTS' )

<<

<<

<<

<<

>>

>>

>>

>>

Answer 1

[Greg Heath]

Mathematically, you have the choice of choosing any combination of the 3 series to be inputs and any, possibly other combination, to be outputs.

Physically, are any considered inputs and others considered outputs?

It might be worthwhile to use nncorr or fft to obtain the 3 autocorrelation functions and the 3 crosscorrelation functions. Then you can see which lags are the most important for prediction.

Use cells for series but convert to doubles for plots.

Greg

[Olumide Oladoyin]

Ok. Thanks. Physically, they are all measurements made from a producing oil well, the water produced and Gas produced are the accompanying fluids that come to surface with the oil produced. So, would your recommendation be to stick to the NAR network with the 3 of them as inputs? Giving me a 3X1500 input cell series? Or should I combine them differently to have as input and target for NARX network?

THANKS Greg

Olumide

[Greg Heath]

Thank you for the explanation.

NARX it is.

However, remember that with multi-output NARX, all feedback signals experience the same set of lags.

Therefore, it may be very worthwhile to look at the significant lags for the 6 correlation functions suggested above.

Greg

[Olumide Oladoyin]

Thanks for the replies...I better get started with the NARX coding and learn more about the correlation functions and how to utilize them, would be sure to put up my code here so I can be sure am on the right path.

Olumide.

[Greg Heath]

Always try to use the simplest possible model.

That would be a 3-input/3-output TIMEDELAYNET using significant lags from 3 autocorrelation functions and 3 cross-correlation functions. Then, you will have future predictions as long as you have input data.

However, if you want to continue to have future predictions after you run out of input data, then you need closed loop feedback.

This is more readily achieved by using NARXNET.

Although you might be able to use NARNET, the total reliance on feedback signals will introduce output-to-input error to a much greater extent than with NARXNET.

I would definitely use NARXNET.

Remenber: After the openloop design is converted to closeloop, test the closeloop net. If the performance is much worse than the openloop performance, then train the closeloop net with initial weights equal to the previous openloop weights!

Good Luck.