NARX closed loop performance for multi-step prediction

I'm analyzing Matlab NARX example for multi-step prediction using maglev dataset but I'm not sure on how calculate closed loop performance.

Below my code:

%%Example NARX

[X,T] = maglev_dataset; % dataset containing 4001 records

% I want to investigate how number of samples affect NARX training
% so I use only a fraction, N samples, of entire dataset

N=60;
delay=2;

X1=X(1:N+delay);
T1=T(1:N+delay);
net = narxnet(1:delay,1:delay,10);

net.divideMode='value'; 

% dividerand should destroy inputs autocorrelation so 
% I should set 'divideblock' ?
net.divideFcn='dividerand';
net.divideParam.trainRatio=0.60;
net.divideParam.testRatio=0.10;
net.divideParam.valRatio=0.20;

[x,xi,ai,t] = preparets(net,X1,{},T1);
net = train(net,x,t,xi,ai);
y = net(x,xi,ai);
perf=perform(net,t,y);
fprintf('Performance open looop :%d\n',perf);

%%now change to closeloop
pred_horizon=20;

% number of samples necessary to set initial state
% I want to test NARX prediction capability starting
% not where training end but after some samples 
x1 = X(118:120);
t1 = T(118:120);

% this is correct? I have some perplexity on how define t2 (usually not available)

t2=T(121:121+pred_horizon-1);
x2 = X(121:121+pred_horizon-1);

[x,xi,ai,t] = preparets(net,x1,{},t1);
[y1,xf,af] = net(x,xi,ai);

% is better to train again network after changing to 
% closeloop with same data utilized in openloop?
[netc,xi,ai] = closeloop(net,xf,af);

[y2,xf,af] = netc(x2,xi,ai);
perfc=perform(netc,t2,y2);
fprintf('Performance closed looop :%d\n',perfc);

err=gsubtract(t2,y2);
fprintf('MSE :%f\n',mse(cell2mat(err))); % same as perfc
subplot(2,1,1);
plot([cell2mat(y2)' cell2mat(t2)']);
legend('NARX output','Real Values');
subplot(2,1,2);
plot(cell2mat(err));
figure;
% does plotting regression chart make any sense ? I obtain weird results
plotregression(t2,y2);

Answer 1

[Greg Heath]

Compare with relevant searches

 closeloop maglev greg   4hits
 closeloop maglev        5 hits
 closeloop  greg        66 hits
 closeloop              99 hits

I like this one:

http://www.mathworks.com/matlabcentral/newsreader/view_thread/332147#912806

Hope this helps.

Thank you for formally accepting my answer

Greg