Calculate RMSE of NARX to evaluate Performance

Hey guys,

i've been working on creating a model predictive controller and using the input and ouput data generated from it to train a NARX to then imitate its behaviour.

The training results seem very good after doing some research on how to interpret the results. Howere, I would like to also calculate the RMSE of the performance and that's where I'm stuck.

I would also like to know how to proceed afterwards if I want to use this specific trained NARX network to test other external data.

Data consist of : Uset [2x11110] and Xset [1x11110]

The code to training the NARX is the following :

% 1 Data Preparation:

inputData = Uset'; % Transpose Xset to have dimensions 11110x1

outputData = Xset'; % Transpose Uset to have dimensions 11110x2

% Create input-output pairs

data = [inputData, outputData];

% 2 Data Partitioning:

% Define the percentages for data partitioning

trainRatio = 0.7;

valRatio = 0.15;

testRatio = 0.15;

% Use the dividerand function to partition the data

[trainInd, valInd, testInd] = dividerand(size(data, 1), trainRatio, valRatio, testRatio);

% Create the training, validation, and testing sets

trainData = data(trainInd, :);

valData = data(valInd, :);

testData = data(testInd, :);

% 3 Prepare Time Series Data:

% Define the input and output time delays

inputDelays = 1:2; % Adjust as needed

outputDelays = 1; % Adjust as needed

% Prepare input and target time series data

trainInputSeries = con2seq(trainData(:, 1:end-1)');

trainTargetSeries = con2seq(trainData(:, end)');

valInputSeries = con2seq(valData(:, 1:end-1)');

valTargetSeries = con2seq(valData(:, end)');

testInputSeries = con2seq(testData(:, 1:end-1)');

testTargetSeries = con2seq(testData(:, end)');

% Create and train the NARX network

net = narxnet(inputDelays, outputDelays,100);

[Xs,Xi,Ai,Ts] = preparets(net, trainInputSeries, {}, trainTargetSeries);

net = train(net, Xs, Ts, Xi, Ai);

view(net)

Any feedback would be appreciated.

Kind regards

Answer 1

[Venu]

Hi @wissal zaher,

I understand you want to calculate RMSE of peformance. The "perform" function in MATLAB, when used with a neural network object and test targets and outputs, by default returns the MSE of the network's performance on the provided dataset. Prepare the test inputs and initial states with "preparets" function. You can do square root for the output of perform function to obtain RMSE.

You can add the below code after view(net) for performance evaluation.

[XsTest, XiTest, AiTest, TsTest] = preparets(net, testInputSeries, {}, testTargetSeries);

testOutput = net(XsTest, XiTest, AiTest);

perf = perform(net, TsTest, testOutput);

RMSE = sqrt(perf);

find the documentation below for reference.

https://www.mathworks.com/help/deeplearning/ref/narxnet.html

You can follow the same procedure with external data also i.e:

1. prepare the data for the network using "con2seq" and "preparets"

2. Pass the prepared input data and initial states directly to the network object "net"