LSTM for data prediction

Hi, I am doing a program for prediction using lstmLayer. For example,

input = [2.49810000000000;1];[2.10350000000000;0];[3.09650000000000;0];[3.81500000000000;0];[13.1828000000000;0];19.4358000000000;0];[15.8167000000000;0];[13.8152000000000;0];[11.6463000000000;0];[13.8282000000000;0];[12.9633000000000;0];[18.1005000000000;1];[22.5192000000000;0];[15.7932000000000;1];[11.6019000000000;1];[6.75210000000000;1];[11.9233000000000;0];[15.1086000000000;0];[34.5000000000000;0];[29.8018000000000;0];[21.7118000000000;0];[16.1591000000000;1];[7.30350000000000;1];[5.43510000000000;1]

expectedOutput = [-4.64682000000000e-15;0;0;0;3.84368400000000;1.09669700000000;6.47763100000000;4.43292000000000;2.04262700000000;3.81629800000000;3.53217500000000;-0.290735500000000;3.25944900000000;-3.41447200000000;-7.10655400000000;-11.7593800000000;2.06697600000000;5.41984200000000;14.1654800000000;11.4626400000000;3.37265600000000;-2.18005900000000;-11.0356600000000;-12.9039900000000]

Command: layers = [ ... sequenceInputLayer(2) lstmLayer(100,'OutputMode','last') fullyConnectedLayer(1) softmaxLayer classificationLayer]

option = trainingOptions('sgdm','MaxEpochs',100);

net = trainNetwork(input,expectedOutput,layers,option);

MATLAB message: Error using trainNetwork (line 140) Invalid training data. Responses must be a vector of categorical responses, or a cell array of categorical response sequences.

Caused by: Error using nnet.internal.cnn.util.TrainNetworkDataValidator/assertValidSequenceResponse (line 279) Invalid training data. Responses must be a vector of categorical responses, or a cell array of categorical response sequences.

The data type of the input is cell and the output is double. However, the error remain exist and the prediction of the program is failed. How can I do the prediction for my case?

Answer 1

[Li Bai]

You have to make expectedOutput as categorical type. What you need to do is

expectedOutput=categorical(expectedOutput); before the line

net = trainNetwork(input,expectedOutput,layers,option);

Answer 2

[israel agbehadji]

You can also try this...It worked in my case as well..

input={[2.49810000000000;1],[2.10350000000000;0],[3.09650000000000;0],[3.81500000000000;0],[13.1828000000000;0],[19.4358000000000;0],[15.8167000000000;0],[13.8152000000000;0],[11.6463000000000;0],[13.8282000000000;0],[12.9633000000000;0],[18.1005000000000;1],[22.5192000000000;0],[15.7932000000000;1],[11.6019000000000;1],[6.75210000000000;1],[11.9233000000000;0],[15.1086000000000;0],[34.5000000000000;0],[29.8018000000000;0],[21.7118000000000;0],[16.1591000000000;1],[7.30350000000000;1],[5.43510000000000;1]} expectedOutput =[-4.64682000000000e-15;0;0;0;3.84368400000000;1.09669700000000;6.47763100000000;4.43292000000000;2.04262700000000;3.81629800000000;3.53217500000000;-0.290735500000000;3.25944900000000;-3.41447200000000;-7.10655400000000;-11.7593800000000;2.06697600000000;5.41984200000000;14.1654800000000;11.4626400000000;3.37265600000000;-2.18005900000000;-11.0356600000000;-12.9039900000000] layers = [ ... sequenceInputLayer(2) lstmLayer(100,'OutputMode','last') fullyConnectedLayer(22) softmaxLayer classificationLayer]

option = trainingOptions('sgdm','MaxEpochs',100); expectedOutput=categorical(expectedOutput); net = trainNetwork(input,expectedOutput,layers,option)

[Bhavna Rajasekaran]

Can the output be a categorical 2D array? Meaning input([x x 1]) and output([y y 1]) time series, where say, x=32, y=32, with time=10, So each is a categoricla cell aray of size 10 -by-1. So each input point has a corresponding response. Currently, i get the following error:

"Invalid training data. Responses must be a vector of categorical responses, or a cell array of categorical response sequences."

I have made categorical response sequences of cell arrays for size 10-by-1, where each cell array is 32-by32 categorical matrix, corrsponds to to input of same size.

[Marcelo Olmedo]

Hello! Did you find any solution to the problem? At home I have no problem training the network with multiple inputs, but I have problems predicting using the statements:

[net,YPred] = predictAndUpdateState(net,XTrain_2(1:5));

numTimeStepsTest = numel(XTrain_2);

for i = 2:numTimeStepsTest

[net,YPred(:,i)] = predictAndUpdateState(net,YPred(:,i-1),'ExecutionEnvironment','cpu');

end

Where XTrain is an input variable with 5 predictors, with the following form

[11.0666700000000;37.7043300000000;3.43672900000000;67.3333300000000;13.0603600000000]

The error: "Incorrect network state. The network expects mini-batches size of 5, but was passed a mini-batch of size 1"

My Code:

%% To train

ET_1 = xlsread(filename1,'F:F');

X_input_1 = xlsread(filename1,'A:E');

XTrain = (tonndata(X_input_1,false,false))'; %input 1

TTrain = (tonndata(ET_1,false,false))'; %expected output 1

%% To predict

ET_2 = xlsread(filename2,'F:F');

X_input_2 = xlsread(filename2,'A:E');

XTest = (tonndata(X_input_2,false,false))'; %input 2

TTest = (tonndata(ET_2,false,false))'; %expected output 2

numFeatures = 5;

numResponses = 1;

numHiddenUnits = 100;

layers = [ ...

sequenceInputLayer(numFeatures)

lstmLayer(numHiddenUnits,'OutputMode','sequence')

fullyConnectedLayer(numResponses)

regressionLayer];

%% Specify the training options

options = trainingOptions('adam', ...

'MaxEpochs',1000, ...

'GradientThreshold',1, ...

'InitialLearnRate',0.005, ...

'LearnRateSchedule','piecewise', ...

'LearnRateDropPeriod',125, ...

'LearnRateDropFactor',0.2, ...

'Verbose',0, ...

'Plots','training-progress');

%% Train LSTM Network

net = trainNetwork(XTrain,TTrain,layers,options);

%% Preparo to predict future

net = predictAndUpdateState(net,XTest);

%% Error

[net,YPred] = predictAndUpdateState(net,XTest(end));

numTimeStepsTest = numel(XTrain_2);

for i = 2:numTimeStepsTest

[net,YPred(:,i)] = predictAndUpdateState(net,YPred(:,i-1),'ExecutionEnvironment','cpu');

end

XTrain and XTest (predictors) are arrays of 8x1 and TTrain and TTest (expected outputs) 1x1

Ideas???

Answer 3

[ahmed shahin]

Dear Marcelo Olmedo,

did you solve this issue?

thanks

shahin

[yu shang]

Dear shahin

did you solve this issue?