LSTM Sequence to Sequence Regression Array Type Problem

Hello, I am studying LSTM Sequence to Sequence Time Series Regression for Indoor Environmental.

When I read MATLAB Documentation for "trainNetwork", I had a problem about array type

following is the MATLAB Documentation text

----

net = trainNetwork(sequences,Y,layers,options) trains a recurrent network (for example, an LSTM or GRU network) for the sequence data specified by sequences and responses specified by Y.

(about Sequence)

Sequence or time series data, specified as an N-by-1 cell array of numeric arrays, where N is the number of observations, or a numeric array representing a single sequence.

For cell array or numeric array input, the dimensions of the numeric arrays containing the sequences depend on the type of data.

(about Y)

N-by-1 cell array of numeric sequences, where N is the number of sequences. The sequences are matrices with R rows, where R is the number of responses. Each sequence must have the same number of time steps as the corresponding predictor sequence.

For sequence-to-sequence regression tasks with one observation, sequences can be a matrix. In this case, Y must be a matrix of responses.

----

The Problems are

1. if my Input data(Array X) has multiple input variable(x1, x2, x3 ,... xn) for Single Response y, Must I change my X's Type to cell?
2. I had tried MATLAB Example, but I didn't understand diffrences in 2 case(Mat to Mat, Cell to Cell)

thank you!

[Ji Hyeon Cho]

X1 = linspace(1,100,100);
X2 = linspace(100,199,100);
X3 = linspace(200,299,100);
Y = linspace(300,399,100);

X1 = (X1 - min(X1)) / (max(X1)-min(X1));
X2 = (X2 - min(X2)) / (max(X2)-min(X2));
X3 = (X3 - min(X3)) / (max(X3)-min(X3));

X_MAT = [X1;X2;X3];
X_Cell = con2seq(X_MAT,100);

Y_MAT = Y;
Y_Cell = con2seq(Y,100);

numFeatures = 3;
numResponses = 1;
numHiddenUnits = 400;

layers = [ ...
    sequenceInputLayer(numFeatures)
    lstmLayer(numHiddenUnits)
    fullyConnectedLayer(numResponses)
    regressionLayer];

options = trainingOptions('adam', ...
    'MaxEpochs',100, ...
    'GradientThreshold',1, ...
    'InitialLearnRate',0.5, ...
    'LearnRateSchedule','piecewise', ...
    'LearnRateDropPeriod',125, ...
    'LearnRateDropFactor',0.2, ...
    'Verbose',1, ...
    'Plots','training-progress');

net_Mat = trainNetwork(X_MAT,Y_MAT,layers,options);
net_Cell = trainNetwork(X_Cell,Y_Cell,layers,options);

for i = 1: 100
   [net_Mat,YpredMat(:,i)] =  predictAndUpdateState(net_Mat,X_MAT(:,i));
end

for i = 1: 100
   [net_Cell,YpredCell(:,i)] =  predictAndUpdateState(net_Cell,X_Cell(:,i));
end


CVRMSE_MAT = sqrt(abs(mean(YpredMat.^2-Y_MAT.^2)))/mean(Y_MAT) * 100;

YpredCell = cell2mat(YpredCell);
Y_Cell = cell2mat(Y_Cell);
CVRMSE_Cell = sqrt(abs(mean(YpredCell.^2-Y_Cell.^2)))/mean(Y_Cell) * 100;

Answer 1

[Divya Gaddipati]

Hi,

I suggest you check the examples on sequence-to-sequence regression that are available in the documentation. Here's the link:

https://www.mathworks.com/help/deeplearning/deep-learning-with-time-series-sequences-and-text.html?s_tid=CRUX_lftnav

To answer your questions:

No. It doesn't necessarily have to be in cell format. In the case where your input data has sequences of different length, then you should be having the data in cells. As matrix doesn't support sequences of different lengths.

[Ji Hyeon Cho]

Thank you Divya!

Your answer is so helpful!