Using LSTM with a featureinputplayer

While reading the lstm and its relative docs, i noticed that a LSTMLayer always connected after a SequenceInputLayer.

When i using lstm with a featureinputlayer and my input data is a 2-D matrix ( shape = (data_length, num_features)), I found that the network can still run. So my question is : will matlab treats it as ONE sequence with ${data_length} tokens. Or $[{data_length} sequences (sequence_length = 1)?

Thanks in advanced for your help!

[Haoming]

In addition, how can i apply tbptt into lstm using matlab

Answer 1

[Gayathri]

Hi @Haoming,

When you use an LSTM layer in MATLAB with a “featureInputLayer” and your input data is a 2-D matrix with shape (data_length, num_features), MATLAB treats the input as one sequence with data_length tokens. For more understanding on which layer to use with LSTMs consider the below points.

• · LSTMs are used to handle long term dependencies. So, LSTM would be used when you have data with some sort of “time ordering”.
• · “sequenceInputLayer” is typically used when your input data is inherently sequential, such as time series data or sentences in NLP tasks. It expects data in a 3-D format: (sequence_length, num_features, num_sequences).
• · For time series data, you can't swap the order of timesteps - this changes the meaning of your data. This is the reason why we use “sequenceInputLayer” before “lstmLayer”.

“featureInputLayer” is generally used for non-sequential data. However, when used with an LSTM, MATLAB can treat the data as a single sequence by default.

For more information on the “sequenceInputLayer” and “featureInputLayer” please refer to the below mentioned links.

https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.featureinputlayer.html

https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.sequenceinputlayer.html

Hope you find this information helpful.

[Haoming]

Thanks! It helps me a lot.