Convolutional LSTM (C-LSTM) in MATLAB

121 vues (au cours des 30 derniers jours)
Jake
Jake le 9 Oct 2018
Modifié(e) : Dieter Mayer le 29 Août 2022
I'd like to train a convolutional neural network with an LSTM layer on the end of it. Similar to what was done in:
  1. https://arxiv.org/pdf/1710.03804.pdf
  2. https://arxiv.org/pdf/1612.01079.pdf
Is this possible?

Réponses (5)

Shounak Mitra
Shounak Mitra le 9 Oct 2018
Hi Jake,
Unfortunately, we do not directly support C-LSTM. We are working on it and it should be available soon.
-- Shounak
  7 commentaires
David Willingham
David Willingham le 26 Août 2022
Hi Dieter,
Apologies for not updating this answers post sooner. This workflow is now supported. the following code will illustrated this:
% Load data
[XTrain,YTrain] = japaneseVowelsTrainData;
% Define layers
layers = [ sequenceInputLayer(12,'Normalization','none', 'MinLength', 9);
convolution1dLayer(3, 16)
batchNormalizationLayer()
reluLayer()
maxPooling1dLayer(2)
convolution1dLayer(5, 32)
batchNormalizationLayer()
reluLayer()
averagePooling1dLayer(2)
lstmLayer(100, 'OutputMode', 'last')
fullyConnectedLayer(9)
softmaxLayer()
classificationLayer()];
options = trainingOptions('adam', ...
'MaxEpochs',10, ...
'MiniBatchSize',27, ...
'SequenceLength','longest');
% Train network
net = trainNetwork(XTrain,YTrain,layers,options);
Dieter Mayer
Dieter Mayer le 29 Août 2022
Modifié(e) : Dieter Mayer le 29 Août 2022
Hi David,
Thanks for your reply! Is this workflow shows a real convolution LSTM (LSTM carries out convolutional operations instead of matrix multiplication) and is not only implied to a input matrix, which is a result of a convolution net work applied before?
Sorry for asking that, I have to learn the syntax of using the deep learning toolbox, I am a beginner. The background is, that I will use such a Conv-LSTM to make precipitation forecasts for grids bases on precipitation radar inputs from several timesteps of the last minutes / hours as discussed in this paper publication

Connectez-vous pour commenter.


Yi Wei
Yi Wei le 17 Déc 2019
Hi, can matlab support C-LSTM now?
  5 commentaires
ytzhak goussha
ytzhak goussha le 23 Fév 2021
Hey,
Sorry I didn't follow this thread and didn't see the questions.
Here is a simplified C-LSTM network.
The input it a 4D image (height x width x channgle x time)
The input type is sqeuntial.
When you need to put CNN segments, you simply unfold->CNN->Fold->flatten and feed to LSTM layer.
Ioana Cretu
Ioana Cretu le 18 Mai 2021
Hi! When I try to train the model I have this error:
Error using trainNetwork (line 170)
Invalid network.
Caused by:
Layer 'fold': Unconnected output. Each layer output must be connected to the input of another layer.
Detected unconnected outputs:
output 'miniBatchSize'
Layer 'unfold': Unconnected input. Each layer input must be connected to the output of another layer.
I connected the layers using this:
lgraph = layerGraph(Layers);
lgraph = connectLayers(lgraph,'fold/miniBatchSize','unfold/miniBatchSize');
What do you think the cause is?

Connectez-vous pour commenter.


Chen
Chen le 25 Août 2021
Please refer to this excellent example in:
It is possible to train the hybrid together.

Jonathan
Jonathan le 4 Août 2022
inputSize = [28 28 1];
filterSize = 5;
numFilters = 20;
numHiddenUnits = 200;
numClasses = 10;
layers = [ ...
sequenceInputLayer(inputSize,'Name','input')
sequenceFoldingLayer('Name','fold')
convolution2dLayer(filterSize,numFilters,'Name','conv')
batchNormalizationLayer('Name','bn')
reluLayer('Name','relu')
sequenceUnfoldingLayer('Name','unfold')
flattenLayer('Name','flatten')
lstmLayer(numHiddenUnits,'OutputMode','last','Name','lstm')
fullyConnectedLayer(numClasses, 'Name','fc')
softmaxLayer('Name','softmax')
classificationLayer('Name','classification')];
lgraph = layerGraph(layers);
lgraph = connectLayers(lgraph,'fold/miniBatchSize','unfold/miniBatchSize');

David Willingham
David Willingham le 26 Août 2022
Updating this answer. This workflow has been supported since R2021. The following example illustrates how to combin CNN's with LSTM layers:
% Load data
[XTrain,YTrain] = japaneseVowelsTrainData;
% Define layers
layers = [ sequenceInputLayer(12,'Normalization','none', 'MinLength', 9);
convolution1dLayer(3, 16)
batchNormalizationLayer()
reluLayer()
maxPooling1dLayer(2)
convolution1dLayer(5, 32)
batchNormalizationLayer()
reluLayer()
averagePooling1dLayer(2)
lstmLayer(100, 'OutputMode', 'last')
fullyConnectedLayer(9)
softmaxLayer()
classificationLayer()];
options = trainingOptions('adam', ...
'MaxEpochs',10, ...
'MiniBatchSize',27, ...
'SequenceLength','longest');
% Train network
net = trainNetwork(XTrain,YTrain,layers,options);

Catégories

En savoir plus sur Sequence and Numeric Feature Data Workflows dans Help Center et File Exchange

Produits


Version

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by