Why is the error thrown: Inputs to concatenations layer must have consistent formats.

lgraph = layerGraph();

tempLayers = imageInputLayer([90 90 3],"Name","imageinput_1");

lgraph = addLayers(lgraph,tempLayers);

tempLayers = imageInputLayer([90 90 3],"Name","imageinput_2");

lgraph = addLayers(lgraph,tempLayers);

tempLayers = imageInputLayer([90 90 3],"Name","imageinput_3");

lgraph = addLayers(lgraph,tempLayers);

tempLayers = [

depthConcatenationLayer(3,"Name","depthcat")

convolution2dLayer([3 3],32,"Name","conv_1","Padding","same")

batchNormalizationLayer("Name","batchnorm_1")

reluLayer("Name","relu_1")

maxPooling2dLayer([5 5],"Name","maxpool","Padding","same")

convolution2dLayer([3 3],32,"Name","conv_2","Padding","same")

batchNormalizationLayer("Name","batchnorm_2")

reluLayer("Name","relu_2")

fullyConnectedLayer(3,"Name","fc")];

lgraph = addLayers(lgraph,tempLayers);

tempLayers = [

sequenceInputLayer(1,"Name","sequence")

lstmLayer(128,"Name","lstm")

dropoutLayer(0.5,"Name","dropout")

fullyConnectedLayer(3,"Name","fc1")];

lgraph = addLayers(lgraph,tempLayers);

tempLayers = [

concatenationLayer(1,2,"Name","concat")

softmaxLayer("Name","softmax")

classificationLayer("Name","classoutput")];

lgraph = addLayers(lgraph,tempLayers);

% clean up helper variable

clear tempLayers;

lgraph = connectLayers(lgraph,"imageinput_1","depthcat/in1");

lgraph = connectLayers(lgraph,"imageinput_2","depthcat/in2");

lgraph = connectLayers(lgraph,"imageinput_3","depthcat/in3");

lgraph = connectLayers(lgraph,"fc","concat/in1");

lgraph = connectLayers(lgraph,"fc1","concat/in2");

plot(lgraph);

Answer 1

[Venu]

Hi @Andrey Puchkov,

I understand that you are concatenating two fully connected layers along 1st dimension based on this line "concatenationLayer(1,2,"Name","concat")". To ensure that the concatenation works along the first dimension, you need to ensure that the dimensions of the tensors from "fc" and "fc1" are consistent along this dimension. Given your current configuration:

• "fc" has dimensions 1(S) x 1(S) x 3(C) x 1(B)
• "fc1" has dimensions 3(C) x 1(B) x 1(T)

To make these dimensions consistent for concatenation along the first dimension, you should reshape the tensors from "fc" and "fc1" such that their first dimensions match. For example, you could reshape "fc" to have dimensions 3(C) x 1(S) x 1(S) x 1(B) and "fc1" to have dimensions 3(C) x 1(B) x 1(T) x 1(S) to ensure consistency along the first dimension.

To reshape the tensors "fc" and "fc1" to ensure consistency along the first dimension, you can use "permute" function followed by "reshape" functions in MATLAB. You can reshape them as shown below:

For "fc" with dimensions 1(S) x 1(S) x 3(C) x 1(B),

fc_permuted = permute(fc, [3, 1, 2, 4]); %change order of dim
fc_reshaped = reshape(fc_reshaped, [3, 1, 1, 1]); %ensure consistency

For "fc1" with dimensions 3(C) x 1(B) x 1(T),

fc1_permuted = permute(fc1, [1, 4, 2, 3]); %change the order dim
fc1_reshaped = reshape(fc1_reshaped, [3, 1, 1, 1]); %ensure consistency

You can refer to these documentations for more clarity:

https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.concatenationlayer.html

https://www.mathworks.com/help/matlab/ref/permute.html

https://www.mathworks.com/help/matlab/ref/reshape.html

Hope this helps!

[Andrey Puchkov]

Hi, Venu,

Yes, the answer was useful! Thank you very much!

I didn't know about the permute function. Before this answer, I used a flattenLayer layer, but it only worked when the output of the LSTM network had a dimension of 1.