Question for 'Layer Input sizes' in custom neural network layers

According to the 'checkLayer' document, there is a description about 'validInputSize' and I am confused about the below description.

"For layers with a single input, specify validInputSize as a vector of integers corresponding to the dimensions of the input data."

what does 'single input' means? Does it means a really signle scalor? like x1? or vector like where ?

According to the 'checkLayer' document, 5 types of layer input are introduced for intermediate layer.

what I want is to make a intermediate layer having 2 input nodes. So, my input is .

In this case, I guess I need to use 'Vector sequences' and input size should be 2 by N by 1, where N is the number of observations.

(two features and each feature is just scalar)

(By the way, does 'observation' means batch size? I'm confused about therminology in Matlab)

Let's say N = 50. Then, input size is 2 by 50 by 1. So, when I validate my layer I need to set

validInputsize = [2 50 1];

If I write code like this, is this a 'single input'?

If it is a 'single input', what is 'multiple inputs'?

'multiple inputs' are used when my custom layer accept two different type of input?

The above figure showing 'two inputs' (if I am correct), one input is 2 by 1 vector like my case I explained in the above, the other input is 2 by 2 amplitude image (black & white). Am I correct?

Lastly, 'fullyConnectedLayer' has 'InputSize' and 'OutputSize' properties. Are they having different definition to input size in a custom layer? What value I choose in 'fullyConnectedLayer' it seems to me just 'one input' and 'one input' according to the definition in a custom layer.

Answer 1

[Uday Pradhan]

Hi Sangmin,

It is my understanding that you want to create an intermediate layer which accepts a 2 – by – 1 vector as input. Since you have posed multiple questions, I will try to answer them one by one:

1. “ What does 'single input' means? Does it means a really single scalar? like x1?”

• No. Single input means the layer expects inputs of a pre – defined size. It may be a 28 x 28 x 3 RGB image or a 780 x 1 row vector. Multiple inputs mean inputs can be of varying sizes. This may happen when a layer is connected to multiple previous layers which have different output dimensions.

2.“By the way, does 'observation' means batch size? I'm confused about terminology in Matlab”

• Observations in this context refer to the number of inputs to the neural network. When we talk about an epoch, this can also mean the size of the mini – batch.

3.“What I want is to make an intermediate layer having 2 input nodes (two features (x1,x2) and each feature is just scalar). I guess I need to use 'Vector sequences' and input size should be 2 by N by 1, where N is the number of observations."

• You can use analyzeNetwork function to see the output of each layer and how they are connected. Once you find out the output of the layer prior to the custom layer you are designing, which according to your description may be 1x2 or 2x1, you may take

validInputsize = [2 1 1] (or [1 2 1]); 

We don’t need to specify the value of N. If you want to check if your layer can process multiple observations at once you can use:

checkLayer(layer, validInputSize, ’ObservationDimension’, 4); 

The example you have given is correct. In this case the custom intermediate layer accepts “multiple” inputs.

Lastly, for the fully Connected Layers, we don’t need to specify the input size (it is automatically determined). If we use:

Layer1 = fullyConnectedLayer(10); 

Then the output of this layer is a “1 x 1 x 10” matrix. As an example, if we have say a “maxpool” layer whose output dimension is “12 x 12 x 20” before our fully connected “Layer1” , then Layer1 decides the output as follows:

Output of Layer1 is calculated as W*X + b where X has size 2880 x 1 and W and b are of sizes 10 x 2880 and 10 x 1 respectively. The output of “maxpool” layer is flattened to form the column vector X.

Hope this helps.

Answer 2

[debojit sharma]

For preparing the type of table shown in the attached figure, how can i get the values for the last column of the table i.e Input Size for each layer in my CNN model?