Setting inputs formats for nested Neural ODE

Hi all,

I am constructing a NN that nests a Neural ODE. The NN has two datasets as inputs: i) The initial values of internal states (InitialValue) that are used to feed the Neural ODE and, ii) The sequences (input_2) that are included in the NN after the Neural ODE. The input_2 and the outputs of the Neural ODE must be summed.

I tried creating the entries of the NN in cell and dlarray format. For the last one I also defined the dimensions 'CBT' and/or 'CB' according the structure of the dataserie, nevertheles the problem persists.

The error I got is the following

%% Generate data
data_1 = randn(1,1000);
data_2 = randn(1,1000);
tspan = 1:1:50;
InitialValue = data_1(:,1:end-length(tspan))';
indices = 1:length(InitialValue);
targets = arrayfun(@(i) data_1(:, i + tspan), indices, 'UniformOutput', false)';
input_2 = arrayfun(@(i) data_2(:, i + tspan), indices, 'UniformOutput', false)';
%% Create neuralnetwork
% NeuralODE layers
OdeLayer = [fullyConnectedLayer(5)
    tanhLayer
    fullyConnectedLayer(1)];
OdeNet = dlnetwork(OdeLayer,Initialize=false);
% Main layer
net = dlnetwork;
Layers =[featureInputLayer(1,'Name','Input 1')
    neuralODELayer(OdeNet,tspan,"Name",'OdeLayer','GradientMode','adjoint')];
% add extra input for adition to NeuralODE output
net = addLayers(net, Layers);
net = addLayers(net, sequenceInputLayer(1,'Name','Input 2'));
net = addLayers(net, additionLayer(2,'Name','adition'));
% connect layers
net = connectLayers(net,'Input 2','adition/in2');
net = connectLayers(net,'OdeLayer','adition/in1');
%% Train Network
% gather inputs and targets
input_1_ds = arrayDatastore(InitialValue,"OutputType","same");
input_2_ds = arrayDatastore(input_2,"OutputType","same");
target_ds = arrayDatastore(targets,"OutputType","same");
cds = combine(input_1_ds, input_2_ds, target_ds);
opt = trainingOptions("adam");
% training
net = trainnet(cds,net,"l2loss",opt);

Training stopped: Error occurred

Error using trainnet (line 54)
Layer "Input 2": Invalid input data. Invalid size of channel dimension. Layer expects input with channel dimension size 1 but received input with size 50.

Thanks in advance for your feedback and comments.

Answer 1

[Binaya]

Hello Sergio,

Based on the code and error message you have shared, it appears that the "Input 2" layer is expecting an input of dimension 1, but it is receiving an input of dimension 50. To resolve this input dimension mismatch, you can modify line 21 as follows:

net = addLayers(net, sequenceInputLayer(50,'Name','Input 2'));

Here, first argument to "sequenceInputLayer" defines the input to the should have a dimension of 50. This change should address the input dimension issue.

Additionally, you might encounter another error about mismatch in the predictions and target dimensions. To fix this, you can use a "globalAveragePooling1dLayer" followed by a "fullyConnectedLayer" to ensure the dimensions align correctly.

Refer to the following documentation links for more details:

1. globalAveragePooling1dLayer: https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.globalaveragepooling1dlayer.html
2. fullyConnectedLayer: https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.fullyconnectedlayer.html

Regards.

[Sergio]

Hi @Binaya, thank you for your answer. Maybe how I set the format of input_2 is not so clear so let me expand a bit the explanation about the dimensions of input_2 and targets.

input_2 and targets have the same dimensions which are: 950 sequences with 50 values each. All of these datasets were obtained from the datasets data_1 and data_2. In other words input_2 and targets have only 1 channel, 950 sequences that represent the minibatches and the 50 values that have each sequence represent the time span. The dimensions of each variable are (1(C) * 950(B) * 50(T)).

If I set the sequenceinput layer to 50 I am giving 50 channels to the NN and this is what I am not looking for.

I created the code following the suggestions presented in the following links

1. https://nl.mathworks.com/matlabcentral/answers/1635140-neural-ode-for-dynamic-systems-with-input-signals
2. https://nl.mathworks.com/matlabcentral/answers/1742320-size-of-the-input-layer-is-different-from-the-expected-input-size

[Binaya]

Hello Sergio,

According to the inputs defined, I was getting the following error when only modified line number 21:

Caused by:
Error using validateTrueValues (line 54)
Size of predictions and targets must match.
Size of predictions:
50(C) × 128(B) × 49(T)
Size of targets:
50(C) × 128(B) × 1(T)

This suggests that, target is of shape: "50(C) × 128(B) × 1(T)".

[Binaya]

Hi Sergio,

Based on your requirements, I made a few changes to your code and maintained the target size to have 1 channel with 50 values that represent the time span.

Please find the following changes to your code:

1. Modify the "targets" and "input_2" definition to have single channel values representing the timespan.

targets = arrayfun(@(i) data_1(:, i + tspan)', indices, 'UniformOutput', false)';
input_2 = arrayfun(@(i) data_2(:, i + tspan)', indices, 'UniformOutput', false)';

2. Modify the "tspan" for "neuralODELayer" function to output 50 values representing the timespan. The provided code definition returned 49 values which led to dimension mismatch.

tspan_2 = 1:1:51;
Layers =[featureInputLayer(1,'Name','Input 1')
    neuralODELayer(OdeNet,tspan_2,"Name",'OdeLayer','GradientMode','adjoint')];

The code runs with the above changes and meets the dimension requirements.

Hope this helps.

[Sergio]

Hi @Binaya,

Thank you very much for your help, it is true, the code worked with your suggested changes. I just added a single quote mark in the section data_1(:, i + tspan)' like you did in the section data_2(:, i + tspan)'

Sergio