Troubles with Experiment Manager Setup for LSTM regression

I need help!

I am managing Experiment Manager with a very simple exercise. I use data stocks for one-step ahead of the close price with a simple LSTM net. The net works well with trainnest(XTest,YTest,layers,options). I want to calibrate the Epochs parameters with Experiment Manager. I followed any suggestions and tutorials on the web but I can not do it. Xtest is an array with 6 cols (open, close, volum etc.) and 2401 (time steps) rows for a given stock. The reponse is a single 2401 vector containing one-time shifted of the close price. As said. I have no problem with trainnet and the net performs well.

I put data, layers and options in the script for Experiment Manager. Here is my code:

function [XTrain_N,YTrain_N,layers,options] = Experiment1_setup1(params)
load dati_net.mat XTrain_N YTrain_N
num_features = 6;
num_responses = 1;
num_hidden_units = 350;
layers = [
    featureInputLayer(6);
    lstmLayer(num_hidden_units, 'OutputMode','last')
    fullyConnectedLayer(num_responses)
    ];
%Training Options
options = trainingOptions("adam", ...
    MaxEpochs=params.MaxEpochs, ...
    SequencePaddingDirection="left",...
    InitialLearnRate=0.001,...
    Shuffle="every-epoch", ...
    ValidationFrequency=50, ...
    GradientThreshold=.93, ...
    L2Regularization=0.00001, ...
    Verbose=false, ...
    Metrics="rmse", ...
    Plots="training-progress");
end

But when I run the experiment I always get:

The following errors occurred while running the experiment:

Errors occurred while validating the setup function: Caused by: Invalid output arguments from setup function. Third-from-last output of setup function must be a layer array or dlnetwork object, but a value of type 'double' was detected.

I tried dozens of trials but I am able to escape the problem. I attach also my data

Thanks in Advance!

[dpb]

The function as shown doesn't return any of the return variables...

[massimo giannini]

Ok so how can I fix it?

Answer 1

[Jaimin]

Hi @massimo giannini,

According to the issue description, you are able to train your model using the“trainnet”function. However, when attempting to tune the epoch hyperparameter value using MATLAB's Experiment Manager app, you encounter an error.

One workaround I found is to use“trainNetwork”instead of“trainnet”in Experiment Manager. To do this

1. Click on "New" -> "Project" -> "Blank Project"
2. Select the 'Built-In Training' experiment from the 'Blank Built-In trainNetwork Experiment' as shown in the image below.
3. After selecting that, configure the parameters as shown in the image below.
4. Set all other parameters according to requirements.

Additionally, there is a modification in"Experiment1_setup1". Here is the updated code.

function [XTrain_N, YTrain_N, layers, options] = SequenceRegressionExperiment_setup2(params)
    load dati_net.mat XTrain_N YTrain_N
    num_features = 6;
    num_responses = 1;
    num_hidden_units = 350;
    layers = [
        featureInputLayer(6);
        lstmLayer(num_hidden_units, 'OutputMode','last')
        fullyConnectedLayer(num_responses)
        regressionLayer
        ];
    %Training Options
    options = trainingOptions("adam", ...
        MaxEpochs=params.MaxEpochs, ...
        SequencePaddingDirection="left",...
        InitialLearnRate=0.001,...
        Shuffle="every-epoch", ...
        ValidationFrequency=50, ...
        GradientThreshold=.93, ...
        L2Regularization=0.00001, ...
        Verbose=false, ...
        Plots="training-progress");
end

I have attached some resources that will be helpful to you.

Difference between “trainnet” and “trainNetwork”: https://www.mathworks.com/matlabcentral/answers/2060029-difference-between-trainnet-and-trainnetwork

Experiment Manager: https://www.mathworks.com/help/deeplearning/ref/experimentmanager-app.html#mw_92d8f99c-f283-4bc5-a5d7-7d779c4831f7

Training Configurations for LSTM: https://www.mathworks.com/help/deeplearning/ug/experiment-with-training-configurations-for-sequence-regression.html

I hope this helps!

[massimo giannini]

Great! Thank you!!

I found a second working alternative yesterday by "try and guess". I converted XTrain_N YTrain_N to a datastore:

load dati_net.mat XTrain_N YTrain_N
adsXTrain = arrayDatastore(XTrain_N);
adsYTrain = arrayDatastore(YTrain_N);
cdsTrain = combine(adsXTrain,adsYTrain);

, erased Metrics from the options and used this string for the function:

function [cdsTrain,layers,lossFcn,options] = Experiment1_setup1(params)
lossFcn="mse";

and now Experiment Manager works.

Anyway thanks for your valuable help