Using CTC los function with trainnet

Hi! I am designing a neural network to do phoneme classifcation for a simple speech recogniser. I've been evaluating the idea of using Connectionist Temporal Classifcation (CTC) as the loss function for this task, as other loss functions haven't shown good results. Both my training and validation data are structured as an array datastore containing MFCC matrixs and the corresponding phoneme label to each matrix.

I've first tried by using a custom loop as the Matlab page suggests for this task, but I've had many problems developing the code, so now I've been trying to implement the CTC loss function into 'trainnet' this way:

numClasses = numel(unique(letras));
% Definir la arquitectura de la red neuronal
layers = [
    sequenceInputLayer(39, "Normalization","zscore")
    convolution1dLayer(15,20,'Padding', 'same')
    tanhLayer
    convolution1dLayer(10,20,'Padding','same')
    tanhLayer
    dropoutLayer(0.4)
    convolution1dLayer(5,20,'Padding','same')
    tanhLayer
    globalAveragePooling1dLayer
    fullyConnectedLayer(numClasses)
    softmaxLayer];
% Definir opciones de entrenamiento
options = trainingOptions('adam', ...
    'MaxEpochs',50, ...
    'MiniBatchSize',64, ...
    'ValidationData', mfcc_dts_validate, ...
    'Plots','training-progress',...
    'Metrics', 'accuracy');
% Entrenar la red neuronal
[net, info] = trainnet(mfcc_dts_train, layers, @modelLoss, options);

Where the model loss function is written as if it was meant to be used with a custom loop:

function [loss,gradients,state] = modelLoss(net,X,T)
Y = predict(net, X);
loss = ctc(Y,T);
% Calculate gradients of loss with respect to learnable parameters.
gradients = dlgradient(loss,net.Learnables);
end

The error I obtain is:

Error using trainnet

Caused by:

Undefined function 'predict' for input arguments of type 'dlarray'.

I'm not quite sure of how to use a custom loss function into trainnet properly, as I don't understand how to use the different inputs and outputs during training as the input data for the custom loss function. I've tried following the different tutorials for custom loops and loss functions, but I haven't seen an example code for this case. If someone has an idea of how to face this, let me know!

Answer 1

[Ayush Aniket]

Hi Bruno,

The error indicates that the input arguments to the 'predict' function does not expect a 'dlarray' object.

The syntax you are using for defining the custom loss function with the 'trainnet' function is not correct. As you have also mentioned, it follows the syntax required when you are training your model within a custom training loop.

As the 'ctc' loss function has 'dlarray' support, you can use the functions directly and the software determines the gradients automatically using automatic differentiation. Refer to the following documentation links to read about defining custom operations for deep learning and the list of functions that have 'dlarray' support:

• https://www.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-operations.html
• https://www.mathworks.com/help/deeplearning/ug/list-of-functions-with-dlarray-support.html

Hence, the correct syntax would be as shown below:

function loss = modelLoss(Y,T)
    loss = ctc(Y,T);
end

or as an anonymous function:

@(Y,T) ctc(Y,T)

Refer to the section about loss functions used in the 'trainnet' MATLAB function below for the syntax:

https://www.mathworks.com/help/deeplearning/ref/trainnet.html#mw_ffa5eeae-b6e0-444e-a464-91e257cef95b

[Bruno Nicolás Addiego Rodríguez]

Thanks a lot! This was really helpful!