training-progress plots not available

Hi all,

I'm training a dlnetwork and want to plot the accuracy, FScore and Loss as it trains. However only Loss is showing as follows:

My training options are as follows:

options = trainingOptions("sgdm",...

LearnRateSchedule="piecewise",...

LearnRateDropPeriod=6,...

LearnRateDropFactor=0.1,...

Momentum=0.9,...

InitialLearnRate=1e-2,...

L2Regularization=0.005,...

ValidationData=dsVal,...

MaxEpochs=18,...

MiniBatchSize=4,...

Metrics = ["accuracy","fscore"], ...

Shuffle="every-epoch",...

CheckpointPath=tempdir,...

VerboseFrequency=10,...

ValidationPatience=4, ...

Plots="training-progress");

[Adarsh]

Hi @Lily,

Sharing the code and the model would be helpful in accurate identification of the issue. Alternatively, you can try implementing the custom "trainingProgressMonitor" as shown in the documentation link below and check if it works for you:

https://www.mathworks.com/help/releases/R2024b/deeplearning/ref/deep.trainingprogressmonitor.html

[Lily]

Hi @Adarsh

Thank you for your reply. Here is the code taken from https://uk.mathworks.com/help/deeplearning/ug/semantic-segmentation-using-deep-learning.html

As I understand, the training-progress should plot the metrics I have specified in training options? But my training finished after 16 epochs and only plotted the Loss.

Still quite new to MATLAB and Image Segmentation so any help or suggestions greatly appreciated!

Lily

[imdsTrain, imdsVal, imdsTest, pxdsTrain, pxdsVal, pxdsTest] = partitionCamVidData(imds,pxds); %randomly split the image and pixel label data into a training, validation and test set.

numTrainingImages = numel(imdsTrain.Files)
numValImages = numel(imdsVal.Files)
numTestingImages = numel(imdsTest.Files)

dsVal = combine(imdsVal,pxdsVal);%Define validation data.

Data Augmentation
dsTrain = combine(imdsTrain,pxdsTrain); %combine pixel and image dataset to then transorm

xTrans = [-10 10];
yTrans = [-10 10];
dsTrain = transform(dsTrain, @(data)augmentImageAndLabel(data,xTrans,yTrans));


Create the Network
imageSize = [720 960 3]; %Specify the network image size.

numClasses = numel(classes); %Specify the number of classes.

network = deeplabv3plus(imageSize,numClasses,"resnet18");%reate a DeepLab v3+ network based on ResNet-18

Balance Classes Using Class Weighting
imageFreq = tbl.PixelCount ./ tbl.ImagePixelCount;
classWeights = median(imageFreq) ./ imageFreq;

Select Training Options
options = trainingOptions("sgdm",...
    LearnRateSchedule="piecewise",...
    LearnRateDropPeriod=6,...              %The learning rate is reduced
    LearnRateDropFactor=0.1,...            %by a factor of 0.1 every 6 epochs
    Momentum=0.9,...
    InitialLearnRate=1e-2,...
    L2Regularization=0.005,...
    ValidationData=dsVal,...
    MaxEpochs=18,...  
    MiniBatchSize=4,...
    Metrics = ["accuracy","fscore"], ...
    Shuffle="every-epoch",...
    CheckpointPath=tempdir,...
    VerboseFrequency=10,...
    ValidationPatience=4, ...
    Plots="training-progress");

%metric = accuracyMetric(AverageType="macro")

Start Training
doTraining = true; %o train the network, set to true.
if doTraining
    [net,info] = trainnet(dsTrain,network,@(Y,T) modelLoss(Y,T,classWeights),options);
end

[colordepth]

Please share the values of the metrics recorded in your 'info' variable. I wonder if they are being stored as NaN for some reason.

tHistory = info.TrainingHistory;  % This table contains the recorded training metrics.
disp(tHistory.TrainingAccuracy);

I cannot imagine why it would contain NaN values, or why the accuracy would not be calculated at each training iteration. This may be worth an investigation though, please share the output of the above commands.

Answer 1

[Adarsh]

Hi @Lily,

I understand that you are following the “https://uk.mathworks.com/help/deeplearning/ug/semantic-segmentation-using-deep-learning.html”

Example to train a segmentation model in MATLAB and facing issues with the “Metric” attribute in the “trainingOptions” object.

However, in a segmentation model, the outputs are pixel-wise results other than just class-based probabilities that are expected in a general classification problem.

From what I found in the documentation, “accuracyMetric” is designed for general classification tasks not for pixel-wise classification tasks, So when the output of the model is passed for calculation of accuracy it receives an pixel-wise array of 3 channel outputs rather than a single array of class-based probabilities, hence due to this inconsistency the Accuracy and F1-Score plots are empty.

For this reason, the metrics for segmentation models is computed in a different manner which is shown in the example that you are following.

In the example, “semanticseg” function is used to run the inference and it returns a “pixelLabelDatastore” object which is then passed to “evaluateSemanticSegmentation” function for the computation of the Metrics required.

The “evaluateSemanticSegmentation” returns the metrics such as “accuracy”, “bfscore”, “global-accuracy”, “iou” and “weighted-iou”.

For more information, you can refer to the following documentation links:

1. https://www.mathworks.com/help/releases/R2024b/deeplearning/ref/deep.metric.accuracymetric.html
2. https://www.mathworks.com/help/releases/R2024b/vision/ref/semanticsegmentationmetrics.html
3. https://www.mathworks.com/help/releases/R2024b/vision/ref/pixellabeldatastore.html
4. https://www.mathworks.com/help/releases/R2024b/vision/ref/evaluatesemanticsegmentation.html

I hope this helps!