Convolutional Autoencoder for Image Enhancement not working

Hello, I have a class project where I am to use deep learning to enhance waveform images. I am curently trying to setup a convolutional autoencoder however, when it runs the deeplearning model doesn't seem to learn.

Below is an image of what I am trying to achieve:

The goal is to highlight strong features that are in the image.

The current model is setup to take in a 128 by 128 waveform image, go thorugh 5 convolutional layers which are then transposed afterwards. The output is then put into a regression layer.

The training and validation data is composed of a combined datastore that has unenhanced and enhanced images. Below is a preview of the training data.

This is the code that I am using for the deeplearning model.

inputSize = [128 128];
epochs = 50;
iLRate = .1;    %InitialLearnRate
lFactor = 0.1;  %LearnRateDropFactor
l2reg = 0.0001; %L2Regularization

checkpointPath = "C:\ModelCheckpoints";

enhancedTrainingData = imageDatastore("C:\TrainingData\Enhanced\");
unenhancedTrainingData = imageDatastore("C:\TrainingData\Unenhanced\");
dsTrain = randomPatchExtractionDatastore(unenhancedTrainingData,enhancedTrainingData,inputSize,"PatchesPerImage",45);

enhancedValData = imageDatastore("C:\ValidationData\Enhanced\");
unenhancedValData = imageDatastore("C:\ValidationData\Unenhanced\");
dsVal = randomPatchExtractionDatastore(unenhancedValData,enhancedValData,inputSize,"PatchesPerImage",10);

imageLayer = imageInputLayer(inputSize);

encodingLayers = [ ...
    convolution2dLayer(3,256,Padding="same"), ...
    reluLayer, ...
    maxPooling2dLayer(2,Padding="same",Stride=2), ...
    convolution2dLayer(3,256,Padding="same"), ...
    reluLayer, ...
    maxPooling2dLayer(2,Padding="same",Stride=2), ...
    convolution2dLayer(3,128,Padding="same"), ...
    reluLayer, ...
    maxPooling2dLayer(2,Padding="same",Stride=2),...
    convolution2dLayer(3,64,Padding="same"), ...
    reluLayer, ...
    maxPooling2dLayer(2,Padding="same",Stride=2),...
    convolution2dLayer(3,32,Padding="same"), ...
    reluLayer, ...
    maxPooling2dLayer(2,Padding="same",Stride=2)];

decodingLayers = [ ...
    transposedConv2dLayer(2,32,Stride=2), ...
    reluLayer, ...
    transposedConv2dLayer(2,64,Stride=2), ...
    reluLayer, ...
    transposedConv2dLayer(2,128,Stride=2), ...
    reluLayer, ...
    transposedConv2dLayer(2,256,Stride=2), ...
    reluLayer, ...
    transposedConv2dLayer(2,256,Stride=2), ...
    reluLayer, ...
    convolution2dLayer(1,1,Padding="same"), ...
    clippedReluLayer(1.0), ...
    regressionLayer];    

layers = [imageLayer,encodingLayers,decodingLayers];

options = trainingOptions("adam", ...
    "MaxEpochs",epochs, ...
    "Plots","training-progress", ...
    "InitialLearnRate",iLRate, ...
    "LearnRateDropFactor",lFactor, ...
    "L2Regularization",l2reg, ...
    "LearnRateDropPeriod",10, ...
    "ValidationData",dsVal, ...
    "LearnRateSchedule",'piecewise', ...
    "ValidationPatience",100, ...
    "CheckpointPath",checkpointPath, ...
    "GradientThresholdMethod","l2norm", ...
    "GradientThreshold",0.01, ...
    "MiniBatchSize",100);

waveEnhancement = trainNetwork(dsTrain,layers,options)

Below is an image of the training progress. I have done multiple trainings with different learning rates, epochs, and the rmse has not dropped below 6000.

Why does my deeplearning model fail to learn and am I missing something for my approach or made an error in the model I have setup?

Answer 1

[Nathan]

Hi Marion,

I know it has been a while, but did you fix the problem ?