How do I output hidden layers to a custom loss function in a regularized autoencoder?

I am creating a regularized autoencoder wherein the latent dimension outputs the results of a regression task while the decoder reconstructs the input image. I would like the network to output the results of the latent layer and the image reconstruction to a mean-squared-error loss function. The documentation for variational autoencoders suggests using a custom training loop, but I am concerned about debugging this custom loop on top of the custom layers I need to implement to tie the encoder and decoder weights. Surely an architecture that is over a decade old has been integrated into the dlnetwork, custom loss, and trainnet functionalities? Is there some way to handle the output in a custom loss function which can then be passed to the trainnet function?

I was anticipating syntax like outputs = outputLayer(numberOfOutputs, Name='out') which could then be added and tied to the model with net = addLayers(net, outputs); net = connectLayers(net, 'encoded', 'out/in1'); net = connectLayers(net, 'reconstruction', 'out/in2). The tensorflow models of this type return a list of outputs, and the multiple output documentation suggests there is similar functionality somewhere in MatLab's deep learning tools.

Answer 1

[Umar]

Hi Joseph,

To address your concerns about debugging a custom training loop and implementing custom layers while utilizing the functionality of dlarray, custom loss, and trainNetwork in MATLAB, it's important to understand that MATLAB's deep learning capabilities are indeed versatile and can accommodate complex architectures like the one you are describing.

In your case, where you want the network to output both the results of the latent layer and the image reconstruction to a mean-squared-error loss function, you can achieve this by defining a custom loss function that takes into account both outputs. You can then pass this custom loss function to the trainNetwork function along with your model architecture.

Regarding handling multiple outputs in a custom loss function, you can utilize MATLAB's dlarray data structure to manage the outputs efficiently. By structuring your outputs appropriately within dlarrays, you can calculate the mean squared error loss between the latent layer output and the reconstructed image output seamlessly.

As for integrating custom layers into your model architecture, you can follow a similar approach to what you anticipated in terms of syntax. You can define your custom output layer using something like 'outputLayer' with the desired number of outputs and then add and connect this layer to your model using 'addLayers' and 'connectLayers' functions respectively.

Hopefully, following these steps will help resolve your problems. Please let me know if you have further questions.

[Joseph Conroy]

Thank you for the clarification.