"The value of 'Layer' is invalid. Layers that require formatted dlarray inputs are not supported." - Why is this error shown when checked the validity of a custom layer?

I be created a customized layer based on attention mechanism for deep learning application using https://in.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layer.html. But while checking the validity of layer using the following code:

layer=CoAtten(Name="atten");
validInputSize = [1 14 1024];
layout = networkDataLayout(validInputSize,"CBT");
layer = initialize(layer,layout);
checkLayer(layer,validInputSize,ObservationDimension=3)

The last line (checkLayer) shows the error like this- ""The value of 'Layer' is invalid. Layers that require formatted dlarray inputs are not supported."

What this error shows? Is the custom layer is invalid? I have used stripdims inside the predict fuction while creating the layer. Is it due to this?

Answer 1

[Venu]

Hi @BIPIN SAMUEL,

The error while using checkLayer "The value of 'Layer' is invalid. Layers that require formatted dlarray inputs are not supported" occurs when the custom layer's predict function does not handle the dlarray inputs correctly. Your inference is correct. In your case, when using "stripdims", it's important to ensure that the input dlarray aligns with the expected input format. This includes verifying the dimensions and data type of input dlarray.

Find the documentation below for dlarray, checkLayer and stripdims to ensure that your custom layer's predict function aligns with the expected input format.

https://www.mathworks.com/help/deeplearning/ref/checklayer.html

https://www.mathworks.com/help/deeplearning/ref/dlarray.html

https://www.mathworks.com/help/deeplearning/ref/dlarray.stripdims.html

Hope this helps!

[BIPIN SAMUEL]

Thank You @Venu I have rectified the issue. While training it is properly working and giving results, but after saving the trained network when I tried to load the saved network "Unable to load instances of class CoAtten into a heterogeneous array." this error is showing. 'CoAtten' is my customized layer. Why it is happening even when it is working properly during training?

[Venu]

Hi @BIPIN SAMUEL,

The error "Unable to load instances of class CoAtten into a heterogeneous array" occurs when the custom layer "CoAtten" is not recognized during the loading process. During training, the layer is used within the context of the MATLAB environment, and it works as expected because it's part of the current session. However, when the trained network is saved and then loaded into a different session, the environment may not recognize the custom layer "CoAtten" unless it has been properly registered.

To address this, you should ensure that the custom layer "CoAtten" is properly registered and available when loading the trained network. This typically involves saving the layer's class definition and ensuring it is properly loaded before attempting to load the trained network.

You can refer to the same documentation that you have mentioned in your question to ensure proper registration and loading of custom layers.

[BIPIN SAMUEL]

Thank you @Venu Is it anything to do with the initialize() function while creating the custom layer as in https://in.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layer-with-formatted-inputs.html. But I have used the initialize() function inside the program for the learnable parameters.

[Matt J]

No, it is not related to the initialize() function. It is related to where your CoAtten.m classdef file is located. Somehow, you have put it in a place where Matlab cannot see it.

[BIPIN SAMUEL]

Thank You @Matt J but when I tried to load the network from another system the error is shown as "The initialize threw an error". This custom layer is created to work in that particular part of the network. How can I rectify this problem? Because for testing different signals I need to train the network again for many hours. If I will be able to save that network properly, I can use it for testing purpose any time. How can I modify the program that helps Matlab to find the layer?

[Venu]

Hi @BIPIN SAMUEL,

Make sure that the "CoAtten.m" file is included with the saved network and is on the MATLAB path or in the current working directory when loading the network on any system. Check the initialize() function of your "CoAtten" class to ensure that it does not rely on external variables or states that might not be available when loading the network. The method should be self-contained.

You can try adding error handling in the initialize() function to catch any issues.

[BIPIN SAMUEL]

Thank You @Venu; now it worked for me.