How to create nnet.layers usable for custom training loops as well as trainNetwork?

Hei all,

I would like to create a layer that can be pretrained in a custom training loop and afterwards finetuned with the trainNetwork function. I created a class with the needed properties and a predict function:

function Z = predict(layer, X)
    Z = sigmoid(fullyconnect(X, layer.weights, layer.biases));
end

Using this function works fine during the pretraining. Using trainNetwork afterwards leads to the error:

   Error using validateDataFormat (line 14)
   Specify 'DataFormat' when first argument is an unlabeled dlarray or numeric array.
   

This is caused by fullyconnect as it needs dlarrays or the correct format string. I try to make it as universal as possible, so i don't want to set the Format property on the fullyconnect command. Is there a way to force trainNetwork to use dlarrays?

Thanks in advance,

Niklas

Answer 1

[Sai Bhargav Avula]

Hi,

As you mentioned the fullyconnect applies fully connect operation to dlarray data. For your custom layer to be made compactable to trainNetwork you should convert that using the fullyConnetedLayer by following template.

Or

You should finetune it by converting it into a dlnetwork.