Unrecognized function or variable 'transformerLayer'

Good evening, I'm trying to simulate a transformer network that evaluates the gain improvement in a TNT network. In the training script I use 'transformerLayer' but at runtime I get the following error:

Unrecognized function or variable 'transformerLayer'.

The Matlab version is R2024b, the Deep Learning Toolbox (ver. 24.2) installed correctly, but the script always stops on this function.

The script code is as follows:

%% 1) Load SINR data
load('sinr_data.mat'); % Load sinr and sampling_rate

%% 2) Prepare the sequences
sequence_length = 50;
num_sequences = length(sinr) - sequence_length;
input2D = zeros(num_sequences, sequence_length);
target = zeros(num_sequences,1);

for i = 1:num_sequences
    input2D(i,:) = sinr(i:i+sequence_length-1);
    target(s) = sinr(i+sequence_length);
end

% For Transformer, sequences must be in 3D format: [1 x 50 x num_sequences]
X = reshape(input2D', [1, sequence_length, num_sequences]);

%% 3) Define the transformer network
layers = [ ...
    sequenceInputLayer(1) % 1 feature per timestep
    transformerLayer(64, 'OutputMode', 'last') % Transformer with 64 units, scalar output per sequence
    fullyConnectedLayer(1) % Project to 1 value
    regressionLayer % Regression
    ];

%% 4) Training options
options = trainingOptions('adam', ...
    'MaxEpochs',20, ...
    'MiniBatchSize',32, ...
    'Shuffle','every-epoch', ...
    'Plots','training-progress');

%% 5) Train
net = trainNetwork(X,target,layers,options);

%% 6) Save
save('trained_transformer.mat','net');
clear;

Thank you in advance for your answers

Roberto.

Answer 1

[Matt J]

I don't see transformerLayer in the documentation anywhere, so I imagine it does not exist, and that you are probabaly running an incomplete fragment of some 3rd party code.

Answer 2

[Walter Roberson]

transformerLayer is not supplied by Mathworks.

You need something like https://github.com/malkhodari/Transformer_MATLAB/blob/main/transformerLayer.m