How do you code Garson's algorithm in matlab to find the relative importance of parameters when training a neural network model?

I am working on a predictive analysis study and came across garson's algorithm but I am facing trouble in programming the formula for it. Here is my code:

%%% Data Inputs

inputs = data(:, 1:4)';

targets = data(:, 5)';

%%% Create a feedforward neural network

net = feedforwardnet([10, 15, 20]);

%%%% Set the number of neurons in each hidden layer

net.layers{1}.size = 10;

net.layers{2}.size = 15;

net.layers{3}.size = 20;

%%%% Change activation functions (e.g., 'tansig' to 'purelin' for output layer)

net.layers{1}.transferFcn = 'tansig'; % Activation function for first hidden layer

net.layers{2}.transferFcn = 'tansig'; % Activation function for second hidden layer

net.layers{3}.transferFcn = 'tansig'; % Activation function for third hidden layer

net.layers{4}.transferFcn = 'purelin'; % Activation function for output layer

%%%% Divide data for training, validation, and testing

net.divideParam.trainRatio = 0.7;

net.divideParam.valRatio = 0.15;

net.divideParam.testRatio = 0.15;

%%%% Train the neural network

[net, tr] = train(net, inputs, targets);

%%%% Calculate Garson's importance

importance = garsonsAlgorithm(net);

%%%% Display feature importances

disp('Feature Importances (Garson''s Algorithm):');

for i = 1:length(importance)

disp(['Importance for x' num2str(i) ': ' num2str(importance(i))]);

end

Here is the garson's algorithm function that I wrote down:

function importance = garsonsAlgorithm(net)

%%% Extract connection weights from the neural network

weights = cell2mat(net.IW);

% Check if there are bias weights

if isfield(net, 'b')

bias_weights = cell2mat(net.b);

weights = [weights; bias_weights];

end

%%% Calculate the absolute values of the weights

absolute_weights = abs(weights);

%%% Calculate the importance for each input feature

importance = sum(absolute_weights, 1);

%%% Normalize importance values

importance = importance / sum(importance);

end

I was wondering if this is correct or not since I am new to programming. Also, would there be any sort of variation of outputs (relative importance) when changing the number of inputs in the garson's algorithm?

I have attached the formula for Garson's algorithm for reference:

Answer 1

[Debraj Maji]

Hi @Haris Saeed,

I understand that you are trying to code Garson's Algorithm in MATLAB. I have implemented the function below for your reference. Please note that this implementation assumes a single output neuron. If your network has multiple output neurons, you'll need to adjust the algorithm accordingly. The importance values are printed as percentages, indicating the relative importance of each input variable to the network's output.

function garsonsAlgorithm(net)
    IW = net.IW{1}; % Input-to-hidden weights
    LW = net.LW{2,1}; % Hidden-to-output weights
    % Number of input and hidden neurons
    numInputs = size(IW, 2);
    numHidden = size(IW, 1);
    % Calculate the connection strengths
    hiddenLayerWeights = abs(IW);
    outputLayerWeights = abs(LW);
    % Calculate the relative importance (RI) for each input neuron to each hidden neuron
    RI = hiddenLayerWeights ./ sum(hiddenLayerWeights, 2);
    % Calculate the contribution of each hidden neuron to the output neuron
    contribution = RI .* outputLayerWeights';
    % Normalize contribution for each input to sum to 1
    inputImportance = contribution ./ sum(contribution, 1);
    
    % Display the importance of each input variable
    for i = 1:numInputs
        fprintf('Input %d importance: %.2f%%\n', i, 100 * inputImportance(i));
    end
end

Given below is the code I used to test the function created above:

X = rand(10, 100); % 10 input features, 100 samples
T = rand(1, 100);  % 1 output, 100 samples

% Create and train a feedforward neural network with one hidden layer
net = feedforwardnet(5); % 5 hidden neurons
net = train(net, X, T);
% Apply Garson's algorithm to the trained network
garsonsAlgorithm(net);

The results so obtained were as follows:

Secondly, changing the number of inputs to a neural network can affect the relative importance scores calculated by Garson's algorithm due to weight redistribution.

I hope this helps,

With regards,

Debraj.

[Haris Saeed]

Thank you mate for your response. This has solved a big problem for me