the squential feature selection with Sequentialfs function

Hi all,

I'm having trouble with feature selection; could someone please assist me?

My code is:

X = trainingFeatures;
y = trainingLabels;
c = cvpartition(y,'k',10);
opts = statset('display','iter');
fun = @(XT,yT,Xt,yt)...
     (sum(~strcmp(yt,classify(Xt,XT,yT,'quadratic'))));
[fs,history] = sequentialfs(fun,X,y,'cv',c,'options',opts);

When we run the code, we get:

Answer 1

[Abhaya]

Hi @lebed toufik,

The error "The covariance matrix of each group in TRAINING must be positive definite." indicates that the covariance matrix used for training data is not definite. This can be due to highly correlated training data. To resolve this issue, you can follow the steps given below.

1. Identify Highly Correlated Features: You can use MATLAB 'corr' function to calculate the correlation matrix and identify features that are highly correlated.
2. Remove Correlated Features: For each pair of features with a correlation above a specified threshold, remove one feature to reduce redundancy.

Please refer to the code given below for better understanding.

corrMatrix = corr(X);  % Compute the correlation matrix of your features
threshold = 0.9;        % Set the correlation threshold

% Find the pairs of features with correlation above the threshold
[rows, cols] = find(abs(corrMatrix) > threshold);

% Remove one feature from each pair of correlated features
X_reduced = X;
for i = 1:length(rows)
    if rows(i) < cols(i)
        X_reduced(:, cols(i)) = [];  % Remove the column from X
    end
end

For more information, please refer to the following MATLAB documentation on 'corr' function.

https://www.mathworks.com/help/stats/corr.html