Grid search in svm

Hi,

I am having training data (train.mat) and testing data (test.mat), I need to perform grid search in this. How can I do svm training with this? Kindly help me with this. Thanks in advance.

[Walter Roberson]

What is to be searched by the grid method?

[SP]

I need to do svm hyperparameter tuning using grid search method.

Answer 1

[Walter Roberson]

Nbox = 15;    %change these as desired
Nkern = 10;   %change these as desired

bcvec = logspace(-3, 3, Nbox);
ksvec = logspace(-3, 3, Nkern);

goodness = zeros(Nbox, Nkern);
for ksidx = 1 : Nkern
    ks = ksvec(ksidx);
    for bcidx = 1 : Nbox
        bc = bcvec(bcidx);
        mdl = fitcsvm(Training_Data, Training_Class, ...
            'Standardize', true, ...
            'KernelFunction','rbf', ...
            'BoxConstraint', bc, ...
            'KernelScale', ks);
        predictions = predict(mdl, Test_Data);
        correct_matches = nnz(predictions == Test_Class);
        goodness(bcidx, ksidx) = correct_matches;
    end
end

best_score = max(goodness(:));
[bcidx, ksidx] = find(goodness == best_score);
best_bc = bcvec(bcidx);
best_ks = ksvec(ksidx);

best_fits = table(best_bc, best_ks, 'VariableNames', {'BoxConstraint', 'KernelScale'});

The output will be a table, best_fits, listing all of the combinations of BoxConstraint and KernelScale that together lead to the best scores discovered, where "score" here is determined [in this code] solely by the number of correct matches.

You would have to change how the goodness is calculated if you wanted to be concerned about aspects such as balancing the accuracies of the various classes. With the current code, if it was (for example) 100% successful in matching the largest class, and 0% successful in all of the other classes, then the goodness calculated by this code might be as high as it gets, even though it might be terrible for the other classes. You would probably be better off calculating the goodness some other way.

[SP]

I can't understand this code. Could you please mention which are the train test files here.

[Walter Roberson]

temp = load('train.mat');
Training_Data = temp.VariableThatContainsTrainingData;
Training_Class = temp.VariableThatContainsTrainingClass;

temp = load('test.mat');
Test_Data = temp.VariableThatContainsTestData;
Test_Class = temp.VariableThatContainsTestClass;

I do not know the variable names in your train.mat and test.mat so I had to fill in placeholders like VariableThatContainsTrainingData .

Also it is not uncommon that the data and class information are combined, often with the class information being the last column of the data. In that case it might look more like

temp = load('train.mat');
Training_Data = temp.VariableThatContainsData(:,1:end-1);
Training_Class = temp.VariableThatContainsData(:,end);

temp = load('test.mat');
Test_Data = temp.VariableThatContainsData(:,1:end-1);
Test_Class = temp.VariableThatContainsData(:,end);

The rest of the code is a straight-forward double-nested loop, going through all defined combinations of BoxConstraint and KernelScale, and looking for the combination that produces the best score.

The only thing unusual about the code is that it would be common for people to assume that there is only one combination that produces the maximum score, but I programmed in the possibility that several combinations each produce the maximum score.

It is not clear to me why you do not just turn on hyperparameter optimization and let the built-in optimization code take care of the situation.

[SP]

I am getting following error message:

Error using classreg.learning.internal.ClassLabel (line 30)

You must pass class labels as a vector.

Error in ClassificationSVM.prepareData (line 627)

allClassNames = levels(classreg.learning.internal.ClassLabel(Y));

Error in classreg.learning.FitTemplate/fit (line 246)

this.PrepareData(X,Y,this.BaseFitObjectArgs{:});

Error in ClassificationSVM.fit (line 240)

this = fit(temp,X,Y);

Error in fitcsvm (line 342)

obj = ClassificationSVM.fit(X,Y,RemainingArgs{:});

[Walter Roberson]

I do not know what code you ended up using, and I do not know what is stored in your mat files

[SP]

I have attached my mat files. It consists of matrix values. The mat files consists of 30 columns in which first 15 columns (1 to 15 columns) belongs to Class_1 and rest of the 15 columns (16 to 30 columns) belongs to Class_2.

[Walter Roberson]

temp = load('train.mat');
Training_Data1 = temp.finaltrain(:,1:end/2);
Training_Data2 = temp.finaltrain(:,end/2+1:end);
Training_Data = [Training_Data1; Training_Data2];
Training_Class = [ones(size(Training_Data1, 1), 1); 2*ones(size(Training_Data2,1),1)];


temp = load('test.mat');
Test_Data1 = temp.finaltest(:,1:end/2);
Test_Data2 = temp.finaltest(:,end/2+1:end);
Test_Data = [Test_Data1; Test_Data2];
Test_Class = [ones(size(Test_Data1, 1), 1); 2*ones(size(Test_Data2,1),1)];


Nbox = 25;    %change these as desired
Nkern = 20;   %change these as desired

bcvec = logspace(-3, 3, Nbox);
ksvec = logspace(-3, 3, Nkern);

goodness = zeros(Nbox, Nkern);

wb = waitbar(0, 'please wait, processing kernel scales');
cleanMe = onCleanup(@() delete(wb));

for ksidx = 1 : Nkern
    waitbar(ksidx/Nkern, wb);

    ks = ksvec(ksidx);
    for bcidx = 1 : Nbox
        bc = bcvec(bcidx);
        mdl = fitcsvm(Training_Data, Training_Class, ...
            'Standardize', true, ...
            'KernelFunction','rbf', ...
            'BoxConstraint', bc, ...
            'KernelScale', ks);
        predictions = predict(mdl, Test_Data);
        correct_matches = nnz(predictions == Test_Class);
        goodness(bcidx, ksidx) = correct_matches;
    end
end

clear cleanMe  %get rid of waitbar

best_score = max(goodness(:));
[bcidx, ksidx] = find(goodness == best_score);
best_bc = bcvec(bcidx);
best_ks = ksvec(ksidx);

best_fits = table(best_bc, best_ks, 'VariableNames', {'BoxConstraint', 'KernelScale'});

best_fits