Customizing the code generated from the Classification Learner App

I am trying to customize the "Weighted kNN"-based classification code generated after a 10-fold cross validation on my data using the Classification Learner App (Using the Generate Function option in the app).

I have 2 queries realted to this!

Query 1:

partitionedModel = crossval(trainedClassifier.ClassificationKNN, 'KFold',10);

After generating the code, I found that the above code is responsible for partitioning the data. I have an imbalanced dataset and I believe that "Stratified" k-fold cross validation ensures equal amount of data is considered from each class.

Query: Does the above code automatically perform Stratified k-fold cross validation? If not, how do I include it in the code?

Query 2:

classificationKNN = fitcknn(...
    predictors, ...
    response, ...
    'Distance', 'Cosine', ...
    'Exponent', [], ...
    'NumNeighbors', 19, ...
    'DistanceWeight', 'SquaredInverse', ...
    'Standardize', false, ...
    'ClassNames', [0; 1; 2]);

The above code shows the classifier parameters. Among them I want to use a Bhattacharya Distance measure (I have used cosine in the code) based on my data. Unfortunately, this distance is not available within the various options.

Query: How to create a custom distance measure for the above?

Kindly help me out with these queries. Thank you in advance.

Answer 1

[Akshat]

Hi Tintumon,

I gather that in the first query, you'd like to find out how to implement Stratified k-fold cross validation. Then, you'd like to find out how to make a distance calculator by yourself.

Now, for the first part, I could not find anything that automatically startifies the dataset in the 'crossval' function's arguments. On the other hand, you can give it a 'cvpartition' object, which is basically how the data is split. You can refer this for more about the 'Partition' argument in 'crossval' here: https://www.mathworks.com/help/stats/crossval.html#mw_e1a8945e-7fad-4fa9-93cb-518884197429

You might want to use the 'cvpartition' function, to get the above object. While doing this, you can set the 'Stratify' flag to true. More information is here: https://www.mathworks.com/help/stats/cvpartition.html#mw_319d54b6-08be-4b1d-b444-97ab793fb03c

The updated code might look something like this:

cvp = cvpartition(response, 'KFold', 10, 'Stratify', true);
partitionedModel = crossval(trainedClassifier.ClassificationKNN, 'Partition', cvp);

Coming to the next query, you can define the custom distance function as a matlab function, and then update the code in the following way to use it as the distance function. Please make sure that the inputs and outputs are consistent with those in the MathWorks given distance functions.

classificationKNN = fitcknn(...
    predictors, ...
    response, ...
    'Distance', @customDistance, ... % Use your custom distance function
    'NumNeighbors', 19, ...
    'DistanceWeight', 'SquaredInverse', ...
    'Standardize', false, ...
    'ClassNames', [0; 1; 2]);

function dists = customDistance(X1, X2)
    % Replace this with your actual Bhattacharyya distance calculation
end

Hope this helps!