Effacer les filtres
Effacer les filtres

How to obtain a ROC curve through cross validation on the out of fold data in cross validation?

10 vues (au cours des 30 derniers jours)
I am using fitcsvm and need to obtain ROC curve for the fold that is not used in training.
Here is the code:
classificationSVM = fitcsvm(...
predictors, ...
response, ...
'KernelFunction', 'linear', ...
'PolynomialOrder', [], ...
'BoxConstraint', 1, ...
'Standardize', true, ...
'ClassNames', categorical(classnames_fitcsvm),'KFold',10);
[fitPosteriorSVM,ScoreTransform] = fitPosterior(classificationSVM);
for the prediction step:
I can not use resubPredict because that is set for training data and when I use kfoldPredict I recive error. I need to find the predicted score for the fold that is not used for training in 10 fold cross validation to be able to run perfcurve.
Any idea?

Réponses (1)

Divya Gaddipati
Divya Gaddipati le 30 Déc 2020
Currently, you have to split the data into training and validation manually to generate results on validation set.
To use 10-fold cross-validation, you can fit the model on 90% of the data, and compute results for the remaining 10% of data which was not used for fitting. You can then loop over each of the 10 subsets to plot the ROC curves for individual runs.
Here is a small example on how to do that:
k = 10;
cvFolds = crossvalind('Kfold', labels, k);
for i = 1:k
testIdx = (cvFolds == i); % get indices of test instances
trainIdx = ~testIdx; % get indices training instances
classificationSVM = fitcsvm(data_features(trainIdx,:), labels(trainIdx), ...
'KernelFunction', 'linear', ...
'PolynomialOrder', [], ...
'BoxConstraint', 1, ...
'Standardize', true);
[predictions, score] = predict(svmModel, data_features(testIdx,:));
[X,Y] = perfcurve(labels(trainIdx), predictions,'versicolor');
xlabel('False positive rate'); ylabel('True positive rate')
title('ROC curve')
Hope this helps!


En savoir plus sur Statistics and Machine Learning Toolbox dans Help Center et File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by