Precision/Recall (perfcurve)

116 vues (au cours des 30 derniers jours)
Baium
Baium le 18 Oct 2019
Modifié(e) : Baium le 23 Oct 2019
Hello,
My naive question is about the precision and recall rates that can be output from the perfcurve function.
As you may already know, this would look like:
[X, Y] = perfcurve(labels, score(:, 2), 'LowFlow', 'XCrit', 'prec', 'YCrit', 'reca');
X and Y, however, are vectors. Normally, what is reported in the literature is a single value. My question is, to get the precision/recall estimates, should I take the mean of the non-NaN values from X (= precision) and the mean of the non-NaN values from Y (= recall) or is there another computation involved into getting a single value that represents these rates?
Thank you for your help.
  1 commentaire
Baium
Baium le 21 Oct 2019
Bump, if I may.

Connectez-vous pour commenter.

Réponse acceptée

Athul Prakash
Athul Prakash le 23 Oct 2019
Modifié(e) : Athul Prakash le 23 Oct 2019
perfcurve() is meant for computing performance curves for your model - such as PR curves or ROC curves. In the literature, you may find these curves used for a more rigorous examination of a model's performance than can be given by a single score.
I recommend looking up the doc linked below. The Algorithms > Pointwise Confidence Bounds section explains what goes on under-the-hood after you call the function with your particular arguments. It samples from your input data multiple times, with different thresholds, to come up with a range of P/R scores, which are used to plot the curve.
To calculate a single value, you may extract counts of TP, FP, TN & FN from your labels and predictions; then manually compute -
Precision = TP / (TP + FP)
Recall = TP / (TP + FN)
Should be done in 2 lines of code.
Alternatively, you may use confmat() to produce a confusion matrix, to readily give you TP/FP/TN/FN counts. And then do the same computation.
%Using this method, here's an anonymous function to calculate precision.
% << for an existing Confision Matrix 'ConfusionMat' >>
precision = @(confusionMat) diag(confusionMat)./sum(confusionMat,2);
% And another for recall
recall = @(confusionMat) diag(confusionMat)./sum(confusionMat,1)';
% Computes a vector of P/R values - each value corresponds to selecting a different label as +ve.
Hope it Helps!
  1 commentaire
Baium
Baium le 23 Oct 2019
Modifié(e) : Baium le 23 Oct 2019
Thank you very much, I ended up manually computing the rates, but I appreciate the function handles you have posted!
P.S. If I'm not mistaken, your precision and recall functions are inverted.

Connectez-vous pour commenter.

Plus de réponses (0)

Produits


Version

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by