MATLAB Answers


Precision/Recall (perfcurve)

Asked by Baium
on 18 Oct 2019
Latest activity Edited by Baium
on 23 Oct 2019
My naive question is about the precision and recall rates that can be output from the perfcurve function.
As you may already know, this would look like:
[X, Y] = perfcurve(labels, score(:, 2), 'LowFlow', 'XCrit', 'prec', 'YCrit', 'reca');
X and Y, however, are vectors. Normally, what is reported in the literature is a single value. My question is, to get the precision/recall estimates, should I take the mean of the non-NaN values from X (= precision) and the mean of the non-NaN values from Y (= recall) or is there another computation involved into getting a single value that represents these rates?
Thank you for your help.

  1 Comment

on 21 Oct 2019
Bump, if I may.

Sign in to comment.




1 Answer

Answer by Athul Prakash on 23 Oct 2019
Edited by Athul Prakash on 23 Oct 2019
 Accepted Answer

perfcurve() is meant for computing performance curves for your model - such as PR curves or ROC curves. In the literature, you may find these curves used for a more rigorous examination of a model's performance than can be given by a single score.
I recommend looking up the doc linked below. The Algorithms > Pointwise Confidence Bounds section explains what goes on under-the-hood after you call the function with your particular arguments. It samples from your input data multiple times, with different thresholds, to come up with a range of P/R scores, which are used to plot the curve.
To calculate a single value, you may extract counts of TP, FP, TN & FN from your labels and predictions; then manually compute -
Precision = TP / (TP + FP)
Recall = TP / (TP + FN)
Should be done in 2 lines of code.
Alternatively, you may use confmat() to produce a confusion matrix, to readily give you TP/FP/TN/FN counts. And then do the same computation.
%Using this method, here's an anonymous function to calculate precision.
% << for an existing Confision Matrix 'ConfusionMat' >>
precision = @(confusionMat) diag(confusionMat)./sum(confusionMat,2);
% And another for recall
recall = @(confusionMat) diag(confusionMat)./sum(confusionMat,1)';
% Computes a vector of P/R values - each value corresponds to selecting a different label as +ve.
Hope it Helps!

  1 Comment

on 23 Oct 2019
Thank you very much, I ended up manually computing the rates, but I appreciate the function handles you have posted!
P.S. If I'm not mistaken, your precision and recall functions are inverted.

Sign in to comment.