perfcurve ROC analysis: dealing with inverse contrast

I am using perfcurve to generate ROC curves for imaging data in which we expect tumor tissue (set as 1 or the positive class) to light up more than normal tissue (set as 0 or the negative class). However, sometimes the images display negative contrast and the normal tissue lights up more than the tumor tissue. In these cases, the perfcurve function is unbiased to which tissue is brighter (i.e. the positive or negative class) and spits out "positive" ROC curves with areas-under-the-curves that are greater than 0.5.

Another interesting phenomenon is that sometimes when the expected trend is there the ROC curves come out negative with AUCs less than 0.5. I am assuming this happens because the perfcurve function sees the normal tissue as the expected brighter signal.

Is there a way to explicitly specify which class needs to be greater than the other class for the diagnosis to be positive?

Answer 1

[Image Analyst]

I'm not sure what you're sending in to perfcurve but it seems like the scores are what's bad and those are determined by your segmentation algorithm. So you need to fix your tumor segmentation algorithm first. If you're getting a bad label/estimate, then it's your segmentation algorithm's fault. I don't think it has anything to do with perfcurve() itself.

[CeeDubbya]

Thanks for the answer. The scores are determined using fitglm, with a binomial distribution and logit link function. See the code below for an example:

    meas = ROC_vals(:,1);
    known = logical(ROC_vals(:,2));
    mdl = fitglm(meas,known,'Distribution','binomial','Link','logit');
    scores = mdl.Fitted.Probability;
    [X,Y,T,AUC,OPTROCPT] = perfcurve(known,scores,true);

Is there a better segmentation algorithm for this application (i.e. imaging data)? The data displays roughly a binomial distribution when plotted on a histogram.

[Image Analyst]

What image properties are you measuring, like tumor area or brightness or something? I don't know what ROC_vals is. You can't use ROC values as your measurement because you don't have an ROC curve yet. You need to change something in your segmentation algorithm first, then get a true positive ratio, then change the thing again and get a new ratio, and so on until you have enough points on your ROC curve to make one.

[CeeDubbya]

ROC_vals is a list of pixel intensity values taken from user defined regions of the image corresponding to tumor and normal tissue, with column 2 containing 1's next to values taken from tumor and 0's next to values taken from normal tissue.