evaluateDe​tectionPre​cision error when using my own dataset

I am trying to use evaluateDetectionPrecision for an iteration of a multiclass RCNN that I have previously made with my own data. I am receiving the following error:

Error using vision.internal.detector.evaluationInputValidation>checkDetectionResultsTable (line 75) The label value in row 1 of the detection results table is undefined. It must be one of the following categorical values: {Warship,Mil_Aircraft}.

In my detection results, the categorical values used for labels are indeed 'Warship' and 'Mil_Aircraft'.

I'm not sure where I am going wrong in adapting the DeepLearningFasterRCNNObjectDetectionExample code for my data.

[Walter Roberson]

And what does the label value of row 1 of the detection results table look like instead?

[ejmoss1]

I have the same issue. The results and the expected results are the same. Each includes three rows of the bonding boxes, relative scores and also labels.

I would apreciate some help to know how I can proceed.

[Brayan Alfonso]

I have the same problem, checking in the results table, i have the value that the program asks me.

I appreciate any help.

[OIPA]

The same error for me. I have checked and the label of the detection result is the defined one...

Has anyone solved this problem?

[Udari De Alwis]

Having the same issue with multi class labels. Appreciate if anyone can share a successful solution for this problem ?

Answer 1

[Anirudh TOPIWALA]

Even I have the same problem. Can anyone please help?

[Walter Roberson]

Could you indicate what we would need to do in order to reproduce this?

Answer 2

[Sara Perez]

Hi!

I have the same problem. It works fine for calculating the accuracy of a single class (where ground truth does not need label information) as the example shown in the documentation.

But for several categories I got the error. "The label value in row 1 of the detection results table is undefined. It must be one of the following categorical values: {car,people}."

I just comment the error message line of matlab internal function (line 80 in evaluationInputValidation.m) and it seems to work fine and return the accuracy scores. More concretely this line of code:

error(message('vision:ObjectDetector:undefinedLabelInDetectionTable', i, msg));

The internal function can be found in: MATLAB\R2018b\toolbox\vision\vision\+vision\+internal\+detector

Hope it helps!

Sara.

Answer 3

[gyenist]

Hello!

The problem is that the input sanity checker for the builtin evalution function gets the ground truth classes from only the first row of the gt table like this:

classes = categories(data{1,2});

So if the detection results have different classes from the first gt row the above error is displayed. If you comment the error message as suggested by @Sara Perez the error will no longer diplay, but the evalution score calculation will only be done for the classes in the first gt row.

Other workarounds to get no error and scores for all classes (in 2020b at least) are

1. to use different table format for your input data as mentioned in https://uk.mathworks.com/help/vision/ref/evaluatedetectionprecision.html - Input Arguments
2. to change the code that gets the class ctegories in line 15 of MATLAB\R2020b\toolbox\vision\vision\+vision\+internal\+detector\evaluationInputValidation.m to

classes = categories(vertcat(data{:,2}));

P.S: I'm not sure if this is already fixed in Matlab 2021 versions. Hope I could help!

Best,

gyenist