Presiction time in Classification Lerner

I would like to mesure the time that classifier takes for prediction. I see that in Classification Lerner toolbox there is parameter prediction speed obs/sec. Can anyone explain what does the obs/sec mean? How can I convert it in to time? Is it possible to calculate prediction time only for correct classifications?

Answer 1

[Aditya Patil]

obs/sec refers to number of observations processed per second. It's inverse would be the time taken for one prediction in seconds.

To calculate time taken for only the correct predictions, you can pass only the correctly predicted observations to the predict function, and calculate the time it takes for it to complete. You can find this option under Editor -> Run -> Run and Time

[EK]

Thank you for your reply. I would like to know if I can do it within classification learner toolbox. I do not see how to select only correctly predicted observation and calculate classification time on them. I am using 2019b version

[Aditya Patil]

Currently, it's not possible to do so using the app. Can you elaborate on the use case? Generally for most models, there won't be any difference in time between correct and incorrect predictions. Even for models where it exists, it should be insignificant.

[EK]

I have a several tasks with the different degree of difficulty. I would like to access the task difficulty by mesuaring accuracy and classification speed of the LDA classifier. As expected when task difficulty increased the accuracy of the classifier decreased and the training time increased. I would expect the same relationship in prediction speed. But prediction speed was not different in all condition. I thought that it is because in the test I have many trials with the accuracy of classifier below chance and therefore the prediction time may not related to my task. I thought If I could sort out the trials leaving only correct predictions the prediction time would more precise reflect the difficulty of the task.