'Find' function doesn't get me all required values

Hello,

I am trying to use the 'find' function in order to locate some data-points that I need to exclude (erase all (:,25)>100)

It works, in that I get back 5 rows in column 25 that are indeed problematic - yet, I've found an additional one, bigger than 100, that 'find' doesn't seem to... find (the problematic :,25 I've found by merely scrolling down the matrix is not among the aforementioned 5 rows MATLAB finds).

I can't seem to get my head around what the problem is so could you please help me with any ideas?

[Image Analyst]

This is not proper syntax:

find (the problematic :,25

What is your array?

[Bianca Elena Ivanof]

This is the actual piece of code that I wrote

probability_judgment=all_operant_data(:,25); % index  PJ column within operant array
faulty_probability_judgment= find(probability_judgment>100) % find rows which contain PJ values >100

what MATLAB gives me is 5 rows which, in column 25, have values >100; yet, when I inspect the matrix myself, I can find at least an additional row which is not among those 5 'find' finds

[Image Analyst]

AGAIN, what is your array? Bianca, we don't have your all_operant_data, so how can we help? Type this on the command line and then paste the result back here:

probability_judgment

and also tell us the exact row numbers that you think find() should return.

[Bianca Elena Ivanof]

You'll find find probability_judgment.mat attached - I preferred to attach it rather than copying and pasting it as a code because there are 7800 rows.

The problem is that I don't know how many data-points>100 I should find - I only know there aren't many experimenter mistakes, there shouldn't be many values>100.

Answer 1

[Star Strider]

We would have to see your code, your matrix, and a bit more detail as to what you want to do. ‘Logical indexing’ rather than using find (unless you need the actual index numbers) might be easier.

EDIT — (15:35 GMT, 2013 04 28)

When I run this:

D = load('Bianca Elena Ivanof probability_judgment.mat');
M = D.probability_judgment;

GT100 = sum(M > 100)
GE100 = sum(M >= 100)

I get:

GT100 =
     5

GE100 =
   256

The 5 values that seem to meet your criterion are:

           111
           660
          7030
          1575
          4020

Which one isn’t on this list? I’ve not seen any evidence that MATLAB’s comparison functions are inaccurate. I doubt that floating-point approximation error could account for the discrepancy you see. Use format short g to look at your vector.

[Bianca Elena Ivanof]

Hey

 probability_judgment=all_operant_data(:,25); % index  PJ column within operant array
faulty_probability_judgment= find(probability_judgment>100) % find rows which contain PJ values >100

Many thanks. Above you find the attached matrix and the exact piece of code that I wrote.

[Bianca Elena Ivanof]

Thank you.

This is weird. When using my code, I get

faulty_probability_judgment =

         605
        5659
        6328
        6336
        6353

which doesn't match what you get.

When I jump to rows 605, 5659, ... in col 25, they indeed contain values >100. By contrast, the 5 rows that you give me above don't contain values >100 (e.g. probability_judgment(111,25)=30)

[Star Strider]

The only conclusion I can draw from that is that we’re not looking at the same data. (I used the data you posted in the Comment to your original Question in response to Image Analyst about an hour ago.)

The plot thickens ...

[Bianca Elena Ivanof]

No, we're both using the same data (see attached)

Which means the plot is really thickening

[Star Strider]

... and we both seem to be in the same universe!

I’m using R2016a, but I doubt that matters.

The code I used to get the five values I posted is:

Q1 = find(M > 100);
M(Q1)

[Bianca Elena Ivanof]

Hello again and thanks

Yes, if I use your piece of code, I get the same values as you. Yet, when I jump to those rows, none contain values >100...

[Star Strider]

Something is wrong somewhere. The code I posted in my Comment on (28 Apr 2016 at 16:14) produced the results in the last part of my original Answer (28 Apr 2016 at 15:36).

[Bianca Elena Ivanof]

No, I've figured it out.

c=find(probability_judgment>100)

gives me the rows containing the problematic values.

probability_judgment(c)

gives me the problematic values themselves, hence the discrepancy between what numbers i would find when using only find(probability_judgment>100) and what you find using the additional step probability_judgment(c). Conclusion: MATLAB never makes mistakes.

I've identified the root of the problem, though. After performing the aforementioned 2 steps I typically delete the problematic rows found by find(probability_judgment>100) like this

probability_judgment=all_operant_data(:,25); 
faulty_probability_judgment= find(probability_judgment>100) 
faulty_probability_judgment= 605
      5659
      6328
      6336
      6353
all_operant_data(605,:)=[]; 
all_operant_data(5659,:)=[];
all_operant_data(6328,:)=[];
all_operant_data(6336,:)=[];
all_operant_data(6353,:)=[];

The problem is that those rows don't get deleted for some reason. When I go back and check the new matrix (which now has -5 rows, obviously), the values found by find and are still there - what I do above using the '[]' technique deletes other rows that have nothing to do with faulty_probability_judgment.

Damn. Matlab loves a soap opera.

[Steven Lord]

Let's look at a smaller example which will make it easier to see what's going on.

x = [1; 2; 3; 4; 5]

Which row of x contains the value 3?

x(2, :) = []

After executing that command, which row of (the new) x contains the value 3? Is the answer to the first two questions the same?

If you want to delete multiple rows, either delete from the bottom up or delete them all at the same time.

faulty_probability_judgment= probability_judgment>100; % No need for find
all_operant_data(faulty_probability_judgment, :) = [];

[Star Strider]

Thank you Steven. Away for a few minutes here.

To emphasize: Always delete from the bottom up or simultaneously to keep the correct indexing.

[Bianca Elena Ivanof]

Hello Steven and Star Strider

Many thanks for your explanations and support - it's dawned on me. OF COURSE that, by applying my initial method, upon deleting the first faulty row (out of 5), the matrix contains one row less. Therefore, if I delete the remaining 4 rows serially (as I did), I will delete the wrong rows (because there will be n-4, n-3, n-2, n-1 rows and the matrix changes linearly with the number of rows I delete).

Thanks!

[Star Strider]

Our collective pleasure!