Using knnsearch to find nearest values ignoring NaN's

Hello,

I have two datasets. A is a high resolution global dataset, and B is a smaller dataset with sparse points. I want to find the indices of lat and lon in A that are closest to B lat and lon, but I want to ignore the NaN values in A and instead get the next closest value that isnt NaN.

Is there any way to do this?

A is a 2D dataset (latxlon) while B is three individual vectors of lat, lon, and data Which is why it makes it hard to use a lot of other commands.

Please let me know if you have any ideas,

Thanks,

Melissa

[Melissa]

Right now this is the code I have, but I get a lot of NaN's in the data because I am still able to search through the entire row and column of A, even if there is only one non NaN in that row and column

NaN = isnan(A); %360x720
[row col] = find(NaN==0); %62641x1 row, 62641x1 col

row = unique(row);
col = unique(col);
lon_A = lon(col);
lat_A = lat(row);

[IDX_lat,D_lat] = knnsearch(lat_A,lat_B);
[IDX_lon,D_lon] = knnsearch(lon_A,lon_B);

nearest_lat = lat_A(IDX_lat);
nearest_lon = lon_A(IDX_lon);

[Sean de Wolski]

I think you need to project your lat/lon into x/y or compute the distance using a custom distance function in knnsearch such as distance() or ecefOffset. Otherwise distance is not measured on a sphere and is useless.

http://www.mathworks.com/help/releases/R2015a/stats/knnsearch.html

[David Young]

Sean - you are right in that computing distances using lat and long as if they were euclidean coordinates is going to create some degree of error. But "useless" is too strong - provided that the lat-long grid is reasonably fine, and that there isn't an issue with wraparound, interpolation using a lat-long coordinate system may well be fine for many problems.

Answer 1

[David Young]

You need to organise the nearest neighbour search so that it uses both latitude and longitude together, not search each separately. Try this, assuming you want to use Euclidean distance in geographical coordinates:

[rowA, colA] = find(~isnan(A));
lon_A = lon(colA);   % I assume lon and lat convert from index to degrees
lat_A = lat(rowA);
idx = knnsearch([lon_A lat_A], [lon_B(:) lat_B(:)]);
nearest_lat = lat_a(idx);
nearest_lon = lon_a(idx);
nearest_Avalue = A(sub2ind(size(A), rowA(idx), colA(idx)));

I haven't checked this as I don't have your data or the statistics toolbox, but if it doesn't work please say what goes wrong in a comment.

[Edit: inserted call to sub2ind to correctly get nearest_Avalue.]

[Melissa]

Thank you David,

Yes, it worked! But the only problem I have now is that the nearest value is in a 419x419 matrix when I would like it to be a vector of values correlated to the other vectors of lat and lon. Is there a way to make it in this format?

Thanks,

Melissa

[David Young]

Sorry - I've corrected the code above.

Answer 2

[Thomas Koelen]

What you could do is:

Lets say you have an array like this:

 A= [10 20 30 40 50 60 70 80 100]

and the smaller array like this:

[13 22]

For now let's just look at

B=[13]

We devide 13 from A and take the absolute.

C=abs(A-B)

this gives us:

C =

       3     7    17    27    37    47    57    67    87

now can find the minimum, and get it's index.

[row,col]=min(C);

This gives index [1,1] which makes sense because 10 is closest to 13!