Time matching across 2 differently sized time series arrays

How can I come up with an array index that provides matching times?

Time1 = [31:0.01:35];
Time2 = [31.01:0.02:35.2];

I want to make an array (Time3) that has ONLY the matching times of Time1 and Time2. Need the fastest solution, running this through a datastore read...

ds = datastore(infile);
while hasdata(ds)
  X = read(ds);
  Time1 = X.{timestamp};
  Time2 = X.{timestamp_b};

end

ARRAYS/MATRICES ARE NOT EQUAL IN SIZE. NO INTERPOLATION WANTED.

Answer 1

[Guillaume]

As per KSSV's answer, the correct function for finding the values that are exactly identical in two vectors is intersect.

However, I suspect that it will fail miserably for your purpose, as I assume you consider the 31.11 in Time1 to be the same as the 31.11 in Time2. However, due to the floating point precision and the way you generate your two arrays, they're not exactly identical. They differ by ~3.5e-15.

Time1 = 31:0.01:35;
Time2 = 31.01:0.02:35.2;
commontime = intersect(Time1, Time2); 
assert(any(commontime == 31.11), '3.11 is missing from intersection')

So, instead if you want to match values that are nearly the same within the calculation error of your values, then you should use ismembertol

Time1 = 31:0.01:35;
Time2 = 31.01:0.02:35.2;
commontime = Time1(ismembertol(Time1, Time2));  %use default 1e-12 tolerance
assert(any(commontime == 31.11), '3.11 is missing from intersection')

Note that the above assumes that values in Time1 are unique. Otherwise, apply unique to commontime afterward.

[Tom W]

Correct in that intersect doesn't catch every matching value (ahem, MATHWORKS). I don't have the function ismembertol. Why doesn't intersect come with a tolerance option??? I found using date strings works, which is rather odd, considering an innate mathematical expression doesn't functionally work (referring to intersection) for floating point numbers.

[Guillaume]

ismembertol was introduced in R2015a. There are other functions with a tolerance option (e.g. uniquetol) but they were all introduced is R2015a, so it's not going to help you.

The problem with intersect is nothing to do with matlab. It's a fundamental problem of storing numbers on a computer, simply because not all numbers can be stored exactly without using infinite memory.

The 31.11 generated by your Time1 expression is actually 31 + something as close as possible to 0.01 + something as close to possible to 0.01 + ... (repeated 11 times), which gives something very close to 31.11, whereas the 31.11 generated by your Time2 expression is actually another something very close to 31.11 but not the exact same something simply because you didn't use the same algorithm to get at it.

Here is a way to emulate the intersection with a tolerance:

function [c, ia, ib] = intersecttol(A, B, tol)
    %note that tol is absolute, with the matlab functions it can be relative
    %note that this does not implement the row option
    %this also assumes that A and B are row vectors
    %input validation required
    [ia, ib] = find(abs(bsxfun(@minus, A.', B)) < tol);
    C = A(ia);
 end

[Tom W]

Thanks @Guillaume -- I'm going to try that emulation and will get back to you. Also thanks for the computational explanation to number storage, I think that stuff is pretty interesting because it ultimately affects EVERYTHING we do these days via computer math! I actually have upgraded to 2016b, but it's not working right, so I'm temporarily reverted to 2014b. The upgrade has been quite a headache!

Answer 2

[KSSV]

doc intersect.

You can find the intersection of two arrays.

[Jan]

@Tom W: Note that intersect will not necessarily work as expected with your test data dur to the limited precision. See the FAQ:

0:0.1:1 == 0.3

So you might need a tolerance.

[Tom W]

Intersect doesn't catch all the matching values.