Comparing two differently-sized arrays

A'ight, I'm throwing in the towel. I have two different arrays, which are indexed by time. Each element of both arrays has two time indices: start time and stop time (The bars separating each cell in the below table image). These arrays are not necessarily the same size, nor can I assume that there will be any matches in their time indecises (and thus I can't rely on intersect). Because my result needs to be time-synched, I can't rely on best match, either. Below is a representation of about the worst-case scenario for what I'll need to be working with(Array1 and Array2), and what I want to end up with (Array1New and Array2New):

The greyed-out cells represnt spans of real-world time for which there is no data, nor any entries in the respective arrays. So Array1 starts and ends after Array2, and Array1 has no entry covering the span of time between elements 4 & 5. E.g.

Array1{1,:} = {data11}    {[16-Feb-2022 08:17:29]}    {[16-Feb-2022 08:27:12]}
Array2{1,:} = {data21}    {[16-Feb-2022 08:00:33]}    {[16-Feb-2022 08:20:18]}
Array1{4,:} = {data24}    {[16-Feb-2022 10:23:38]}    {[16-Feb-2022 10:37:56]}
Array1{5,:} = {data24}    {[16-Feb-2022 10:52:31]}    {[16-Feb-2022 11:06:15]}

I need to either reformat Array1 and Array2 to match Array1New and Array2 new, or find some way to call the desired sections of Array 1 and Array2 when performing operations wich use elements both Arrays as inputs. Put another way, because this seems to be very confusing to others:

Option A) Reformat Array1 and Array2 so that they cover the same real-world timeframe, have the same timestamps, and the same number of elements. New elements which don't map back to an old element (e.g. Array1New(1,1)) should be filled with pre-determined filler data, while elements which do map should have data copied over (e.g. Array1New(2,1)=Array1(1,1)).

Option B) Have some syntax or which allows me to achieve the net result of Option A without reshaping Array1 and Array2 (in case such would be faster/better).

Thus far, I have the following code note that my actual data inputs are structure arrays, hence the specific braces used.

...

TimeIdcs1 = [Record1.StartTime;Record1.StopTime]'
TimeIdcs2 = [Record2.StartTime;Record2.StopTime]'

%Combine timestamps into single vector, remove duplicates, and sort
TimeRecord=unique(vertcat(TimeIdcs1,TimeIdcs2));
%Use circshift to create single, combined timestamp array
TimeRecord(:,2)=circshift(TimeRecord,-1);
TimeRecord(end,:)=[];
%Create Array1New and Array2New
TimeRkrdLength = length(TimeRecord);
Array1New = cell(OvrLpRkrdLength,3);
Array2New = cell(OvrLpRkrdLength,3);
for i=1:TimeRkrdLength
    Array1New{i,2}=TimeRecord(i,1);
    Array1New{i,3}=TimeRecord(i,2);
    Array1New{i,2}=TimeRecord(i,1);
    Array1New{i,3}=TimeRecord(i,2);
end

%Define mapping of records using intersect if possible
[~,Array1Start,TimeRecordStart] = intersect(TimeIdcs1(:,1),TimeRecord(:,1));
[~,~,TimeRecordStop] = intersect(TimeIdcs1(:,2),TimeRecord(:,2));
for i=1:length(TimeRecordStart)
    Array1New{TimeRecordStart(i):TimeRecordStop(i),i}=Array1(Array1Start(i))
    %Not really sure how to do this, as there does not appear to be a way
    %to block assign a set of values to an entire column of a cell array
end

[Jan]

Please post some Matlab code, which produces the inputs and the wanted output for the shown test case. It is not efficient if the readers guess, how the grey areas are represented in the original data. NaNs? Or do you have 4 input vectors with the start and stop times?

[Gabriel Stanley]

I have 4 input vectors with the start and stop times (StartA, StopA, StartB, StopB). So the grey areas don't exist in the data set. I also don't have the code which creates the data, just it's output.

[Jan]

To suggest some code for the solution, I have to test it with input data. It is not efficient, if I guess the exact representation (4 sorted double vectors?!) and invent some test data. So it would be your turn to offer some input data.

Are the times sorted? Can they overlap for one of the arrays also? Can a phase have the time zero, such that a value appears in the list of start and stop times?

What is the wanted output? What does this mean: "perform element-by-element operations on the overlaping sections"?

I assume, the solution can be implemented fast, if the problem is defined clearly.

[Gabriel Stanley]

Ok, I've included an example of what I'm dealing with, which is the best I can give you. I've figured out how to create a single vector of all start & stop times without repeats, but am still trying to figure out how to use those indices to eaither search through my original datasets as part of any functions which will invoke both of them, or how to expand both of them so they have to same number of elements and the same timestamps.

[Jan]

When I open the mat-file, I find 2 variables "Array1" and "Array2". Both contain {n x 3} cell arrays. The contents of the first row or Array1 is:

{31×26 double}    {[16-Feb-2022 08:17:33]}    {[16-Feb-2022 08:20:18]}

The connection to the posted picture is weak, so I assume these are two different examples.

I suggest, that you concatenate all datetime elements in one vector, sort it, use the 2nd output of sort() to find which output element belongs to which input variable. Then you have integer indices, which are much easier to handle than the datetime objects. I assume, this takes about 10 lines of code only.

I still do not have any idea of what you want as output and what "perform element-by-element operations on the overlaping sections" means. It is not efficient, if I spend more time in questions for clarifications and leave this discussion now.

[Gabriel Stanley]

And I don't know how to more concretely describe what I'm trying to achive: The two arrays cover around the same range of time, but are both discontinuous and slightly asynchronous. I need to reshape both arrays such that they have the same number of elements, with the same timestaps (the datetime entries), filling in any data element (the 31x26 double matrices) which didn't exist in the prior cell arrays with predetermined junk data (e.g. in the picture provided, the reshaped Array2 would have a data matrix of zeros in position 12, while positions 10 and 11 would have the same data as element 7 of the original Array2).

[Timo Dietz]

Hi, I had a look at your .mat file. What exactly should be compared? Each time frame definition provides a value array. But what do you mean with "comparing" - with respect to the time information? The "Result" row above isn't clear to me.

[Gabriel Stanley]

I've editied the original question to try to improve clarity, By "comparing" I mean perform any operation/function on the data covering a given span of real-world time. Note that everything I've provided are manually-created examples. The number of elements in either array, the exact timestamps of either array, the total amount of time covered in either array, etc. can all vary. I am only assured that both arrays will cover roughly the same time (i.e. both will cover roughly the same 1/2 hour from 08:00-08:30, the same two hours from 17:13-19:17, etc.).

Answer 1

[Mohammad Sami]

The best way to do what you are requesting is to convert your arrays to timetables and then either use the synchronize function or the Live Script Synchronize timetable task to combine these timetables. You can find more details in the documentation here. As the function may have changed over time you can refer to the documentation for your version of Matlab.

R2020a+ Live Task https://www.mathworks.com/help/releases/R2021a/matlab/ref/synchronizetimetables.html

R2016b+ Synchronize function https://www.mathworks.com/help/releases/R2021a/matlab/ref/timetable.synchronize.html

https://www.mathworks.com/help/releases/R2021a/matlab/ref/timetable.html

Some examples

https://www.mathworks.com/help/releases/R2021a/matlab/matlab_prog/create-timetables.html

https://www.mathworks.com/help/releases/R2021a/matlab/matlab_prog/combine-timetables-and-synchronize-their-data.html

https://www.mathworks.com/help/releases/R2021a/matlab/matlab_prog/retime-and-synchronize-timetables-using-different-methods.html