Find duration of signal

I have a set of signals that due to some unknown error have been acquired with millions of empty datapoints. Is there a way I can automatically detect the moment the signal ends so I can delete everything from that point onwards?

Thanks everyone.

[Xavier]

Can you upload a copy of the raw data?

[Marc Elmeua]

Ofcourse, I have modified a bit the original as it was too heavy, but it's the same concept.

[Star Strider]

There are no actual ‘missing’ values, just a very low amplitude signal after about element 40:

T1 = readtable('sampledata.csv');
figure
semilogy(abs(T1{:,1}))
grid

producing:

That would appear to be legitimate data.

.

[Xavier]

This depends on how you define legitimate data, if you unplug a probe/sensor from an oscilloscope it will continue to display numbers and a waveform. If you're trying to measure a signal then it is fair to call recorded noise invalid data

Answer 1

[Xavier]

Something like this may work for you

Raw data:

data = readmatrix('sampledata.csv');

avgdat = movmean(abs(data), 5);
threshold = 0.01;
data(avgdat < threshold) = [];

plot(data)

[Marc Elmeua]

This worked perfectly!! thanks a lot.

[Xavier]

No problem!

Answer 2

[Steven Lord]

If the empty data points are represented by NaN then find the 'last' 1 element of the vector that isfinite.

x = [1:10 NaN(1, 5)]
find(isfinite(x), 1, 'last')

If the vector has no missing data inside it, you could instead just rmmissing.

rmmissing(x)

[Marc Elmeua]

Thanks for the reply! The missing values aren't really missing in the matlab file. They are missing in the original data collection software but as soon as you retrieve them, they turn into zeros so this code will not work :(

I'm in touch with the developers of the software but they still haven't been able to solve this.