reduce density of a time series

19 vues (au cours des 30 derniers jours)
jakobs
jakobs le 11 Nov 2019
Modifié(e) : Daniel M le 12 Nov 2019
I have a time series a of a river discharge with over 700000 points. I'd like to "clean" the data and delete all irrelevant points (points which share almost the same value with their neigbouring points). I imagined a code which compares two values (e.g. a_i - a_i+1) and if the result is below a certain threshold the second value gets deleted.
I already tried functions like downsampling, but I am afraid that I will loose important information about minima and maxima. Is there maybe somewhere a link to this question?
  3 commentaires
jakobs
jakobs le 12 Nov 2019
Hello,
thanks for the help. The plottet result is here.
The values of the second figure range inbetween 10^10 (at the beginning) and 10^5 (shortly after).
figure.png
Daniel M
Daniel M le 12 Nov 2019
Modifié(e) : Daniel M le 12 Nov 2019
Actually if you don't mind can you run it again but this time change the bottom plot to semilogy and detrend your data when you input it using detrend(X).
Also it wouldn't hurt if you could draw or otherwise indicate what you would like your output signal to look like. Is it just a smoother version of the original?
(Or I could look at your data if you uploaded).

Connectez-vous pour commenter.

Réponses (1)

Adam Danz
Adam Danz le 11 Nov 2019
Modifié(e) : Adam Danz le 12 Nov 2019
" I imagined a code which compares two values (e.g. a_i - a_i+1) and if the result is below a certain threshold the second value gets deleted."
The diff() function does the first half of your description. I'll assume your dates are stored in a column vector named "dates" and they are in datetime format. If they are not in datetime format some very small modifications will need to be made.
theshold = hours(1); %can be any duration: minutes(30), hours(12) days(2), etc....
rm = [false; abs(diff(dates))<threshold]; %logical vector of datetimes to remove
datesClean = dates(~rm);
% OR
% dates(rm) = []; % to keep variable name
If you're working with a timetable, you'll apply the rm vector to the rows of the table.
  2 commentaires
Daniel M
Daniel M le 11 Nov 2019
This idea should work even if OP is not talking about a timeseries object, but just regular data as a function of time.
Adam Danz
Adam Danz le 11 Nov 2019
Modifié(e) : Adam Danz le 11 Nov 2019
Yeah; it's been my experiences that the term "time series" is more generally used by folks in this forum to describe their data more often than describing the use of Matlab's timeseries objects. Unless they specifically indicate one or the other, I assume they are speaking more generally. Good point!

Connectez-vous pour commenter.

Catégories

En savoir plus sur MATLAB dans Help Center et File Exchange

Produits


Version

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by