30-day running running mean to the detrended timeseries

Dear Sir or Madam,

I hope you are well.

I have some houlry water level obersvations. First I needed to detrend them linearly which I did. Next step, I need to use a 30-day running mean to the detrended timeseries. I have figured out that I needed to use movmean function or smoothdata function to smooth the data using different windows, but I am not sure how to apply the 30-day running mean. Below is the code that I have been using. If someone could help me with that, I would much appreciate it. Also, I have attached my original data. Thank you.

Yours faithfully,

Ali

window=2;
MeanDetrendedData=smoothdata(DetrendedData,'movmean',window);
plot(DateNum,DitrendedData,DateNum,MeanDetrendedData)
datetick('x')

Answer 1

[Star Strider]

This requires reading it as a table, converting it to a timetable and then use the retime function —

T1 = readtable('https://www.mathworks.com/matlabcentral/answers/uploaded_files/623923/Observation%20Data.csv', 'VariableNamingRule','preserve');
TT1 = table2timetable(T1);                                                          % Copy & Convert To 'timetable'
TT1 = retime(TT1, 'hourly','fillwithmissing');                                      % Interpolate If Necessary & Require Hourly Time Step
TT1 = retime(TT1,'hourly',@(x)movmean(x,30*24));                                    % Use 'movmean'

figure
subplot(2,1,1)
plot(T1.Date, T1.Obs)
grid
xlabel('Time')
ylabel('Obs')
title('Original Table')
subplot(2,1,2)
plot(TT1.Date, TT1.Obs)
grid
xlabel('Time')
ylabel('Obs')
title('Timetable With ''movmean''')

I do not see much difference in the plots or numerical output, however it does not throw any errors, so I assume it is working correctly.

EDIT — (20 May 2021 at 14:06)

This is likely closer to what you want —

TT12 = movmean(TT1.Obs,30*24,'omitnan', 'Endpoints','shrink');                                            % Use 'movmean'

figure
subplot(2,1,1)
plot(T1.Date, T1.Obs)
grid
xlabel('Time')
ylabel('Obs')
title('Original Table')
subplot(2,1,2)
plot(TT1.Date, TT12)
grid
xlabel('Time')
ylabel('Obs')
title('Timetable With ''movmean''')

The ‘spikes’ are artifacts due to the discontinuities. Eliminating then would likely not be easy, and would likely involve simply detecting them using islocalmax and islocalmin, and then simply deleting them.

.

[Ali Saremi]

Thank you for helping me with this question.

[Star Strider]

As always, my pleasure!

It occurred to me later that filing the gaps with a constant (specifically 0) removes the ‘spikes’ at the discontinuities.

T1 = readtable('https://www.mathworks.com/matlabcentral/answers/uploaded_files/623923/Observation%20Data.csv', 'VariableNamingRule','preserve');
TT1 = table2timetable(T1);                                                          % Copy & Convert To 'timetable'
TT1 = retime(TT1, 'hourly','fillwithconstant');                                      % Interpolate If Necessary & Require Hourly Time Step
TT1 = retime(TT1,'hourly',@(x)movmean(x,30*24));                                    % Use 'movmean'
figure
subplot(2,1,1)
plot(T1.Date, T1.Obs)
grid
xlabel('Time')
ylabel('Obs')
title('Original Table')
subplot(2,1,2)
plot(TT1.Date, TT1.Obs)
grid
xlabel('Time')
ylabel('Obs')
title('Timetable With ''movmean''')

TT12 = movmean(TT1.Obs,30*24,'omitnan', 'Endpoints','shrink');                                            % Use 'movmean'
figure
subplot(2,1,1)
plot(T1.Date, T1.Obs)
grid
xlabel('Time')
ylabel('Obs')
title('Original Table')
subplot(2,1,2)
plot(TT1.Date, TT12)
grid
xlabel('Time')
ylabel('Obs')
title('Timetable With ''movmean''')

.