How to interpolate under specific condition?

Hi everyone. I would appreciate your help on this.

I have a daily timeseries of temperature observations for 2006-2016.

I want to reconstruct missing values (see attached) with an interpolation method only if the missing values cover less that 3 days. If the gap is larger (>4days), I want NaN values to be preserved. Column A contains the dates I have temperature observations (NaT indicates temperature is NaN) and Column C is the full time sequence 2006-2016.

How can I do this? I am totally confused... The interpolation method could be anything, I have not decided on this yet.

Thank you in advance!

PS. I am on R2019a.

Answer 1

[Stephan Ciobanu]

Hi, this code should work:

The file Daily data.xlsx must be in the same directory as your script!

DATA = importdata('Daily data.xlsx'); % importing all data

MeanTemp = DATA.data.Sheet1; % creating an array containg the mean Temp measured

pos = find(isnan(MeanTemp)); % find NaN elements

x_val = (1:length(MeanTemp))'; % creatin a 'pseudo-timeline' vector

for i = 4:length(pos) % finding the index of 4 or more consecutive NaN
    
    if (pos(i)==pos(i-2)+2 && pos(i)==pos(i-1)+1 && pos(i)==pos(i-3)+3)
        consec_days(i)=true;
    end
end
consec_nan = find(consec_days==true); % index of pos where we want to keep NaN

y = pos;
y(consec_nan-3) = 0;
y(consec_nan-2) = 0;
y(consec_nan-1) = 0;
y(consec_nan) = 0;

k = find(y==0); % excluding 4 or more consecutive days in interpolation
y(k)=[]; % NaN to be interpolated

%--------INTERPOLATION-----------------%

MeanTemp(pos)=[]; % values to be interpolated
x_val(pos)=[];
x_unknown_temp = [x_val;y];
x_unknown_temp = sort(x_unknown_temp);
MeanTemp_interpolated = spline(x_val,MeanTemp,x_unknown_temp);      
MeanTemp_interpolated1 = interp1(x_val,MeanTemp,x_unknown_temp);    
%---------INTERPOLATION-END-------------%

% Creating a matrix with 2 rows 
% containg the values of NaN interpolated
% in both cases
% (spline first column, interp1 second column)

Values_interpolated(:,1) = MeanTemp_interpolated(y);    
Values_interpolated(:,2) = MeanTemp_interpolated1(y);

% Date-time line below:
datetime1 = datetime(2006,01,01):caldays(1):datetime(2016,12,31);

% String_Values_interpolated contains Values_interpolated
% and the relative day
String_Values_interpolated = num2cell(Values_interpolated(1:end,1:2));
String_Values_interpolated(:,3) = cellstr(datetime1(y))';

[Daphne PARLIARI]

Thank you Stefan!

Your answer is really elaborate, but let me change my thought a bit, in order to do what I want in a simpler way.

Is there any way I can use fillmissing, excluding a certain rangeof the temeratures table?

Something like

interpolated_Temp = fillmissing(Daily_Meteo.Daily_T(1: 3385, 3390:end), 'linear');

[Stephan Ciobanu]

Yes! There's a shorter and faster way to do this:

interpolated_Temp = fillmissing(Daily_Meteo.Daily_T,'linear','MaxGap',G);
% Where G specifies a maximum gap size to fill. Gaps larger than G will not be 
% filled. 
% In your case should be G=4;

[Daphne PARLIARI]

Thank you but when I try it I get

Error using fillmissing/parseInputs (line 464)

Parameter name must be 'EndValues', 'SamplePoints', or 'DataVariables'.

Could it be because I am on R2019a?

[Stephan Ciobanu]

It might be, I'm using R2020b and it works.

Using the script above you can add those lines at the end:

Temp=DATA.data.Sheet1;
Temp(y)=MeanTemp_interpolated(y)
% or
% Temp(y)=MeanTemp_interpolated1(y)

[Daphne PARLIARI]

I am sorry, where should I add the script above?

Answer 2

[Steven Lord]

If you were using release R2020b or later, I would recommend calling the fillmissing function with the 'MaxGap' option. I'm leaving this as an answer even though you're using release R2019a for future reference by other users or in case you can and are willing to upgrade.

[Daphne PARLIARI]

Yes you are absolutely right. MaxGap doesn't work for me but it is exactly what I need.