How to correct for drift in a signal not using detrend

I have a 30 hour data set, where I am interested in taking out the drift the last 10 hours of the dataset. It is very important to keep the dataset at the same length as I started as I have other sensors in the system that I am looking at with this signal. I have tried:

sdata = oxygen_unisense.data;
oxygen_unisense.detrend = detrend(sdata(3.715958429982246e+06:end));
 oxygen_unisense.mean = oxygen_unisense.data - oxygen_unisense.detrend;

but this puts the part of the signal to zero.

Anyone with suggestions?

Data: https://www.dropbox.com/s/c8uy84e2ndem1nl/oxygen_unisensemathelp.mat?dl=0

[Rebecca Ellis]

lf = polyfit(oxygen_unisense.time(3715958.429982246:end),oxygen_unisense.data(3715958.429982246:end),1);
plot(oxygen_unisense.time(3715958.429982246:end),oxygen_unisense.time(3715958.429982246:end)*lf(1) + lf(2),'--k');
plot(oxygen_unisense.time(3715958.429982246:end),oxygen_unisense.time(3715958.429982246:end)*-lf(1) + oxygen_unisense.data(3715958.429982246:end))
trying this aswell

[dpb]

You'll have to be able to find the breaks in the signal at the points of interest first.

Then, just detrend that section and substitute that result into the overall signal at that location in the original.

ADDENDUM

You'll have to do something about adjusting level to keep continuity at the breakpoint, however...

Alternatively, just return the slope from linear fit and adjust the portion by the ratio it would return should do it.

[Rebecca Ellis]

Thank you for your reply, what would the code be for this? The points if interest are (3.715958429982246e+06:end). Would the polyfit function work?

[jonas]

You could also use findchangepts to segment the entire signal and detrend each segment individually

[Rebecca Ellis]

I only want to change one part of the data (The point of interest = 3.715958429982246e+06 to the end. It is drifting linearly.

[dpb]

I don't understand what that means, precisely, then.

|detrend| doesn't "zero", it removes linear trend unless use the optional 'constant' argument with which it simply subtracts the mean.

The 'linear' option is default; I'd forgotten but actually you can specify breakpoints with default as written which should just be "the cat's meow" once you identify the section(s) of interest.

[jonas]

@Rebecca: 3.715958429982246e+06 is not a valid index (although MATLAB seems to round it automatically)

[Rebecca Ellis]

sdata = oxygen_unisense.data;
sd=sdata(3.715958429982246e+06:5.838709e6); %Take a section of interest

Hz=64;

%Filter the data to lowpass filter it
b = ones(1,(round(Hz*1000)+1))/(round(Hz*1000)+1);
filtdat=filtfilt(b,1,sd); % This takes a while.

%Substract the filtered from the original signal
sdff=sd-filtdat;

figure
subplot(2,1,1)
plot(sd)
hold all
plot(filtdat)
xlabel('Datapoint')
ylabel('Data')
legend('Original Data','Filtered Data')

subplot(2,1,2)
plot(sdff)
xlabel('Datapoint')
ylabel('Data')

oxygen_unisense.data(3.715958429982246e+06:5.838709e6) =sdff;

So it kind of works but now I need to get the values back up to 250 rather than zero.

[Rebecca Ellis]

sdata = oxygen_unisense.data; sd=sdata(3.715958429982246e+06:5.838709e6); %Take a section of interest

Hz=64;

%Filter the data to lowpass filter it b = ones(1,(round(Hz*1000)+1))/(round(Hz*1000)+1); filtdat=filtfilt(b,1,sd); % This takes a while.

%Substract the filtered from the original signal sdff=sd-filtdat;

figure subplot(2,1,1) plot(sd) hold all plot(filtdat) xlabel('Datapoint') ylabel('Data') legend('Original Data','Filtered Data')

subplot(2,1,2) plot(sdff) xlabel('Datapoint') ylabel('Data')

oxygen_unisense.data(3.715958429982246e+06:5.838709e6) =sdff+250;

This works,

Thank you for your advise.

[Image Analyst]

Rebecca, why is the data in some third party drop box instead of attached here? Come on, make it easy for people to help you.

http://www.mathworks.com/matlabcentral/answers/13205#answer_18099

[Rebecca Ellis]

SOrry the data is too big to attach, i tried to attach it.

[dpb]

Only need the section of interest and it could be decimated by 10X or more and saved as .mat file and be just as useful as the full thing.

detrend ought to be able to do what your code does automagically given the breakpoint(s).