How the increase the affect of lowpass filter on a field data?

Hello,

I would like to apply lowpass filter to my temperature data to remove the high frequency noise. I have applied simply 'lowpass' and could not see a significant affect on my data. I have seen several question regarding the lowpass but I am quite new to this area, so maybe missed the point.

sampling_rate=10; %seconds
fs=1/sampling_rate; %Hz
fpass=0:0.01:fs/2;
 
T35_lp=lowpass(T35,0.01,fs);

Here you can see the applied filter and original data below. They are almost same.

Is there any way to improve the affect of lowpass filter? I appreciate any suggestions.

Thanks!

Answer 1

[Star Strider]

The cutoff frequency of the lowpass filter is too high.

Try these —

LD = load('T35.mat');
T35 = LD.T35;
sampling_interval = 10;
Fs = 1/sampling_interval;
L = numel(T35);
t = linspace(0, L-1, L)/Fs;

[FTT35,Fv] = FFT1(T35, t);

figure
semilogx(Fv, abs(FTT35)*2)
grid
xlabel('Frequency')
ylabel('Magnitude')
title('T35: Fourier Transform')
xlim([0 Fs/2])

Fco = 1E-5;
T35_filt1 = lowpass(T35, Fco, Fs, 'ImpulseResponse','iir');
Fco = 8E-7;
T35_filt2 = lowpass(T35, Fco, Fs, 'ImpulseResponse','iir');

figure
plot(t, T35, 'DisplayName','Original (Unfiltered)')
hold on
plot(t, T35_filt1, 'DisplayName','Filtered (F_{co} = 1\times 10^{-5})', 'LineWidth',2)
plot(t, T35_filt2, 'DisplayName','Filtered (F_{co} = 8\times 10^{-7})', 'LineWidth',2)
hold off
grid
legend('Location','best')

function [FTs1,Fv] = FFT1(s,t)
    s = s(:);
    t = t(:);
    L = numel(t);
    Fs = 1/mean(diff(t));
    Fn = Fs/2;
    NFFT = 2^nextpow2(L);
    FTs = fft((s - mean(s)).*hann(L), NFFT)/sum(hann(L));
    Fv = linspace(0, 1, NFFT/2+1)*Fn;
    Iv = 1:numel(Fv);
    FTs1 = FTs(Iv);
end

.

[Cakil]

Thank you very much @Star Strider! The answer was definitely what I was trying to get and now I realize that I couldn't have reach that point without the professional touch. I believe I can use Fouier Transform in similar cases to understand applicable Fco magnitude.

[Star Strider]

As always, my pleasure!

The first thing I do when deciding on how to process a signal is to calculate its Fourier transform. (I also compare the mean of the differences in the sampling times and the standard deviation (std) of those differences to determine if the sampling times are consistent or if I need to use the resample function first to create uniform sampling intervals, since that is necessary to do further signal processing.) That tells me if there is broadband noise (in that instance, frequency selective filters are only marginally useful, and the sgolayfilt approach is likely preferable), and if the noise is band-limited, what the appropriate frequency is to filter it out.

I wrote ‘FFT1’ to avoid having to type all that out whenever I wanted to do a Fourier transform. Feel free to use it.

Another option is the pspectrum function, since it offers time-frequency analysis as well as computing the power spectrum. (I prefer it to spectrogram for time-frequency analyses.)

[Cakil]

Thank you for the brief information about how to approach and where to start to processing data. It might be confusing to decide which method to use, so it is really a valuable feedback for me. Also I will definitely using the FFT1. If I understand correctly, spectrogram is an alternative to Fourier transform to understand the frequency variation of the time series data.

[Star Strider]

As always, my pleasure!

Your interpretation of the spectrogram is correct.

Answer 2

[Donya]

Hello,

I would like to know how the Fco is found?

[Star Strider]

By looking at the Fourier transform of the original signal. (That is the reason I calculate it in every signal processing problem I encounter.)

The findpeaks function (using the negative of the absolute value of the Fourier transform output), or the islocalmin function can help to localise the frequencies of the minimum values. These can be used to find the best value for ‘Fco’. The choice is somewhat empirical, so it is usually necesary to experiment, since it may not be obvious initially what the best value for it is.