What to do when the filter kernel is longer than the input signal?

I need to filter out frequenzies between 0.002 and 0.02 Hz meaning that I get an extremely long band pass filter kernel, 800 001 points. I have 30 minutes of input signal sampled at 100 Hz meaning that the input signal is 180 001 points.

If I do a convolution of the input signal and the filter kernel I get a L+M-1=180001+800001-1=980001 points long signal. Normally I would then remove M-1 points from each end to remove the end effects of the convolution.

But what happens when the filter kernel is so much longer than the input signal? It is ofcourse impossible to remove M-1 points from each end of the output since this is longer than the output itself. The commutative property of convolution suggests however that the input signal and the impulse response (=filter kernel) can be exchanged. L*M=M*L.

In this case it would be sufficient to remove L-1 points from each side of the output signal to remove end effects, which is no problem at all.

So, if I cut out the 180001 points in the middle of this output signal, do I then have a perfectly filtered signal without end effects?

My initial tests might suggests that this is not the case. If I first filter out a low frequency signal from random noise by using my BP filter, then add random noise containing only frequencies above my frequency band (filtered by a HP version of my filter) to this signal and then filter out my frequency intervall again (with the BP version) the two curves differs quite a bit, see image below. Blue curve is the original signal and the red one is after addition of random noise and filtration.

Is this just as good as it gets or is there any particular reason to why the curves are not better aligned?

To increase speed I am doing the "Convolution" in the frequency domain meaning that I do an fft of both "signals" before I multiply them. I have made room in both the fft's to take care of the full length output after convolution so circular convolution should not occur,

fft_length = 2^nextpow2(length(input)+KernelLength-1);

Answer 1

[Ayush]

Hey Quist,

I understand that you are trying to filter out frequencies between 0.002 and 0.02 Hz using a long filter kernel. However, you are facing challenges in removing the end effects caused by the convolution process. Normally, you would remove points from each end of the output signal, but when the filter kernel is much longer than the input signal, this becomes impossible.

You are considering cutting out the middle portion of the output signal to remove the end effects, but you are unsure if this will result in a perfectly filtered signal. Additionally, you have noticed differences between the filtered signal and the expected response when adding random noise and performing subsequent filtering.

Cutting out 180001 points from the middle of the output signal will not eliminate end effects caused by convolution. These effects occur due to the convolution extending beyond the input signal boundaries. Differences between the filtered signal and the expected response can arise from various factors, including filter characteristics, noise, and numerical precision. Achieving a perfectly aligned filtered signal without deviations is challenging. Optimizing the filter design, increasing length, mitigating noise, and exploring alternative techniques can help minimize differences, but perfect alignment may not be attainable in practice.

Hope this helps!

Regards,

Ayush Goyal