noisy plot after calculating first derivative

Hi everyone

I have a problem.

I have a very smooth plots in MATLAB (fisrt capture which is the original y). I ataach one of them. whenever, I calculate the first, sencon and ... derivative of these data with finite differemnce equations, the graphs are not smooth anymore (first deivative=y').

all data are from numerical model (Abaqus).

what should I do?

is smoothing the soulution? if yes, which type of smoothing

Thanks

Answer 1

[John D'Errico]

You should recognize that differentiation is a noise amplification operator. So any small amounts of noise in your signal, even a tiny amount, will translate into a noisier mess in the derivative. Second derivativves are yet more difficult to estimate.

Should you try to smooth the derivative estimate? NO!!! (Was that a sufficiently emphatic no?) Instead, you need to smooth the signal itself, since that is where the noise lies. Why is that?

Suppose that your noisy function values are each corrupted with noise. Then when you compute the derivative, the noise is still there, and is amplified. But worse, now the noise in the derivative estimates is no longer what is called white noise, but now the neighboring points will be correlated with each other. Anyway, it is best if you just smooth the data itself, and then compute a derivative signal from that. You might use the function smooth, or perhaps a smoothing spline would be a good idea. Without having your data itself, it is difficult to know what may be best, but either of those tools would be a good choice.

[Fahime Sokhangou]

@John D'Errico Dear John

Thanks for your rapid answer,

But the data are from Abaqus and there are not experimental data.

I have 40 points and they have discrete y value that comes from a finite element software.

I can provide my Matlab code here.

[Fahime Sokhangou]

the next step, the data will come from experimental, and for sure there is experimental noise.

How I can understand that the noise is because of experimental or due to differentiation.

Thanks

[Torsten]

I thought you get the original y data and not the y' data, don't you ?

[Fahime Sokhangou]

@Torsten the first one is y and the second image is y'.

[Torsten]

Differentiation does not introduce noise, but amplifies it. So noise is always due to noise in the data.

[Fahime Sokhangou]

@Torsten please look at foolowing picture which is the second derivative of my function based on finite difference equation.

the function is not noisy because it is from Abaqus .

[John D'Errico]

Remember that regardless of how you got the numbers, there will be stuff happening. That "stuff" can arise from many sources, not all of which are random. It might be some sort of approximation error, in whatever produced your data. It might even be floating point trash, down in the least significant bits.

In your case, the "data" apparently comes from finite element software. But then you need to consider that those numbers are not themselves EXACT. Finite elements is itself an approximate solution to a mathematically posed problem, modeling a process using some mathematics, then approximating that result using a simple set of low order segments. It is not exact. It may seem like it is, but that does not make it so. It is a mathematical model of a system, and has its own errors in the approximation. And those errors are part of the stuff I describe above. They are not random noise, but they still act like noise, so when you try to differentiate it, you get amplified crap happening.

Just because a computer program produces numbers with many digits reported, those spare digits may be nothing more than a mirage. They make you think they are correct.

[Fahime Sokhangou]

@John D'Errico thank you so much for your complete response. yes, you are right. By seeing with eyes, from the original function seems so smooth, am I right?

So, the only solution will be smoothing with spline?

[Torsten]

@Fahime Sokhangou

Sorry, but I cannot believe that you get such a saw tooth approximation for the second derivative from a smooth function.

[Fahime Sokhangou]

@Torsten maybe by eye, it seems smooth but above @John D'Errico explained that since my data is from finite element software and they are not exact.

[Fahime Sokhangou]

@Torsten the fourth derivative based on dinite difference equation is like following image.

[Fahime Sokhangou]

@John D'Errico thanks a lot. I did smoothing and it works for first, second and third derivative with cubic spline (101 points) but for the fourth mode. it is still noisy.

I may use spline with more points.

[Fahime Sokhangou]

@John D'Errico

I still have problem in choosing the smoothing function.it has oscilatory data at the end when I calculate the second derivative.

Can you please guide a little in terms of choosing the smoothing method.

please help me in choosing the method of smoothing for the data from abaqus. I want a good method to get rid of oscilatiory data after callculation of the second derivative and plotting it.

thanks in advance