precision problem ? why ans for 10 ^ 5 * 0.0633 is 6.3299e+03

anyway to fix this ?

>> 10 ^ 5 * 0.0633

ans =

6.329999999999999e+03

>> 10 ^ 6 * 0.0633

ans =

6.329999999999999e+04

Answer 1

[KALYAN ACHARJYA]

>> 10 ^ 5 * 0.0633
ans =
   6.3300e+03
>> format shortG
>> 10 ^ 5 * 0.0633
ans =
         6330

See the other Format options, you can set the different format the output display

https://in.mathworks.com/help/matlab/matlab_env/format-output.html

[Steven Lord]

Using a different display format masks the issue.

andy, can you post for me the exact decimal value of one divided by three? The only characters you're allowed to use to write that answer are the digits 0 through 9 and a single . for a decimal point. You're not allowed to say that the answer is anything repeating; you must write all the decimal places of your answer explicitly.

Now multiply the answer you gave me by three. The result will not be equal to one. If you had been able to write an infinite number of decimal places it would have been, but you didn't. You rounded off one divided by three and that roundoff error affected the answer.

Answer 2

[James Tursa]

Normal floating point arithmetic effects. See this link:

https://www.mathworks.com/matlabcentral/answers/57444-faq-why-is-0-3-0-2-0-1-not-equal-to-zero?s_tid=srchtitle

The root issue is that 0.0633 cannot be represented exactly in IEEE double floating point format, and downstream calculations using this will be affected as a result. Calculations that mathematically should result in a known value often do not. You need to write floating point code that is tolerant of these effects. E.g., this is the closest number to 0.0633 that IEEE double precision can represent exactly

>> num2strexact(0.0633,'fixed')
ans =
    '0.0632999999999999951594276126343174837529659271240234375'