Same calcuation, but hand-result is different from code result

I have this code:

aoamid = (2*0.6*sqrt(1-((0)/4)^2)/(pi^2)+0.6/(8*pi))
CLnew = (aoamid+0.01)/(2*sqrt(1-((0)/4)^2)/(pi^2)+1/(8*pi))
m = (CLnew-0.6)/(0.01)
aoafinal = aoamid - 0.6/m

However, the result from hand-calculation is different from the answer generated by the code: (left is hand-calculated result)

Answer 1

[Star Strider]

This simply appears to be a precision issue. Using round on the intermediate results provides the same result as the calculator —

format longG
aoamid = (2*0.6*sqrt(1-((0)/4)^2)/(pi^2)+0.6/(8*pi))

aoamid = 
          0.14545866183459

CLnew = (aoamid+0.01)/(2*sqrt(1-((0)/4)^2)/(pi^2)+1/(8*pi))

CLnew = 
         0.641248832653383

m = (CLnew-0.6)/(0.01)

m = 
           4.1248832653383

aoafinal = aoamid - 0.6/m

aoafinal = 
      3.05311331771918e-16

Check = round(aoamid,4) - 0.6/round(m,4)

Check = 
      4.19282891706563e-05

.

Answer 2

[Image Analyst]

If you use more digits than 4 (like the script uses) you'll see that the values are very close -- both essentially zero being around 1e-16. When you're dealing with numbers in the 0.1 to 100 range or whatever, anything that's a difference that's less than about 1e-8 or so is essentially zero and due to truncation error.

format long g
aoamid = (2*0.6*sqrt(1-((0)/4)^2)/(pi^2)+0.6/(8*pi))

aoamid = 
          0.14545866183459

CLnew = (aoamid+0.01)/(2*sqrt(1-((0)/4)^2)/(pi^2)+1/(8*pi))

CLnew = 
         0.641248832653383

m = (CLnew-0.6)/(0.01)

m = 
           4.1248832653383

aoafinal = aoamid - 0.6/m

aoafinal = 
      3.05311331771918e-16

aoafinal_using4digits = 0.1455 - 0.6 / m

aoafinal_using4digits = 
      4.13381654106593e-05

% now do with a lot more decimal places:
aoafinal_fullprecision = 0.14545866183459 - 0.6 / 4.1248832653383

aoafinal_fullprecision = 
       7.7715611723761e-16

See, the difference is around 1e-16 when using script variables or full precision of about 14 decimal places, but when you use only 4 decimal places, the difference (error) is more, like 1e-5.