error decimal aproximation, extra decimal

3 vues (au cours des 30 derniers jours)
Fernando Ramirez Huaca
Fernando Ramirez Huaca le 11 Mai 2020
Modifié(e) : James Tursa le 11 Mai 2020
why in Matlab 0.94-1=-0.0600000000000001? Why is there an extra decimal 1?

Réponses (1)

James Tursa
James Tursa le 11 Mai 2020
Modifié(e) : James Tursa le 11 Mai 2020
Welcome to the world of floating point arithmetic! See this link:
Bottom line ... 0.94 cannot be represented exactly in IEEE double precision. That trailing 1 is actually part of more digits in an exact floating point binary to decimal conversion:
>> fprintf('%.60f\n',0.94-1)
-0.060000000000000053290705182007513940334320068359375000000000
The 0.94 all by itself converted into decimal is:
>> fprintf('%.60f\n',0.94)
0.939999999999999946709294817992486059665679931640625000000000

Catégories

En savoir plus sur Get Started with MATLAB dans Help Center et File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by