error decimal aproximation, extra decimal
    4 vues (au cours des 30 derniers jours)
  
       Afficher commentaires plus anciens
    
why in Matlab 0.94-1=-0.0600000000000001? Why is there an extra decimal  1? 
0 commentaires
Réponses (1)
  James Tursa
      
      
 le 11 Mai 2020
        
      Modifié(e) : James Tursa
      
      
 le 11 Mai 2020
  
      Welcome to the world of floating point arithmetic!  See this link:
Bottom line ... 0.94 cannot be represented exactly in IEEE double precision.  That trailing 1 is actually part of more digits in an exact floating point binary to decimal conversion:
>> fprintf('%.60f\n',0.94-1)
-0.060000000000000053290705182007513940334320068359375000000000
The 0.94 all by itself converted into decimal is:
>> fprintf('%.60f\n',0.94)
0.939999999999999946709294817992486059665679931640625000000000
0 commentaires
Voir également
Catégories
				En savoir plus sur Logical dans Help Center et File Exchange
			
	Produits
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!

