How to improve double precision?
    3 vues (au cours des 30 derniers jours)
  
       Afficher commentaires plus anciens
    
    NoYeah
 le 23 Juil 2020
  
    
    
    
    
    Modifié(e) : Stephen23
      
      
 le 25 Juil 2020
            I have made below code
a=11;
b=256;
c=a/b
and the output said c=0, which was not I had expected
So I have used cast operator
a=11;
b=256;
c=double(a/b)
and the result returns the same value; 0
I have searched Matlab document and found that the default precision of double data type is 32
so It should return 0.04296875 instead of 0. But still get 0 for this kind of codes
How to fix it properly?
4 commentaires
  dpb
      
      
 le 23 Juil 2020
				I asked you to type in teh above commands on  your system and tell us what you get?
Réponse acceptée
  Walter Roberson
      
      
 le 23 Juil 2020
        Either your a or your b is type uint8. When you work with int8, int16, int32, uint8, uint16, or uint32, then the operands are converted to double(), and the operation is done, and then the result is converted back to the most restricted type used.
Operations for int64 and uint64 are handled differently in some, because 64 bit integers cannot be converted to double without losing precision.
Plus de réponses (0)
Voir également
Catégories
				En savoir plus sur Logical dans Help Center et File Exchange
			
	Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!



