Effacer les filtres
Effacer les filtres

Rounding using 'fix'... possibly a bug?

3 vues (au cours des 30 derniers jours)
Brad
Brad le 14 Oct 2013
Commenté : Brad le 15 Oct 2013
Could someone tell me if this is a bug, or is there some logical reason 'fix' would round:
.3/.1 = 2
whereas
.4/.1 = 4
K>> fix(.3/.1)
ans =
2
K>> fix(.4/.1)
ans =
4
K>> fix(.7/.1)
ans =
6
K>> fix(.5/.1)
ans =
5
  1 commentaire
Jan
Jan le 14 Oct 2013
It is such surprising that the well known floating point effects appear for beginners usually for the value 0.3 . Usually it is:
find((0:0.1:1) == 0.3) % replies [] !!!

Connectez-vous pour commenter.

Réponse acceptée

James Tursa
James Tursa le 14 Oct 2013
To see what is going on you can examine the exact floating point values. Note that none of your original numbers except 0.5 can be represented in IEEE double format exactly:
>> num2strexact(.7)
ans =
0.6999999999999999555910790149937383830547332763671875
>> num2strexact(.5)
ans =
0.5
>> num2strexact(.4)
ans =
0.40000000000000002220446049250313080847263336181640625
>> num2strexact(.3)
ans =
0.299999999999999988897769753748434595763683319091796875
>> num2strexact(.1)
ans =
0.1000000000000000055511151231257827021181583404541015625
>> num2strexact(.7/.1)
ans =
6.99999999999999911182158029987476766109466552734375
>> num2strexact(.5/.1)
ans =
5
>> num2strexact(.4/.1)
ans =
4
>> num2strexact(.3/.1)
ans =
2.999999999999999555910790149937383830547332763671875
You can find num2strexact here:
  1 commentaire
Brad
Brad le 15 Oct 2013
Thank you James, much appreciated!

Connectez-vous pour commenter.

Plus de réponses (1)

Image Analyst
Image Analyst le 14 Oct 2013
Looks like another case explained by the FAQ.

Catégories

En savoir plus sur Coordinate Transformations and Trajectories dans Help Center et File Exchange

Produits

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by