Why 0.35 divide 0.001 return double, and 0.34 divide 0.001 return int.

simple codes:
>> 0.35/0.001

ans =

  350.0000

and

  >> 0.34/0.001

ans =

   340

I'm curious why this is. If use 0.351/0.001, it return 351.0000, and 0.352/0.001, it return 352.

[zongxian]

This is from when I use zeros(0.34/0.001) or zeros(0.35/0.001), the second will have an error.

Answer 1

[Walter Roberson]

MATLAB does not represent numbers with fractions in decimal (not even in the Symbolic toolbox, but you have to push hard to prove that for the Symbolic toolbox).

MATLAB itself represents most numbers in IEEE 754 Double Precision, which is a representation that uses one sign bit, 11 bits of binary exponent, and 52 bits of binary fraction. For technical reasons, this gives 53 bits of precision: numbers are effectively represented as a 53 bit integer times a power of 2 (that might be negative). The only fractions that can be exactly represented are those involving powers of 2. It is like having an unreduced fraction in which the denominator is always 2^53. Other denominators such as 10 are only possible if they are powers of 2: for example the system can exactly represent 16ths, but never 3rds or 10ths, not exactly.

This system has exactly the same variety of limitations that sticking strictly to a fixed number of decimal places would have. For example if you pick any fixed number of decimal places, you can never exactly represent 1/3 or 1/7, and instead those end up in decimal being infinite repeating decimals, just like how 1/3 in decimal starts 0.333333333333333 but continues on infinitely. Now in decimal, take that number and multiply it by 3 again, and what you get is 0.999999999999999 rather than exactly 1.

Just so, in binary, 1/10 is an infinite repeating number, and if you truncate it to any finite number of decimal places, and multiply by 10 you do not get exactly 1 back.

Now, with some multiples of 1/10 in binary, when you multiply by 10, the result rounds to the representation of an integer, but that is not the case for all multiples of 1/10 in finite binary: for some of them, after multiplication you get a number that is 1 bit different than an exact integer, just like the 0.99999999999999 is 1 trailing decimal place different from an exact decimal integer.

So 0.34/0.001 with both numbers expressed in finite binary approximation ends up rounding to an exact integer but 0.35/0.001 with both numbers expressed in finite binary approximation ends up rounding to one bit different from an exact integer.

The moral of the story is: Don't Do That!! Avoid using fractions to calculate integer indices, because fractions are only approximations.

Example: instead of

for f=0.01:0.01:1
  G(100*f)=f.^2;
end

You can use

for fi = 1:100
  f=fi/100;
  G(fi) = f.^2;
 end

Answer 2

[James Tursa]

Welcome to the world of floating point arithmetic. In one case, the result is 340 exactly so it prints without any trailing 0's after the decimal point. In the other case, the result is not 350 exactly so it prints with trailing 0's after the decimal point. This is all a result of the fact that the original values you are using cannot be represented exactly in IEEE double precision, so you get round-off errors that behave slightly differently in one case vs the other case. See this link for more discussion on this:

https://www.mathworks.com/matlabcentral/answers/57444-faq-why-is-0-3-0-2-0-1-not-equal-to-zero

E.g.,

>> fprintf('%.50f\n',0.001)
0.00100000000000000002081668171172168513294309377670
>> fprintf('%.50f\n',0.35)
0.34999999999999997779553950749686919152736663818359
>> fprintf('%.50f\n',0.35/0.001)
349.99999999999994315658113919198513031005859375000000
>> fprintf('%.50f\n',0.34)
0.34000000000000002442490654175344388931989669799805
>> fprintf('%.50f\n',0.34/0.001)
340.00000000000000000000000000000000000000000000000000
>> 
>> 350 == 0.35/0.001
ans =
  logical
   0
>> 340 == 0.34/0.001
ans =
  logical
   1

Answer 3

[Subhamoy Saha]

Not sure why it is showing like that but the case is both are double.

a=0.351/.001
whos a
b=0.352/.001
whos b