Computer words consist of a finite number of bits. This means that the binary encoding of variables is only an approximation of an arbitrarily precise real-world value. Therefore, the limitations of the binary representation automatically introduce limitations on the precision of the value.
The range of a number gives the limits of the representation, while the precision gives the distance between successive numbers in the representation. The range and precision of a fixed-point number depend on the length of the word and the scaling.
To maximize precision, make the slope as small as possible while keeping the range adequately large.
Rounding involves going from high precision to lower precision and produces quantization errors and computational noise.
- Rounding Mode: Ceiling
- Rounding Mode: Convergent
- Rounding Mode: Floor
- Rounding Mode: Nearest
- Rounding Mode: Round
- Rounding Mode: Simplest
- Rounding Mode: Zero
Fixed-point Simulink® blocks support seven different rounding modes.
Net slope and bias precision, detecting precision loss, underflow, and overflow.
This example shows how to detect fixed-point constant precision loss.
How to avoid precision loss by overriding the data types in your model with scaled doubles.