Main Content

Quantization

Precision, range, and scaling of fixed-point data types

When developing a dynamic system using floating-point arithmetic, you generally do not have to worry about numerical limitations since floating-point data types have high precision and range. Conversely, when working with fixed-point arithmetic, you must consider these factors when developing dynamic systems:

  • Quantization

    Fixed-point values are rounded. Therefore, the output signal to the plant and the input signal to the control system do not have the same characteristics as the ideal discrete-time signal.

  • Overflow

    Adding two sufficiently large negative or positive values can produce a result that does not fit into the representation. This will have an adverse effect on the control system.

  • Computational noise

    The accumulated errors that result from the rounding of individual terms within the realization introduce noise into the control signal.

  • Limit cycles

    In the ideal system, the output of a stable transfer function (digital filter) approaches some constant for a constant input. With quantization, limit cycles occur where the output oscillates between two values in steady state.

Categories

  • Precision
    Limitations on precision, effects of rounding and padding
  • Range
    Limitations on range, underflows and overflows, saturation and wrapping
  • Scaling
    Effects of scaling on fixed-point arithmetic, binary-point only scaling, slope-bias scaling, scaling for speed, and scaling for maximum precision

Featured Examples