Prevent rounding of a number.

I'm using the following code to check and correct extreme values in a matrix. The maximum value can not be 1, it must be one quantisable segment below 1. In this example bitDepth is 16 and so maxValue should calculate to be the number commented after it. However, MATLAB just rounds maxValue up to 1 and so my code is failing further along. How can I make MATLAB keep the accuracy of this number?
%Check for clipping
maxValue = 1-(1/2^bitDepth); %0.99998474121093744448884876;
sampleValue = inputBuffer(currentFrameSample, currentChannel);
if sampleValue > maxValue
inputBuffer(currentFrameSample, currentChannel) = maxValue;
elseif sampleValue < (-1.0)
inputBuffer(currentFrameSample, currentChannel) = (-1.0);
end
Thanks in advance, Matt.

 Réponse acceptée

Matt Fig
Matt Fig le 15 Nov 2012
Modifié(e) : Matt Fig le 15 Nov 2012

0 votes

Why do you think MATLAB is rounding 0.99998474121093744448884876 to 1?
F = 0.99998474121093744448884876;
F==1 % No
F<1 % Yes
% Let's look closely
fprintf('\n\n\t%17.17f\n\n',1-(1/2^16))

Plus de réponses (1)

Matt
Matt le 15 Nov 2012

0 votes

Feeling very stupid right now -_-
Thanks very much for you're help! I was convinced it was the rounding that was causing an issue but after you showed me the value was infact correct, I found it was actually a problem further down the code!

Produits

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by