Something wrong with "floor" or "fix" functions or it's my code?

Hello,

row and col contain a lot of values of rows and cols. I'm trying to scale down the rows and cols to 1/4 sizes (rs,cs) and ps is the linear index in the scaled down matrix.

rs = fix((row-1)/4) + 1; %scale down of rows

cs = fix((col-1)/4) + 1; %scale down of cols

ps = (cs - 1)*140 + rs; %linear index in 140x140 matrix

It complains that the Index must not exceed 19600. I isolated the problem whenever the col = 559, the cs = 141 instead of 140. If I run the code individually on the command window >> cs = floor((559-1)/4) + 1; or >> cs = fix((559-1)/4) + 1; it gives me the correct value 140.

Anyone sees the problem that I can't see? I appreciate your help.

[Cris LaPierre]

If the error only appears when you pass in a vector, then you will likely need to share you data for us to see it. You can save your variables to a mat file and then attach that to your post.

[DB]

See attached variables: Pointers, row, col(they are int32).

Below are the calculations for row and col before the calculation above:

row = rem(Pointers-1,560) + 1;

col = (Pointers-row)/560 + 1;

Appreciate anyone who can help spot the problem.

Answer 1

[Stephen23]

"Something wrong with "floor" or "fix" functions"

Nope, in fact those functions do absolutely nothing in your code.

"or it's my code?"

Yes.

"Anyone sees the problem that I can't see?"

The problem is that your data are pure integers. MATLAB clearly tells us that they are INT32 class:

S = load('variables.mat')

S = struct with fields:
    Pointers: [2038×1 int32]
         col: [2038×1 int32]
         row: [2038×1 int32]

When performing arithmetic on integer class values, any fractional values will be rounded in the results. This is explained here:

https://www.mathworks.com/help/matlab/matlab_prog/integers.html

which states "Arithmetic operations that involve both integers and floating-point numbers always result in an integer data type. MATLAB rounds the result..."

Calling FIX or FLOOR afterwards on some integers does absolutely nothing. And naturally some input values will round up, as your example values show:

col = int32(559) % use the correct INT32 type, not like your test.

col = int32 559

cs = fix((col-1)/4) + 1

cs = int32 141

Lets do it step-by-step:

tmp = col-1

tmp = int32 558

tmp = tmp/4 % note this is already ROUNDed, and the output is also integer type!

tmp = int32 140

tmp = tmp+1

tmp = int32 141

tmp = fix(tmp) % ... so FIX/FLOOR does absolutely nothing.

tmp = int32 141

If you do not want rounding then either:

• modify your algorithm to work correctly with that integer type, or
• do not perform arithmetic on integer data type: convert that data to DOUBLE, do your arithmetic, and then FIX/FLOOR as required:

fix((double(col)-1)/4) + 1

ans = 140

This is probably what you expected that intermediate value to be:

(double(col)-1)/4 % not 140

ans = 139.5000

Note that floating point has other issues that you will need to consider.

Even more important: learn to debug. Code does not care what you think it should be doing. It is your task to look at what it really is doing. And the first and most basic step in that, is to look at the data at every step.

[Walter Roberson]

You are right, my guess about the cause turns out to wrong for the circumstances.

Answer 2

[Walter Roberson]

I predict that your col was not calculated using pure integers, and instead was calculated by a process that divided by a number and later multiplied by the number. For example if you had (0:0.1:5) * 10 then you will not get back perfect integers:

x = 0:0.1:5;
x(x*10 ~= round(x*10))

ans = 1×10

0.3000 0.6000 0.7000 1.2000 1.4000 2.3000 2.4000 2.6000 2.7000 3.1000