Need help to delete garbage values from a Matrix

% Create the .dat file
clc

format short

tout1=tout;
tout2=tout;
r=size(tout1);
n=r(1,1);
for i=2:n
    if mod(tout1(i)-tout1(i-1), 0.00005) ~= 0
        %if(tout1 > )
        if mod(tout1(i), 0.00005) ~= 0
            tout1(i) = 0;
        else
            tout1(i+1) = 0;
        end
    end
    
end
tout1 = unique(tout1);

tout=tout1;

the above code is written to delete garbage values that are being generated in the a matrix TOUT when i insert display block in simulink. But it makes every third row zero when the above code is implemented. so instead of getting 20001 values from 20005, i am getting 13386 values whic generates error saying that the dimentions of the matrix are not consistant.

why is it deleting extra values which satify the condition in the first loop?

Will like some help please!!!!!!!

[Guillaume]

You've failed to explain what your definition of a garbage value is, and since your code is very bizarre and not commented, it's really not clear what it is.

Note that your code doesn't do any deletion unless you consider that taking the unique values of a vector is a deletion.

[David Goodmanson]

Hi Tejas,

With the mod commands it looks like you may be using error-prone exact equality checks with floating point numbers.

[Guillaume]

That's one of the many bizarre things about the code. Also the code appears to check that the difference between continuous numbers is a multiple of 0.00005 but later on that it's the numbers themselves that are multiple, which is a completely different thing. The difference between two numbers can be a multiple of 0.00005 without any of the numbers being a multiple.

Other bizarre thing, is that the difference is between index i and i-1 but it's i or i+1 that is later modified.

[Tejas Rivonkar]

well the code doesn't delete the values rather makes the garbage value zero. the unique removes the repeated zeros only leaving on value behind.

sorry for not providing the comment for the code. i will provide them as follows

% Create the .dat file
clc
format short
tout1=tout;      %assigning matrix tout to new variable tout1
tout2=tout;      %then assigned another variable tout2 to check the matrix tout
r=size(tout1);   %size of tout is assigned to variable r 
n=r(1,1);        %n is count that is the first row and first column of r
for i=2:n        %starts from 2 since the index starts from 1 in MATLAB
    if mod(tout1(i)-tout1(i-1), 0.00005) ~= 0   %the diffence should be equal throughout i.e. 0.00005
       
        if mod(tout1(i), 0.00005) ~= 0     %incase above loop is not satified then enter this loop to decide to make which value zero
            tout1(i) = 0;
        else
            tout1(i+1) = 0;
        end
    end
    
end
tout1 = unique(tout1); %to delete the repeated zeros
tout=tout1;

if the code is used on a model that is ran for 1 sec, it is supposed to generate 20001 values in tout matrix but instead it genrates 20006 values. the above code is part of Code used for COMTRADE file generation as mentioned below

% Create the .dat file
clc

format short

tout1=tout;
tout2=tout;
r=size(tout1);
n=r(1,1);
for i=2:n
    if mod(tout1(i)-tout1(i-1), 0.00005) ~= 0
        %if(tout1 > )
        if mod(tout1(i), 0.00005) ~= 0
            tout1(i) = 0;
        else
            tout1(i+1) = 0;
        end
    end
    
end
tout1 = unique(tout1);

tout=tout1;

NS1 = size(tout);                 % to find the number of samples
NS2 = NS1(1,1);
FileName = input('Please enter a file name:', 's');
FN_cfg = [FileName,'.cfg'];
FN_dat = [FileName,'.dat'];

Data =[transpose((1:1:NS2)) tout M];
csvwrite(FN_dat,Data)

following is the file(tout2) that is being produced when the part of the code is ran for 1 sec

when the code is ran it becomes as mentioned in x

but for some reason it is deleting every value after 2 execution

please let me know if there is any problem with the logic that is being used.

Answer 1

[Guillaume]

First, as pointed out by David, your co

de doesn't take into account the properties of floating point numbers. I would strongly advise you read up about it, The floating point guide is a very good starting point. The consequence of it is that your code is not going to work the way you expect. Many numbers cannot be stored exactly and thus many numbers that you think should be a multiple of 0.00005 are not for a computer:

>> mod(0.0006 - 0.00055, 0.00005)
ans =
      4.99999999999999e-05

See that for a computer (note that it's not restricted to matlab), the difference between 0.0006 and 0.00055 is not 0.00005. That's because neither 0.0006 nor 0.00055 can be stored exactly, 0.0006 is stored as a number slightly smaller (0.000599999999999999947437878677902745039318688213825225830078125 exactly) and 0.00055 as a number slightly larger (0.000550000000000000033133218391157015503267757594585418701171875) so the difference is slightly smaller than 0.0005 (off by ~8e-20).

So, as is your check is not going to work. That's not even taking into account the other bugs such as checking the difference between index i and i-1 but modifying index i or i+1.

Secondly, you still haven't spelled out what the criteria for garbage is. I'm going to take a guess and assume that all your numbers should be multiple of 0.0005 (taking into account precision of floating point numbers). Note that if numbers are multiple of 0.0005 it makes no sense checking if there difference is also a multiple of 0.0005, it's guaranteed.

%tout: a vector of number. The below removes numbers that are not multiple of 0.00005
tol = 1e-10;  %abritrary tolerance below which numbers are considered equal
tout(abs(mod(tout, 0.00005)) > tol) = []

Note: 0.00005 is one of these rare numbers that can be stored exactly. So, in the above you could actually do an exact check (but not on the difference!). However, for most other divisors you can't, so it's better to always use a tolerance.

[Tejas Rivonkar]

yes as you said the difference between the two values is suppoed to be 0.00005 as i have provided the step of 0.00005 in the Powergui of the model. so each step is supposed to be 0.00005.

now when i put a display block in the model, random values start to enter which are not steps of 0.00005 creating extra values which distort the matrix consistency.

for 1 sec its producing 5 extra values

for 2 sec its producing 16 extra values

for 3 sec its producing 1864 extra values

after that it remains constant.

for less then 1 sec it remains 5 extra values and for more then 3 sec it remains 1864 extra values.

so i want to know how will i rectify it.

[Guillaume]

"so i want to know how will i rectify it."

If the signal values must be multiple of 0.00005, then the code I've written above will do it:

%tout: a vector of number. The below removes numbers that are not multiple of 0.00005
tol = 1e-10;  %abritrary tolerance below which numbers are considered equal
tout(abs(mod(tout, 0.00005)) > tol) = []

it indeed removes the first 5 values in your demo file.

If you want the difference between consecutive values to be in step of 0.00005 then I'm not sure what you want to be done when it's not the case. Unless they're at the beginning or the end, removing values is certainly not going to bring you back your 0.00005 step.