how to filter columns

Hello friends,

I have this script. But is very slow!

I want to filter all columns beginning with column first. After that, I want, in the place of column first, to put the second column and, in the place of column second to put the column first and filter again. And go on!

My purpose is decrease the time!

Thanks by support!

clear all
clc
q = 1;
p = 2;
r1 = 1;
r2 = 1;

tic
% Example: Filtering a matrix
load ('tum.mat'); % tum have 576 x 1000

for i1 = 1:1000
    % Filter rows where the first column is greater than 15
    z1 = k2(k2(:,1) == 0, :); 
    s1 = sum(z1, 1);
    r(r1,r2) = max(s1);
    
    temp_row = k2(:,q);
    k2(:,q) = k2(:,p);
    k2(:,p) = temp_row;
    p = p + 1;
    r1 = r1 + 1;
end
toc

Elapsed time is 337.582496 seconds.

[the cyclist]

Can you upload the input data? I tried running your code with a random array, but it threw an error:

clear all
clc
q = 1;
p = 2;
r1 = 1;
r2 = 1;

tic
% Example: Filtering a matrix
k2 =rand(576,1000);
% load ('tum.mat'); % tum have 576 x 1000

for i1 = 1:1000
    % Filter rows where the first column is greater than 15
    z1 = k2(k2(:,1) == 0, :); 
    s1 = sum(z1, 1);
    r(r1,r2) = max(s1);
    
    temp_row = k2(:,q);
    k2(:,q) = k2(:,p);
    k2(:,p) = temp_row;
    p = p + 1;
    r1 = r1 + 1;
end

Index in position 2 exceeds array bounds. Index must not exceed 1000.

toc

[Stephen23]

"I tried running your code with a random array, but it threw an error:"

It cannot run on data of the stated size, because p ends up being greater than the number of columns.

"Elapsed time is 337.582496 seconds."

That is incredible. It would be interesting to know which particular lines or operations are the main contributors.

"My purpose is decrease the time!"

Then why are you swapping columns around? That will be a big performance hit.

Explain what you are actually trying to achieve https://xyproblem.info/

[AIRTON]

Hi Stephen

I have loaded my file load ('tum.mat'). Note that I have changed the name of file to 'tum1.mat".

As I said, I want to filter all colums. Example

the first filter

column 1 filtred column 2 column 3 go on

0 1 0

0 0 0

0 1 1

... ... ...

sum = 2 sum = 1

the second filter

column 2 filtred column 1 column 3 go on

0 1 0

0 0 0

0 1 1

... ... ...

sum = 2 sum = 1

thanks

[AIRTON]

Hi Stephen

As to the msg

"Index in position 2 exceeds array bounds. Index must not exceed 1000."

Try to change "for i1 = 1:1000" to ""for i1 = 1:999"

Thanks

[Image Analyst]

If k2 is a matrix of random numbers, you will never have the numbers be zero so this won't work

k2(:,1) == 0 % Will never happen!

[Matt J]

Elapsed time is 337.582496 seconds.

It does seem impossible for such a small loop to be that slow. Certainly everything excluding the load() statement is very fast, as illustrated in my timing comparison below.

The only explanation I can think of is that you have additional variables in tum.mat that are getting loaded, and which are a lot larger than k2. Or, you are loading from a super slow hard drive.

Answer 1

[Matt J]

m=576; 
n=1000;
k2=randi([0,1],m,n).*rand(m,n);

tic;

 r=max( (k2==0)'*k2 ,[],2);

toc;

Elapsed time is 0.024019 seconds.

[Matt J]

Just to demonstrate that the results agree:

m=576; 
n=1000;
k2=randi([0,1],m,n).*rand(m,n);

tic;
 Result0=max( (k2==0)'*k2 ,[],2);
toc

Elapsed time is 0.007451 seconds.

q = 1;
p = 1;
r1 = 1;
r2 = 1;

tic

for i1 = 1:n
    % Filter rows where the first column is greater than 15
    z1 = k2(k2(:,1) == 0, :); 
    s1 = sum(z1, 1);
    r(r1,r2) = max(s1);
    
    p = mod(p,n)+1;
    temp_row = k2(:,q);
    k2(:,q) = k2(:,p);
    k2(:,p) = temp_row;


    r1 = r1 + 1;

end
toc

Elapsed time is 0.417982 seconds.

Result1=r;

difference=max(abs(Result0-Result1))

difference = 8.5265e-14

[AIRTON]

Hello Matt

Thanks you Very much for your answer!

Please, can you help me as to filter all columns equal zero of the First row of my Matrix.

my_matrix =...
   [1 0 0 0 1 0 0 1 0
    1 1 0 1 0 1 1 0 1
    0 1 1 1 1 1 1 0 1]

my_matrix = 3×9

1 0 0 0 1 0 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 1 1 1 1 0 1

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

[Matt J]

@AIRTO is "my_matrix" the same as "k2" in your original code? If so, what help is required? Why isn't the answer I gave applicable to this example? I get no errors or odd behavior when I run it:

k2=[1 0 0 0 1 0 0 1 0
    1 1 0 1 0 1 1 0 1
    0 1 1 1 1 1 1 0 1];

r=max( (k2==0)'*k2 ,[],2)

r = 9×1

1 1 2 1 1 1 1 2 1

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

[AIRTON]

Hello Matt

excuse me go back with my doubt. You gave this script:

clear all

clc

n=20;

load ('w2.mat');

q = 1;

p = 1;

r1 = 1;

r2 = 1;

for i1 = 1:n

% Filter rows where the first column is equal 0

z1 = w2(w2(:,1) == 0,:);

s1 = sum(z1, 1);

r(r1,r2) = max(s1);

p = mod(p,n)+1;

temp_row = w2(:,q);

w2(:,q) = w2(:,p);

w2(:,p) = temp_row;

r1 = r1 + 1;

end

It has worked well with my w2.mat file.

I want to gain the same result with w3.mat file, however without transposing it.

is it possible?

I have loaded the files!

thanks!

[Stephen23]

@AIRTO: using numbered variables like that is a mistake. Much better to use one container array and indexing:

S = dir('./w*.mat');
for k = 1:numel(S)
    F = fullfile(S(k).folder,S(k).name);
    C = struct2cell(load(F));
    M = C{1}; % this is your matrix.
    ... do whatever you want with your matrix
    S(k).r1 = r1; % save whatever values you want
end