Find groups of duplicate values and preserving order?

Hi all,

I have a 100x1 double matrix

matrix=[100--100].*rand(100,1)-100;
C=unique(matrix);

I would love to create a length(C) x length(matrix) matrix which contains the positioning index from matrix to later conduct a function.

Thank you for your time,

[Rik]

It is not clear to me what the desired output would be. Can you give a small explicit example with inputs and outputs to make clear what you mean?

[Trung Ngo]

Hi Rik, sorry for being unclear.

Let's say I have a matrix of 100x1 double and C is a unique(matrix). I want to search C(i) that match the index of matrix and return an index matrix of that search position.

It is because for each element from matrix, I have to load a corresponding matrix of 60mb and this consumes a lot of time. I don't know how can I group the dupplicate value from matrix and show its original order.

[Rik]

Can you make a small example to explain what you want?

[Trung Ngo]

Hi Rik, here is my code

[B, I] = sort(check_value);
lat_sorted=lat(I,:);
lon_sorted=lon(I,:);
for i=1:length(B)
    for j=1:length(B)-1
        if B(i+j)==B(i)
            if B(i)==11
                L = load('110.mat');
            elseif B(i)==12
                L = load('120.mat');
            elseif B(i)==13
                L = load('130.mat');
            elseif B(i)==14
                L = load('140.mat');
            elseif B(i)==15
                L = load('150.mat');
            elseif B(i)==16
                L = load('160.mat');
            elseif B(i)==17
                L = load('170.mat');
            elseif B(i)==18
                L = load('180.mat');
            elseif B(i)==19
                L = load('190.mat');    
            elseif B(i)==191
                L = load('191.mat');
            elseif B(i)==192
                L = load('192.mat');
            elseif B(i)==193
                L = load('193.mat');
            elseif B(i)==21
                L = load('210.mat');
            elseif B(i)==22
                L = load('220.mat');
            elseif B(i)==23
                L = load('230.mat');
            elseif B(i)==24
                L = load('240.mat');
            elseif B(i)==25
                L = load('250.mat');
            elseif B(i)==26
                L = load('260.mat');
            elseif B(i)==27
                L = load('270.mat');
            elseif B(i)==28
                L = load('280.mat');
            elseif B(i)==29
                L = load('290.mat');    
            elseif B(i)==291
                L = load('291.mat');
            elseif B(i)==292
                L = load('292.mat');
            elseif B(i)==293
                L = load('293.mat');
            elseif B(i)==31
                L = load('310.mat');
            elseif B(i)==32
                L = load('320.mat');
            elseif B(i)==33
                L = load('330.mat');
            elseif B(i)==34
                L = load('340.mat');
            elseif B(i)==35
                L = load('350.mat');
            elseif B(i)==36
                L = load('360.mat');
            elseif B(i)==37
                L = load('370.mat');
            elseif B(i)==38
                L = load('380.mat');
            elseif B(i)==39
                L = load('390.mat');    
            elseif B(i)==391
                L = load('391.mat');
            elseif B(i)==392
                L = load('392.mat');
            elseif B(i)==393
                L = load('393.mat');
            end
            for l=i:i+j
                tf(l) = interp2(L.x,L.y,L.is_land,lon_sorted(l),lat_sorted(l),'cubic');
            end
       elseif B(i+j)~=B(i)
            continue;
       end
    end
end

As you can see B(1)=B(2)=B(3)=23 and I do not want to load them 3 times, I just want to load them 1 time only and run tf(l) 3 times for l=i:i+j (in this case i=1 and j is from 1 to 2)

Answer 1

[Rik]

The code below is far from optimal, but it should help you out. It assumes there is always an L to load, and you have enough memory to store all the different mat files in a library. These assumptions are to memoization. You could improve the readability of this code by creating a masterlist of indices and file names, instead of endless elseifs. Then it is very easy to add or remove a new mat and it makes your code more readable. You can also check if the output of ismember sums to 1 to check for valid inputs.

function L=load_mat(Bi)
persistent L_libary
if isempty(L_libary),L_libary={};end
if numel(L_libary)>=Bi && ~isempty(L_libary{Bi})
    L=L_libary{Bi};
end
if Bi==11
    L = load('110.mat');
elseif Bi==12
    L = load('120.mat');
elseif Bi==13
    L = load('130.mat');
elseif Bi==14
    L = load('140.mat');
elseif Bi==15
    L = load('150.mat');
elseif Bi==16
    L = load('160.mat');
elseif Bi==17
    L = load('170.mat');
elseif Bi==18
    L = load('180.mat');
elseif Bi==19
    L = load('190.mat');
elseif Bi==191
    L = load('191.mat');
elseif Bi==192
    L = load('192.mat');
elseif Bi==193
    L = load('193.mat');
elseif Bi==21
    L = load('210.mat');
elseif Bi==22
    L = load('220.mat');
elseif Bi==23
    L = load('230.mat');
elseif Bi==24
    L = load('240.mat');
elseif Bi==25
    L = load('250.mat');
elseif Bi==26
    L = load('260.mat');
elseif Bi==27
    L = load('270.mat');
elseif Bi==28
    L = load('280.mat');
elseif Bi==29
    L = load('290.mat');
elseif Bi==291
    L = load('291.mat');
elseif Bi==292
    L = load('292.mat');
elseif Bi==293
    L = load('293.mat');
elseif Bi==31
    L = load('310.mat');
elseif Bi==32
    L = load('320.mat');
elseif Bi==33
    L = load('330.mat');
elseif Bi==34
    L = load('340.mat');
elseif Bi==35
    L = load('350.mat');
elseif Bi==36
    L = load('360.mat');
elseif Bi==37
    L = load('370.mat');
elseif Bi==38
    L = load('380.mat');
elseif Bi==39
    L = load('390.mat');
elseif Bi==391
    L = load('391.mat');
elseif Bi==392
    L = load('392.mat');
elseif Bi==393
    L = load('393.mat');
end
L_libary{Bi}=L;
end

[Trung Ngo]

Hi Rik, do you think this one is a good idea

ii = [0, diff(B(:)')==0,0];
i1 = strfind(ii,[0 1]);
i2 = strfind(ii,[1 0]);
out = [B(i1),i1(:),i2(:)];

I searched around and come up with this thread: https://www.mathworks.com/matlabcentral/answers/224769-how-to-get-the-starting-and-ending-index-of-repeated-numbers-in-an-array

However, it does not account for the element from B without repetition.

[Rik]

What I meant is that you do something like this:

[B, I] = sort(check_value);
lat_sorted=lat(I,:);
lon_sorted=lon(I,:);
for i=1:length(B)
    for j=1:length(B)-1
        if B(i+j)==B(i)
            L=load_mat(B(i))
            for l=i:i+j
                tf(l) = interp2(L.x,L.y,L.is_land,lon_sorted(l),lat_sorted(l),'cubic');
            end
       elseif B(i+j)~=B(i)
            continue;
       end
    end
end

Of course the function in my answer needs to be on the Matlab path (so e.g. in your current folder).

The function in my answer makes sure you only load a mat file once by storing it in a persistent variable. A persistent variable is reset with some syntaxes of clear, or by editing the function file it is contained in. It does not persist over Matlab sessions.

[Trung Ngo]

Hi Rik, thank you for your details reply,

This is what I have done so far. I create a matrix that will group the dupplicate elements' index from B in 1 row.

clearvars;
load('B.mat');
k=[0,2,0];
l=diff(B);
l_add = [1; l];
in=find(l_add~=0);
out=find(l_add==0);
max=max(diff(in));
result=zeros(length(in),max+1);
for i=1:length(in)
    for j=1:max+1
        if i<length(in)
            if j==1
                result(i,j)=B(in(i));
            elseif j==2
                result(i,j)=in(i);
            else
                for l=3:max+1
                    if in(i)+l-2<in(i+1)
                        result(i,l)=in(i)+l-2;
                    elseif in(i)==in(i+1)-1
                        continue
                    else
                        continue
                    end
                end
            end
        else
            if j==1
                result(i,j)=B(in(i));
            elseif j==2
                result(i,j)=in(i);
            else
                for l=3:max+1
                    if in(i)+l-2<=out(length(out))
                        result(i,l)=in(i)+l-2;
                    end
                end
            end
        end
    end
end

From there I can reduce the search amount from 20 to 13 and apply your load_mat function, which will reduce most of the computing time

[Trung Ngo]

By the way, I think there is a typo in your code, should it be

function L=load_mat(Bi)

if numel(L_libary)>=Bi && ~isempty(L_libary{Bi})

[Rik]

Well spotted. I've edited my answer accordingly. (I've shortened your comment to include only the change for readability of this thread)

[Trung Ngo]

Thanks Rik for your help, wish you the best