speed of loop for calculating group version

Hi every body. I have a code in matlab which I want to generalize it to group version:

the prior code is :

statsCont = all(max(abs(bsxfun(@minus, Beta(:,statsIt), Beta(:,1:statsIt))),[],1) ./ (Lambda(statsIt)+Lambda(1:statsIt)) - (3/(2*nobs*cStats)) <= 0);

Now I want to generalize it to group version, so if I replace previous code with:

    for i=1:statsIt-1       
            temp_g=[];
            for j=1:ngroups
               temp_g=[temp_g ,norm( Beta(groups==j,statsIt)- Beta(groups==j,i),2)];
            end
             if((max(temp_g)/(Lambda(statsIt)+Lambda(i))) - (3/(cStats*nobs*2)) > 0)      
             statsCont= false; 
              break;
            end
           end

The speed of this code is much more slower than previous code. Could you please help me by a faster code?

Answer 1

[Jan]

This is a very bad idea:

temp_g = [];
for j = 1:ngroups
   temp_g = [temp_g ,norm(Beta(groups==j,statsIt) - Beta(groups==j,i),2)];
end

Letting an array grow iteratively is very expensive. If you do this e.g. for 1000 elements, Matlab have to allocate memory for sum(1:1000) elements: 1 at first, than 2 and the formerly existing elements is copied, then 3 elements and the 2 former elements are copied and so on. Solution: Pre-allocation!

While norm calculates the square root of each element, collecting the absolute values is sufficient, if you only need the largest norm value. Then you need 1 sqrt only.

for i=1:statsIt-1
    temp_g = zeros(1, ngroups);  % Allocate final array size
    for j = 1:ngroups
       match     = (groups==j);
       temp_g(j) = abs(Beta(match, statsIt)- Beta(match, i));
    end
    max_temp_g = sqrt(max(temp_g));
    if max_temp_g / (Lambda(statsIt) + Lambda(i)) - (3 / (cStats*nobs*2)) > 0
       statsCont = false; 
       break;
    end
 end

[Mahsa Taheri]

Thank you very much for this answer, actually I tried this code, but it is not as fast as I want.

I can't underestand why this group version can't be as fast as all(max....), which is in first line of question. Is there any thing else to speed up my code?

[Jan]

" but it is not as fast as I want."

I do not know, which speed you need. But I assume, my suggestion is faster than the original code. If you post the timings, it would be clearer, if the wanted speed is possible. I cannot test the code by myself, because you did not post some inputs data.

Why do you create all distances and search for the maximum value for a comparison instead of comparing the distances directly?

% UNTESTED!!!
for i = 1:statsIt-1
    limit = (3 * (Lambda(statsIt) + Lambda(i)) / (cStats*nobs*2))^2;
    for j = 1:ngroups
       match  = (groups==j);
       temp_g = abs(Beta(match, statsIt)- Beta(match, i));
       if temp_g > limit
           statsCont = false; 
           break;
       end
    end
end

Now the calculation stops, when the first element exceeds the limit. This is cheaper then getting all elements and checking the maximum element.

But without input data, I cannot test if this code replies exactly the same as the original one.