Running repeated anova on all elements of large matrix

Dear Experts,

I have the following problem. I have a large (3344) stack of matrices, stacked along the 3rd dimension. Each of these matrices is symmetric (319*319). The matrices are repeated measurements of 4 conditions for 836 subjects (therefore they are 836*4=3344).

Now, I would like to run a repeated measures ANOVA for each element of these matrices to obtain a final 319*319 matrix which is the F test value for each element for the factor "condition" across subjects. I understand that an easy way of doing this would be to loop through each element of the matrix, build a table where each of the 4 conditions becomes a column and each subject is a row and then use fitrm and ranova to obtain an F value. However, this method is computationally very slow and I was wondering if someone had suggestions to speed it up (besides running it only on half of the matrix of course). The reason for this is that the resulting F matrix needs to be the input in a permutation test, so ideally this would have to be repeated hundreds/thousands of times.

So far the (slow) code I have is this:

stack is the 3344 matrices.

G is a vector which determines which condition each 319*319 matrix belongs to.

Fmat=nan(size(stack, 1), size(stack, 2));
for i=1:size(stack, 1)
    for j=1:size(stack, 1)
        if i==j
            Fmat(i,j)=nan; % the diagonal is always nan
        else
            % format the data as required by matlab
            t = table([subidx],squeeze(stack(i, j, G==1)), squeeze(stack(i, j, G==2)), squeeze(stack(i, j, G==3)), squeeze(stack(i, j, G==4)), 'VariableNames',{'IDs','d1', 'd2', 'd3', 'd4'});
            Meas = dataset([1 2 3 4]','VarNames',{'Measurements'});
            % run repeated measures ANOVA
            rm = fitrm(t,'d1-d4 ~ IDs-1','WithinDesign',Meas);
            ranovatbl = ranova(rm);
            %save output
            Fmat(i,j)=ranovatbl.F(1);
        end
    end
end

Thank you very much for any suggestions,

Leonardo Tozzi

Answer 1

[Jeff Miller]

If you really just want the F value, you can probably get it much faster by looking up the formulas for the one-factor within-Ss ANOVA and computing the F directly from those--that is, forget about fitrm and ranova. fitrm and ranova are doing a lot more work than you need, starting with decoding the model specification from a string and figuring out what computations need to be done. For example, this seems to have the relevant computational formulas: link

Answer 2

[Leonardo Tozzi]

Dear Jeff,

This is very useful, thank you very much! The execution time is now down to ~5 seconds. I am copying the solution here, in case anyone is interested.

Here is the modified code:

M is the stack of matrices

condsvec is a vector which determines which condition each matrix belongs to

subsvec is a vector which determines which subject each matrix belongs to

N = size(M,1);
subnum=max(find(condsvec==1)); % find number of subjects (assumption is complete data)
condnum=max(condsvec);

% create F map 
% Avoiding for loops by calculating Fs manually
% Reference: https://www.discoveringstatistics.com/repository/onewayrmhand.pdf

% Step 1: Calculate SSt

grandvar=var(M,0,3);
SSt=grandvar.*(N-1);

% Step 2: Calculate the Within-Participants SS 

for sub=1:subnum
    
   varssubs(:,:,sub)=var(M(:,:,find(subsvec==sub)),0,3).*(condnum-1);
    
end

SSw=sum(varssubs, 3);
dfw=subnum*(condnum-1);

% Step 3: Calculate the Model Sum of Squares 

grandmean=mean(M,3);

for cond=1:condnum
    
    groupssm(:,:,cond)=((mean(M(:,:,find(condsvec==condnum)), 3)-grandmean).^2)*subnum;
    
end

SSm=sum(groupssm,3);
dfm=condnum-1;

% Step 4: Calculate the Residual Sum of Squares 

SSr=SSw-SSm;
dfr=dfw-dfm;

% Step 5: Calculate the Mean Squares

MSm=SSm/dfm;
MSr=SSr/dfr;

% Step 6: Calculate The F-Ratio 

Fmat=MSm./MSr;