anova1, anova2, or ranova?

I have a set of data. I have 1 dependant variable measured 3 times from 5 subjects. Each of the 3 measurements represents a different level (A, B, and C). Based on my understanding, I should use a repeated measures ANOVA because I have multiple measurements from each subject. However, I am unable to get any results. I have tried making each subject a different group, and treating all subjects as the same group. I tried a 1-way ANOVA, which to my understanding treats every measure as a different subject, which seems wrong. I tried a 2-way ANOVA where subjects are treated as an independant variable. Using multcompare after anova1 and anova2 give me unexpected results. Based on the data, it seems pretty evident that column 1 and column 3 should both be significantly different than column 2. However, only column 1 vs column 2 is significant. I am unsure if I am doing something wrong, or if there is a different statistical test I should be using. Any help would be greatly appreciated.

clear,clc
close all

DepVar=[101.053914979877,1.10267662721688,39.9882799952772;216.136311110424,1.25119134782814,70.1362721192365;26.5334559925966,1.54372037819131,81.1211148269482;231.177535625968,0.890020347986787,135.886529812943;140.395359593013,0.00626713939985805,37.2109788038507];

% 1-way ANOVA
[p1,tbl,stats1]=anova1(DepVar);
c1=multcompare(stats1);

% 2-way ANOVA
[p2,tbl2,stats2]=anova2(DepVar,1);
c2=multcompare(stats2);

% RM ANOVA
sub=table([1 2 3]','VariableNames',{'Treatment'});

subject1=['1'; '2'; '3'; '4'; '5'];
t1=table(subject1,DepVar(:,1),DepVar(:,2),DepVar(:,3),'VariableNames',{'subject','A','B','C'});
rm1=fitrm(t1,'A-C ~ subject','WithinDesign',sub);
ranovatbl1=ranova(rm1,'WithinModel','Treatment');
c3=multcompare(rm1,'Treatment');

subject2=['1'; '1'; '1'; '1'; '1'];
t2=table(subject2,DepVar(:,1),DepVar(:,2),DepVar(:,3),'VariableNames',{'subject','A','B','C'});
rm2=fitrm(t2,'A-C ~ subject','WithinDesign',sub);
ranovatbl2=ranova(rm2,'WithinModel','Treatment');
c4=multcompare(rm2,'Treatment');

[Scott MacKenzie]

Just to clear things up a bit, it is not correct to say "each of the 3 measurements represents a different independant variable (A, B, and C)". What you have is one independent variable with three levels: A, B, and C.

[Tyler Tuttle]

Thanks for catching that. The question has been edited.

[Scott MacKenzie]

Can I ask.... What is your dependent variable? For example, the first data value is 101.05. What is this? What are the units?

What is your independent variable? You can choose a generic term like "Treatment", but often there is a more descriptive name.

And what are the levels, A, B, and C?

[Tyler Tuttle]

They represent the measured stiffness (units of kilopascals) for 3 different biological materials taken from 5 animals of the same species. Think of A as being like muscle, B as being like fat, and C as being a material with a combination of muscle and fat. So the overall question is whether the stiffness varies between muscle, fat, and a combination of muscle+fat.

[Scott MacKenzie]

OK, thanks. I've been trying a few things with your data. Like you, I am struggling with how to do analyses on data like these in MATLAB. I've been doing analyses like these with other tools for many years, but not with MATLAB.

The first thing I did was analyse your data in a different tool. Before we get to MATLAB, let's examine results that are known to be correct. Using a commercial stats app, the Anova table is

So, the effect of Bio_Material on Stiffness (kPa) was statistically significant, F(2,8) = 10.439, p < .01. A Tukey/Kramer post hoc multiple comparisons test yields

So, the difference was only significant between the Muscle and Fat conditions (p < .05).

I'm fairly confident these analyses are spot on.

So, the challenge now is to do this in MATLAB. As it turns out, your second approach, using anova2, was the correct one. These statements

DepVar=[101.053914979877,1.10267662721688,39.9882799952772;216.136311110424,1.25119134782814,70.1362721192365;26.5334559925966,1.54372037819131,81.1211148269482;231.177535625968,0.890020347986787,135.886529812943;140.395359593013,0.00626713939985805,37.2109788038507];
[p2,tbl2,stats2]=anova2(DepVar, 1);
c2=multcompare(stats2, 'display', 'off')

yield the following Anova table

Here, we see the same F-statistic, p-value, and MS values. The output also includes the results of a multiple comaprisons tests:

c2 =

1 2 53.236 142.1 230.97 0.0045969

1 3 -18.674 70.191 159.06 0.12062

2 3 -160.77 -71.91 16.955 0.11131

Tukey/Kramer is the default test with multcompare, so this is an apples with apples comparison. The result is exactly the same. The difference between the Muscle (1) and Fat (2) conditions was statistically significant (p = .0046). The difference between the other pairs was not.

Hope this helps.

[Tyler Tuttle]

Thanks so much! I do have one more question, but I guess it's more about the statistical result than matlab. Based on looking at the data, I would expect that muscle+fat would be significantly different than fat considering muscle+fat stiffness ranges from 39 to 135, but fat stiffness ranges from basically 0 to 1.5. The stiffness values are different by basically 2 orders of magnitude. Running a paired t-test between the groups leads to significance (p=0.0156). Why is it that the Tukey/Kramer does not show significance, when intuitively you would expect there to be a difference? The fact that the statistical results do not match up to what seems obvious in the data makes me not trust my statistical analysis. Admittedly I do not have much knowledge on the underlying math behing the different types of tests, I just try to apply the tests as correctly as possible based on their descriptions.

[Scott MacKenzie]

Hmm, yes, good question. As you note, there is also a fairly a wide difference between the fat and muscle+fat conditions:

I'd wager that the reason for the insignificant pairwise comparison between the fat and muscle+fat conditions is the small sample size (n = 5). If you had the same differences with a larger n, then I'm guessing the fat vs. muscle+fat pairwise difference would also be significant. BTW, the Fischer PLSD, Scheffe, and Bonferroni/Dunn post hoc tests show the same outcome:

I have an additional question for you. What were the stiffness measrements on? Just curious.

[Tyler Tuttle]

The stiffness measurements are basically the slope of a line fit to tensile-test data from the 3 different tissues.

I've done some similar analysis in the past with small sample size (n=6,7,8) and generally get significance from differences much less drastic than the differences here.

[Scott MacKenzie]

Sorry, to drag this on, but when you say "tissue", what are your referring to? Tissue from what? Again, just curious to know a bit more about the source of the data.

[Tyler Tuttle]

The tissue is the muscle, fat, and muscle+fat from rats. Sacrifice the rats, remove the tissue, test it on a uniaxial tensile testing machine, record force and sample dimensions, plot stress vs stretch, then fit a line to analyse the stiffness.

[Scott MacKenzie]

Hey, thanks for the clarification. Good luck with your research.

[Tyler Tuttle]

Thanks for the help!

[Adam Danz]

That looks like an official answer for the answers section, Scott! 🙂

[Scott MacKenzie]

Adam: To be honest, I'm not sure what you mean. Are you suggesting that I copy and paste my "comment" (which I assume you are referring to) into a "answer" box? Then, Tyler (since its his question) clicks "accept" so it is thereafter seen as the answer. I've noticed occasional disconnects between comments and answers, but I'm new to this, so I don't yet understand the nuances.

[Adam Danz]

Yes, answers are often developed within comment sections as the question becomes clearer. When a definite solution arises, summarizing that solution or copying it to an official answer makes it more visible to future visitors of the thread whether its accepted or not and it gives users the opportunity to give you credit for your work by accepting or voting for your answer. Plus the thread moves into the answered pool rather than lingering in the unanswered queue.

[Scott MacKenzie]

OK, got it. Thanks for the clarification. I'll summary my comments into an answer and move them to an answer box. Cheers.