How do you print the F-stat in a multcompare test?

Heads up, I'm a MATLAB novice. I'm trying to run a Fisher's test on a one-way ANOVA that I conducted comparing the difference in 3 therapy groups. How do I print out the F-statistic for this multiple comparisons test?

The code I'm using:

%data input

ingroups = [5,4,6,7,5,6,7,5,5,6;6,7,7,5,7,7,8,6,7,7;7,8,7,7,8,5,6,7,7,7];

Mon = [5 4 6 7 5 6 7 5 5 6];

Bon = [6 7 7 5 7 7 8 6 7 7];

Feel = [7 8 7 7 8 5 6 7 7 7];

%transpose data rows in to columns for anova1

ingroups2 = transpose(ingroups);

%one-way ANOVA (treats columns as separate groups)

[p,tbl,stats] = anova1(ingroups2);

%Fisher's protected test (least significant difference)

[c,m,h,gnames] = multcompare(stats,'CType','lsd');

Answer 1

[Adam Danz]

You are performing a one-way ANOVA which means your null hypothesis is that there is no difference in the population means of your 3 levels of some factor. This assumes the data in each factor are normally distributed. Below is a quick eye-ball check of that assumption. It's difficult to call these data normally distributed since you have so few data and they are intergers within a narrow range. If these are your real data, I would not be comfortable confirming the normality assumption and therefore I would not be comfortable relying on any parametric test including an ANOVA.

ingroups = [5,4,6,7,5,6,7,5,5,6;6,7,7,5,7,7,8,6,7,7;7,8,7,7,8,5,6,7,7,7]'; % Notice transpose!
tiledlayout(3,1)
nexttile(); histogram(ingroups(:,1)); title('Column 1')
nexttile(); histogram(ingroups(:,2)); title('Column 2')
nexttile(); histogram(ingroups(:,3)); title('Column 3')

Assuming the data provided are example data and that your real data are approximately normally distributed, I'll move on to your original question. The Fisher Exact Test is designed for 2x2 contingency tables. Since you have 3 levels, you'd likely want to use a chi-squared test instead. See Data Science Blog and Analysis of Categorical Data (PDF) for more info.

If you're just trying to get the F-statistic from the ANOVA table,

[p,tbl,stats] = anova1(ingroups,[],'off');

tbl(cellfun('isempty',tbl)) = {nan};
T = cell2table(tbl(2:end,:), 'VariableNames', tbl(1,:))

T = 3×6 table
      Source        SS     df      MS         F        Prob>F  
    ___________    ____    __    _______    ______    _________

    {'Columns'}     9.8     2        4.9    6.1822    0.0061588
    {'Error'  }    21.4    27    0.79259       NaN          NaN
    {'Total'  }    31.2    29        NaN       NaN          NaN

F = T.F(1)

F = 6.1822

[Darla Bonagura]

Hi Adam! Thank you so much for your response. These are in fact sample data for a statistics homework assignment I have (I'm a PhD student). I'm trying to keep MATLAB scripts for all of my assignments so I'll be able to use them with real data later on.

[Adam Danz]

This is what drives me a little crazy in statistics. There is a movement in the stats community to persuade researchers from using significance testing (i.e., p-values) and one reason is the general trend to ignore assumption violations (see this list of literature under "Notes on the p-value"). While 1-way ANOVAs are robust to normality assumption, if there is significant kurtosis, the ANOVA results could be meaningless and with a small sample size, kurtosis could be difficult to recognize. Alternatively, non-parametric tests are free from assumptions related to the shape of distributions and there's usually a non-parametric alternative to parametric tests. Bootstrapped confidence intervals using the percentile method are a great alternative to testing for differences between distributions without worrying about normality. At the very least, it would be nice to see stats assignments using ANOVA provide data that are as close to normally distributed as possible.

[Darla Bonagura]

Yes, I definitely see what you mean. I appreciate your thoughts. I was able to retrieve the F-statistic from the anova1 command I ran (and as you pasted above). But what if I wanted to see the F-statistic for the multiple comparisons test I'm running of an ANOVA? (e.g., Bonferroni). Thank you again for your time and help!

[Adam Danz]

You're using the Lease Significant Difference (LSD) test in the line below,

%Fisher's protected test (least significant difference)

[c,m,h,gnames] = multcompare(stats,'CType','lsd');

This test is use when the ANOVA results are significant, indicating that at least 1 group mean is significantly different from the others. The LSD test computes the smallest significant difference between all paired means while ignoring other means outside of each paired comparison (e.g. t-test) and results in a "signficant difference" if any of the paired means differ by an amount larger than the smallest significant difference. It does not use an F-statistics like the ANOVA does. In fact, none of the multiple comparison options do. You can verify this by looking into how each method is computed in the documentation.

[Darla Bonagura]

Got it, thank you!

Answer 2

[Scott MacKenzie]

@Darla Bonagura There is no F-statistic for post hoc pairwise comparisons tests such as Bonferonni, Scheffe, and so on. There is, of course, an F-statistic for the ANOVA that precedes the post hoc test. The post hoc tests work by comparing the difference in each pair of test conditions with a critical value (computed in different ways for each test). The critical value is computed with a desired significance value, which is most often p < .05. If that difference is greater than the critical value, those two conditions are said to differ significantly (p < .05).

[Darla Bonagura]

Thank you so much!