# Which pairwise hypothesis test is used on multcompare

22 views (last 30 days)
Reinder Dorman on 3 Dec 2020
Commented: Ive J on 26 Oct 2022
HI All,
I have a feeling this question should be easily answered, so I suspect I might just be missing something.
On the function page of multcompare it aptly states that it performs the appropriate pairwise testing based on the stats input. But I'm trying to find what actual test is used. On the function page it keeps talking about a "hypothesis test", but I can't seem to find which actually test is being used.
In the "More about" section, it talks about t-tests, but I'm not sure if this isn't used as an example or if it means that is simply performs t-tests on the input (which would be performs on the the appropriate type of data, for instance on the ranks when using Kruskal-Wallis?).
Any help is greatly appreciated.
Ive J on 5 Dec 2020
Edited: Ive J on 6 Dec 2020
Yes it is correct. But the actual test being used depends on the background test; for instance a test like ANOVA would require a t-tes while a non-parametric ones like Kriskal-Wallis would requie a non-parametric one like Wilcoxon. Regardless, the function uses the means or meanranks from stat structure.
[~, ~, stats] = anova1(MPG, Origin, 'off')
stats =
struct with fields:
gnames: {6×1 cell}
n: [64 15 9 3 2 1]
source: 'anova1'
means: [21.1328 31.8000 28.4444 23.6667 22.5000 28]
df: 88
s: 7.0513
[~, ~, stats] = kruskalwallis(MPG, Origin, 'off')
stats =
struct with fields:
gnames: {6×1 cell}
n: [64 15 9 3 2 1]
source: 'kruskalwallis'
meanranks: [38.6719 75.2000 62.1111 49.3333 43.7500 67.5000]
sumt: 1266
You can check this further by
edit multcompare
So, you can realize that the function calculates critical values using tinv (t inverse CDF) for parametric tests (anova1 or anova2), while it uses norminv (normal inverse CDF) for non-parametric (KW or Friedman) ones.

Scott MacKenzie on 30 Jul 2021
The pairwise comparisons test used by default with multcompare is Tukey-Kramer. Using the ctype option, you can change the test to Bonferroni, Scheffe, Dunn-Sidak, or Fisher's LSD. Generally, the different tests yield the same or similar outcomes.
The discussion on t-tests in "More About" has nothing to do with how the tests are done in multcompare. This discussion, which I just read, is a nicely written explanation on why a customized test, such as Tukey-Kramer, is needed in the first place. The key point is this: When there are many group means, there are also many pairs to compare. If you applied an ordinary t-test in this situation, the alpha value would apply to each comparison, so the chance of incorrectly finding a significant difference would increase with the number of comparisons. All the pairwise comparisons tests accommodate the latter issue (incorrectly finding a significant difference) by applying a more conservative criterion than would exist if a simple t-test were used for each pairwise comparison.
Hamed Nassar on 7 Nov 2021
Tukey-Kramer, Bonferroni, Scheffe, Dunn-Sidak, etc. are not statistical tests. They are post-hoc correction schemes to adjust the p-value resulting from the original test, being Anov1, ttest, Freidman, etc.

Frederik Bergmann on 26 Oct 2022
So basically the question is still unanswered isn't it? What type of test is done to compare groups?
For example, if I do a Friedman test and submit the stats struct to mutlcompare, will I get the results of a Wilcoxon signed rank test - which should be done in that case?
unfortunately the output of multcompare does not provide any insight. There is just one column in the c matrix containing the pvalue of each comparison (row). But what if I want to know the t stats or r stat?
Ive J on 26 Oct 2022
Tukey-Kramer is by default performed for Friedman non-parametric test. The function is a general purpose comment for different post-hoc tests on different omnibus test, meaning user must decide which post-hoc is suited for the corresponding omnibus test.
Tukey-Kramer in MATLAB is same as Nemenyi test, so you can safely use it for Friedman test. You can also compare the stats from MATLAB multcompare and PMCMRplus::frdAllPairsNemenyiTest function from R package PMCMRplus.

### Categories

Find more on Analysis of Variance and Covariance in Help Center and File Exchange

R2020b

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by