I am comparing 4 groups of samples using SPSS. The groups are 4 bathymetry groups (>0m, >-1m, >-2m, >-3m). Within each group are a number of samples n. Each sample has a recorded seagrass cover % per meter squared. n ranges between 6 and 65 for all 4 samples. Samples are not normally distributed. A kruskal wallis test is used to determine significant differences in the distribution of seagrass cover % per meter squared. When I compare all 4 groups (pairwise comparisons), my results show that one group is significantly different from the rest. However; when I remove that group from the test and redo the kruskal wallis test with the remaining 3 groups (that didn't show significant difference from each other in the previous test), one of the groups is now significantly different. If i remove that group, leaving me with 2 groups (which were not significantly different in the two previous test), and I do a Mann-Whitney U test, it says that the groups are now significantly different. Please explain what is going on here.
2
-
3$\begingroup$ You are using the incorrect post hoc test. Mann-Whitney (a) does not use the same rankings as the K-S, so no surprise you are getting strange results, and (b) does not use the pooled variance assumed by the K-S null. Try Dunn's test or, even better, the Conover-Iman test. $\endgroup$– AlexisCommented Apr 15, 2023 at 20:08
-
1$\begingroup$ I'd add that the sequential procedure you describe -- find a group showing a difference then remove that group and look again -- without accounting for the impact of that sequential process on the properties of your inference is not likely just by happenstance to have properties you / your audience would find desirable. $\endgroup$– Glen_bCommented Apr 16, 2023 at 3:24
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
You are using the incorrect post hoc test. Mann-Whitney (a) does not use the same rankings as the K-S, so no surprise you are getting strange results, and (b) does not use the pooled variance assumed by the K-S null. Try Dunn's test or, even better, the Conover-Iman test.
