4
$\begingroup$

I have 3 sampling sites where I took 3 water samples in each site. I chose to do the Kruskal-Wallis test to find out if there is a significant difference in total nitrogen between the sites. I did not choose ANOVA, because normality and homoscedasticity were not always respected and I have a small sample size.

When, there is a significant difference between the sites, Xlstat offers me the option of using the Dunn, the Conover-Iman and the Steel-Dwass-Critchlow-Fligner test as a pairwise comparison test. Also, I can do the Bonferroni correction with the Dunn and Conover-Iman test. I would like to know which test is the most appropriate for my analysis and why?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

Use the generalization of the Wilcoxon-Mann-Whitney-Kruskal-Wallis tests: the proportional odds ordinal logistic model. Once you fit the model with two indicator independent variables representing 3 groups you can test any pairwise contrasts you want. Contrasts will be on the log odds scale. This is all easily done with the R rms package orm and contrast.rms functions. More details may be found in the Nonparametrics chapter in BBR.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you for the answer and I have others questions : 1. Why you don't suggest to me the Kruskal-Wallis test with one of the comparison tests mentioned in my previous question? 2. Can you send me some references about the proportional odds ordinal logistic model? I have a hard time to found some comprehensible information about this method? 3. By the name of your proposed method, I assume that we need ordinal data to use this method. This is not the case for me. So, is it possible to explain to me how to use this method with my set of data? $\endgroup$
    – Simon Leclair
    Commented Oct 27, 2021 at 11:54
  • $\begingroup$ Also, I don’t have access to your BBR link. Is it possible that you did a mistake when you write it? $\endgroup$
    – Simon Leclair
    Commented Oct 27, 2021 at 11:54
  • $\begingroup$ Sorry the web server is back up now. $\endgroup$
    – Frank Harrell
    Commented Oct 27, 2021 at 14:34
  • $\begingroup$ Thank you and also, can you explain to me why you don't suggest the Kruskal-Wallis test for my data set? $\endgroup$
    – Simon Leclair
    Commented Oct 27, 2021 at 16:18
  • 1
    $\begingroup$ Because I don't suggest K-W for any dataset since using its generaization gives you so much more, e.g., any contrast of interest. $\endgroup$
    – Frank Harrell
    Commented Oct 28, 2021 at 13:11
4
$\begingroup$

It's possible to get a significant result with the Kruskal-Wallis test when you have three observations each from three sites. For example, if site one had concentrations of (1,2,3), and the other sites had observations, respectively of, (4,5,6) and (7,8,9).

Practically though, environmental water quality data is usually quite variable, based on a whole host of factors. Usually you need several to many observations – (say, something like, one sample per site per week for say 20 or 30 weeks, and then maybe repeated for two years) – to be able to make any reasonable conclusions.

Personally, I like Dunn (1964) test as a post-hoc to K-W. I don't have a great justification for this opinion. It's probably the most traditional one. It keeps the ranking of the observations from the original K-W test, and then performs an analysis similar to the original analysis. In my experience it's relatively conservative, even without adjusting the p-values.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you for the information, but can we use the Dunn test with a small set of data like mine? $\endgroup$
    – Simon Leclair
    Commented Oct 27, 2021 at 11:56
  • $\begingroup$ Yes. You can use any of the post-hoc tests you mention, even with a small data set.... Well, having typed that, I don't know if it's a good idea for all of them or not with a really small data set. ... I think if you're okay using K-W with a small data set, you'd be okay using Dunn on that data set $\endgroup$
    – Sal Mangiafico
    Commented Oct 27, 2021 at 14:32
  • $\begingroup$ Thank you for the information $\endgroup$
    – Simon Leclair
    Commented Oct 27, 2021 at 16:17
  • $\begingroup$ Do you know a test that is better than Kruskal-Wallis for a small data set? $\endgroup$
    – Simon Leclair
    Commented Oct 28, 2021 at 9:53
  • $\begingroup$ There is no way that p-values are useful in ultra small sample situations. Instead compute confidence intervals, which fully capture the limits of the information in your dataset. You can compute confidence intervals for concordance probabilities (pairwise comparisons) or for odds ratios from proportional odds models (pairwise or any contrast you want). $\endgroup$
    – Frank Harrell
    Commented Oct 28, 2021 at 13:12

Not the answer you're looking for? Browse other questions tagged or ask your own question.