3
$\begingroup$

I have this dataset:

Treatment data
T1 0
T1 0
T1 0
T10 0
T10 0
T10 0
T11 0.2
T11 0.2
T11 0.2
T12 0
T12 0
T12 0
T2 0.7
T2 0.6
T2 0.6
T3 0.8
T3 0.7
T3 0.8
T4 0.3
T4 0.3
T4 0.4
T5 0
T5 0
T5 0
T6 0.7
T6 0.7
T6 0.5
T7 0
T7 0
T7 0
T8 0.8
T8 0.7
T8 0.8
T9 0
T9 0
T9 0

After Scheirer–Ray–Hare test with significant results, I did Dunn test with Bonferroni correction for multiple comparisons. Code: Test = dunnTest(Data ~ Treatment, data=dataset, method="bh"). Here are some results:

Comparison Z P.unadj P.adj
... ... ... ...
T1 - T3 -2.92771374 0.003414643 0.01878053
T10 - T3 -2.92771374 0.003414643 0.02048786
T11 - T3 -1.61958633 0.105321169 0.21722491
T12 - T3 -2.92771374 0.003414643 0.02253664
T2 - T3 -0.66444567 0.506405109 0.74272749
... ... ... ...
T1 - T9 0.00000000 1.000000000 1.000000000
T10 - T9 0.00000000 1.000000000 1.000000000
T11 - T9 1.30812742 0.190830096 0.38166019
T12 - T9 0.00000000 1.000000000 1.000000000
T2 - T9 2.26326807 0.023619169 0.08204554
T3 - T9 2.92771374 0.003414643 0.11268321

As you can see, P after adjustment gives very weird results. For example, T1-T3 is 0.01878053 (significant), but T3-T9 is 0.11268321 (nonsignificant) even though T1 and T9 had exactly the same data (all zero), and P before adjustment are exactly the same as well.

Am I doing it wrong somewehere?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

According to the package docs I presume you are using (dunn.test) the method 'bh' is the 'Benjamini-Hochberg adjustment' not the bonferroni correction. This might explain why you get different adjusted values for tests with the same p values, as this method orders the tests and then adjusts values based on the ordering. I'm not sure exactly how this method deals with duplicate p values, but it may be the case that T1-T3 is arbitrarily assigned earlier in the ordering than T3-T9 and so they get different adjusted p values.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Oh, I see. I used the code from somewhere else so I assumed it was bonferroni. Thank you. $\endgroup$
    – MD P
    Commented Jul 17, 2023 at 10:34
  • $\begingroup$ You may want to consider using the Holm method instead as this has more power and still controls the familywise error rate. $\endgroup$
    – A. Bollans
    Commented Jul 17, 2023 at 10:40
  • 2
    $\begingroup$ Also if this has answered your question please consider accepting the answer :) $\endgroup$
    – A. Bollans
    Commented Jul 17, 2023 at 10:41

Not the answer you're looking for? Browse other questions tagged or ask your own question.