2
$\begingroup$

I'm doing an analysis of a master's student in which I need to compare a continuous numerical variable between two groups. The samples are independent and each group contains only three observations each. The issue is that the treatment was very efficient compared to the control, the database came a little strange.

The treatment was so efficient that the bacteria counts zeroed in the three cases that received the drug (it's an antibiotic that is being tested, which totally killed the bacteria). In one of the cases I have, for example, I have: control = c(17.38 , 17.33 , 17.16) and treatment = c(0 , 0 , 0). In the Mann-Whiney test, but these tests that use ranks do not show the presence of ties, behaving very strangely, displaying warnings of their possible not working and very well with the same p-value all the comparisons I made. Next, I thought about using the median test, which creates a 2x2 table and applies Fisher's Exact Test, but I don't know if it would be suitable.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ You may have been told that a. Non-normal data require non-parametric tests and that b. The Wilcoxon is parametric whereas the t-test isn't. In both cases you would have been told wrong. Anyway, the point of an antibiotic is to completely kill the bacteria - otherwise a second-gen antibiotic-refractory bacteria reproliferates. I would analyze the binary endpoint using Fishers or exact binomial, and then as a sensitivity analysis, you can consider a quasi-Poisson model for cells/uL - a good technique since the data are positive, skewed, and have a mean-variance relationship. $\endgroup$
    – AdamO
    Commented Oct 31, 2022 at 20:26
  • $\begingroup$ You mean binarize the data and do a kind of median test? like, gather all the data, take the median and see how many points above the median you have in the control and in the treatment? and then do an exact fisher test? because i dont want to make a model, but just test the mean difference, you know? thank you in advance $\endgroup$
    – Tales Pereira
    Commented Oct 31, 2022 at 22:35
  • $\begingroup$ Whoever said do a median test? The Fisher and Exact Binomial tests are tests of categorical data. The Wilcoxon is only a test of medians when you have symmetric data, in which case it's also a (an underpowered) test of means. But that's a strong assumption to make - and also not likely. $\endgroup$
    – AdamO
    Commented Oct 31, 2022 at 22:52
  • 1
    $\begingroup$ And if you want a robust test of MEAN difference then you ought to use a t-test! - Of course, you can't apply an unequal variance assumption when there's NO variance in the active arm. $\endgroup$
    – AdamO
    Commented Oct 31, 2022 at 22:52
  • $\begingroup$ Why not just us the 'BO test'? It's the inter-occular impact test because the difference hits you right between the eyes. (Bloody Obvious here in Australia.) Your understanding of the results is unlikely to be improved with a statistical test, so just present the data. $\endgroup$
    – Michael Lew
    Commented Nov 6, 2022 at 20:47

1 Answer 1

Reset to default
3
$\begingroup$

With three observations in the control with values all about 17, and three observations in the treatment all with values of 0, I don't think a hypothesis test will tell you much.

Unless you need to conduct one for performative reasons.

In that case, one option is the Fisher-Pitman permutation test.

control = c(17.38 , 17.33 , 17.16)
treatment = c(0 , 0 , 0)

Value = c(control, treatment)
Group = factor(c(rep("Control", length(control)), rep("Treatment", length(treatment))))

library(coin)

oneway_test(Value ~ Group, distribution="exact")

   ### Exact Two-Sample Fisher-Pitman Permutation Test
   ###
   ### Z = 2.236, p-value = 0.1

As you mention, a test of the medians may also make sense.

library(coin)

median_test(Value ~ Group, distribution="exact")

   ### Exact Two-Sample Brown-Mood Median Test
   ### Z = 2.2361, p-value = 0.1

This should be similar to the chi-square test underlying Mood's median test.

chisq.test(matrix(c(0,3,3,0),nrow=2), simulate.p.value=TRUE,B=10000)

   ### Pearson's Chi-squared test with simulated p-value (based on 10000 replicates)
   ### 
   ### X-squared = 6, df = NA, p-value = 0.1056

A Wilcoxon-Mann-Whitney test would make sense, but I would use an implementation with an exact option.

library(coin)

wilcox_test(Value ~ Group, distribution="exact")

   ### Exact Wilcoxon-Mann-Whitney Test
   ###
   ### Z = 2.0868, p-value = 0.1

Finally, with the caveat that I wrote the function, you could investigate the difference in means by a simple permutation test of the values. This function simply permutes the values and determines how often this result is at least as extreme as the original difference in the means.

library(rcompanion)

percentileTest(x=control, y=treatment, test="mean", r=10000)

###           p.value
###   p-value  0.1041
Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.