8
$\begingroup$

I've read in the library manual for test ks.test that

The possible values "two.sided", "less" and "greater" of alternative specify the null hypothesis that the true distribution function of x is equal to, not less than or not greater than the hypothesized distribution function (one-sample case) or the distribution function of y (two-sample case), respectively. This is a comparison of cumulative distribution functions, and the test statistic is the maximum difference in value, with the statistic in the "greater" alternative being D^+ = max[F_x(u) - F_y(u)]. Thus in the two-sample case alternative = "greater" includes distributions for which x is stochastically smaller than y (the CDF of x lies above and hence to the left of that for y), in contrast to t.test or wilcox.test.

Unfortunately I failed to understand this difference between (I guess one sided) wilcox.test and ks.test. It seems, that both test for displacement of one distribution versus another. Does anyone can shed some light on it, please?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
8
$\begingroup$

Both are testing for displacement of the x variable with respect to the y variable, but the 2 tests have opposite meanings for the term "greater" (and therefor also or "less").

In the ks.test "greater" means that the CDF of 'x' is higher than the CDF of 'y' which means that things like the mean and the median will be smaller values in 'x' than in 'y' if the CDF of 'x' is "greater" than the CDF of 'y'. In 'wicox.test' and 't.test' the mean, median, etc. will be greater in 'x' than in 'y' if you believe that the alternative of "greater" is true.

An example from R:

> x <- rnorm(25)
> y <- rnorm(25, 1)
> 
> ks.test(x,y, alt='greater')

        Two-sample Kolmogorov-Smirnov test

data:  x and y 
D = 0.6, p-value = 0.0001625
alternative hypothesis: two-sided 

> wilcox.test( x, y, alt='greater' )

        Wilcoxon rank sum test

data:  x and y 
W = 127, p-value = 0.9999
alternative hypothesis: true location shift is greater than 0 

> wilcox.test( x, y, alt='less' )

        Wilcoxon rank sum test

data:  x and y 
W = 127, p-value = 0.000101
alternative hypothesis: true location shift is less than 0

Here I generated 2 samples from a normal distribution, both with sample size 25 and standard deviation of 1. The x variable comes from a distribution of mean 0 and the y variable from a distribution of mean 1. You can see the results of ks.test give a very significant result testing in the "greater" direction even though x has the smaller mean, this is because the CDF of x is above that of y. The wilcox.test function shows lack of significance in the "greater" direction, but similar level of significance in the "less" direction.

Both tests are different approaches to testing the same idea, but what "greater" and "less" mean to the 2 tests are different (and conceptually opposite).

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you! Do you happen to know, which test has better power? I guess it is Wilcoxon-Mann-Whitney, but I don't know how to support it. $\endgroup$
    – Adam Ryczkowski
    Commented Oct 15, 2012 at 18:24
  • $\begingroup$ Which has greater power depends on the shapes of the distributions and how they differ, otherwise why would both still be commonly used? $\endgroup$
    – Greg Snow
    Commented Oct 15, 2012 at 18:29
  • $\begingroup$ That's interesting. Can you please tell me, how does the relative power of both tests depend on both distributions (or at least point me to a literature)? $\endgroup$
    – Adam Ryczkowski
    Commented Oct 15, 2012 at 18:35
  • 1
    $\begingroup$ From my experience, the fact that something is commonly used in statistics does not necessary imply it is the correct procedure... $\endgroup$
    – Adam Ryczkowski
    Commented Oct 15, 2012 at 18:37
  • $\begingroup$ Just look at how the test statistic works. The ks test looks at the distance between the ECDF's and the wilcox test looks at the probability of a getting a larger value from y than x when choosing one value at random from each. Find distributions that show a large difference one way, but no the other and you will see large differences in power. If one procedure were clearly dominated by the other, and it was easy to show/explaine that, then I would expect it to fall out of disfavor. This does not say that either is "correct". $\endgroup$
    – Greg Snow
    Commented Oct 15, 2012 at 19:22
7
$\begingroup$

I disagree with R and @GregSnow on this point. Here is why.

set.seed(123)
x <- rnorm(25)
y <- rnorm(25, sd=5)
ks.test(x, y, alternative='greater')
#    Two-sample Kolmogorov-Smirnov test
#
# data:  x and y
# D^+ = 0.44, p-value = 0.007907
# alternative hypothesis: the CDF of x lies above that of y
#
ks.test(x, y, alternative='less')
#    Two-sample Kolmogorov-Smirnov test
#
# data:  x and y
# D^- = 0.36, p-value = 0.03916
# alternative hypothesis: the CDF of x lies below that of y

In the case that the variances are different, the CDF of x lies both above and below that of y. So rejecting the one-sided hypothesis does not mean anything about the mean or the median. It does not even say anything about stochastic dominance.

You can find more about this misconception there and some discussion there. In my opinion, the one sided KS test has some theoretical attractiveness, but it is wrong to draw a conclusion about the first moment from it. For this purpose, the Wilcoxon test is more appropriate.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ I don't think that we disagree as much as you expanded your answer beyond the scope of what I was discussing (and how I interpreted the documentation). I should have clarified the limitations that I worked with being only looking at cases where we are looking at a location shift without a change in spread/shape, a common but not needed assumption. You bring up a very important point in understanding the appropriateness of the test when the cdf's cross (or could cross). $\endgroup$
    – Greg Snow
    Commented Sep 23, 2014 at 15:50
  • 1
    $\begingroup$ @GregSnow Thanks for your constructive comment. I also agree that the disagreement is not radical :-) I suggest to leave the post as is to highlight the issue. However, if you have any suggestion to change the tone, or make my answer more constructive I would be happy to do it. $\endgroup$
    – gui11aume
    Commented Sep 23, 2014 at 16:25

Not the answer you're looking for? Browse other questions tagged or ask your own question.