Questions tagged [two-sample]
The two-sample problem is: given samples X and Y from two distributions, test whether the two underlying distributions are the same. One of the most common classical nonparametric approach is the Kolmogorov-Smirnov test.
108
questions
26
votes
2
answers
23k
views
2 Sample Kolmogorov-Smirnov vs. Anderson-Darling vs Cramer-von-Mises
I was wondering what are the criteria to use Kolmogorov-Smirnov, Cramer-von-Mises, and Anderson-Darling when comparing 2 ECDFS. I know the mathematics of how each differ, but if I have some ECDF data, ...
- 383
9
votes
3
answers
19k
views
8
votes
2
answers
36k
views
what is the difference between a two-sample t-test and a paired t-test
While I was glancing at hypothesis tests, I saw paired and two-sample t-test but couldn't understand the difference. For the explanation of these two tests, I saw the following sentence
" Two-...
- 505
7
votes
1
answer
3k
views
Earth Movers Distance and Maximum Mean Discrepency
By Kantorovich-Rubinstein duality the Earth Movers Distance (EMD)/Wasserstein Metric is equivalent to Maximum Mean Discrepancy (MMD) correct? See here for a more thorough explanation. Why then does ...
- 681
6
votes
2
answers
3k
views
What difference between Mann-whintey U-test and Kolmogorov-Smirnov test on truncated log normal distributions?
I have two populations who have been exposed to two different websites that should bring them to donations: one with a progress bar that pushes them to give (B, segment 2) and the other not (A, ...
6
votes
3
answers
2k
views
A Kernel Two Sample Test and Curse of Dimensionality
Gretton et al describes the Kernel Maximum Mean Discrepancy, a measure of distance between distributions. In order to compare two distributions, it turns out you can do much better than, say, taking ...
- 450
5
votes
1
answer
85
views
Why am I observing non-uniformly distributed (negatively skewed) p-values for two-sample tests of mixture distributions when the null is true?
I am interested in generating Gaussian mixture distributions as the null distributions for a series of two-sample test simulations. It is a well established fact that p-values follow a uniform ...
5
votes
1
answer
253
views
Why is the two-sample test giving me inconsistent results?
I am applying a two-sample t-test to determine whether we have software regressions on latency measurements.
Procedure
Run the test for build b1 and gather 60 latency measurements.
Run the test for ...
- 187
5
votes
1
answer
5k
views
Can Friedman's test be used with two samples?
When talking about Friedman's test, it commonly comes accompanied by a whole name of "The Friedman's test for three or more correlated samples".
The question is, could results be valid if I apply ...
- 287
5
votes
1
answer
2k
views
R function for weighted two-sample t-test **with Welch-adjusted t statistic**?
I'm conducting a hypothesis test for the difference between two groups. The file dat1 contains all observations of measure for ...
- 129
4
votes
2
answers
2k
views
Testing for equal proportions when sample sizes are very small
Suppose I observe binary data for two samples (hopefully the notation below is obvious) and I wish to test the hypotheses:
$$H_0: p_1 = p_2$$
$$H_A: p_1 \neq p_2$$
I know there is a $z$-test for doing ...
- 1,373
4
votes
1
answer
1k
views
Method to justify claim that two samples come from the same distribution
I know of ways to test "whether" two data sets come from the same distribution, in the sense that I can treat the hypothesis that they are from the same distribution as the null hypothesis. However, ...
- 1,108
4
votes
1
answer
1k
views
Robust two-sample test with triplicate measurements?
When testing for a difference in mean between two conditions, biologists typically use a $t$-test, and wring their hands endlessly about how to justify removing outliers. Whereas I typically use a ...
- 543
4
votes
0
answers
122
views
Do asymptotic statistics "solve" the Behrens-Fisher problem?
The Behrens-Fisher problem concerns comparing two means from independent (maybe multivariate) samples in a way robust to heteroskedasticity in the populations being compared. It seems that if one ...
- 1,373
3
votes
3
answers
2k
views
Two one-sided hypothesis tests instead of a two-sided test?
In hypothesis testing, the guidance is to use a one-sided test (alternative "greater" or "lesser") if we don't care about errors in one of the directions. If we do care about errors in both directions ...
- 2,600