Questions tagged [two-sample]
The two-sample problem is: given samples X and Y from two distributions, test whether the two underlying distributions are the same. One of the most common classical nonparametric approach is the Kolmogorov-Smirnov test.
108
questions
0
votes
0
answers
33
views
Is bootstrapping inherently Frequentist? If so, how do we do a Bayesian non-parametric two-sample test?
I normally use frequentist statistics but I now want to use Bayesian statistics as I want to carry out a two-sample (randomised control trial) test that includes prior information. I have an existing ...
- 913
1
vote
0
answers
43
views
Deriving Sample Size for two samples t test
I am trying to figure out what sample size recommendations to make in this case when the samples have different sizes. Since their sizes are different the variance of each will be different.
Suppose ...
- 345
1
vote
1
answer
23
views
Two-sample test of difference in probability mass at the extremes of the empirical distributions
I am running an experiment that will generate a dependent variable (DV) in two treatments, T1 and T2.
One of the hypotheses I want to test is whether the distribution of the DV in T1 has more mass at ...
1
vote
1
answer
63
views
How to perform a 2 sample Kolmogorov-Smirnov test controlling for a type II error instead of type 1
I have four completely different sets of two samples and want to check for each set if both samples come from the same distribution, so four Kolmogorov-Smirnov tests total. If it's going to be ...
- 21
0
votes
0
answers
46
views
How to Run a Standard Two-Sample t-test when given delta and not the two means
I am trying to find the p-value to test a null hypothesis using a Standard Two-Sample t-Test. Usually I would use
...
0
votes
0
answers
20
views
Validity of Bootstrap Inference for Bounds
I am facing the following problem: I have access to iid $X_1,\dots ,X_{N_X}$ and $Y_1, \dots, Y_{N_Y}$ from $F_X$ and $F_Y$, respectively, where $X$ stochastically dominates $Y$.
My goal is to conduct ...
- 1
0
votes
0
answers
11
views
Two-sample statistical significance test taking into account group membership (proportions) and numeric comparison
Say I have two samples of observations, S1 and S2, which can be of unequal sizes. For each observation, I have two variables: G (a categorical group assignment, say it is binary with values A/B), and ...
- 171
1
vote
1
answer
78
views
Two sample proportions: How to test the hypothesis that the difference in proportions is less than a specified threshold?
Say I have treatment and control, perfectly randomized.
I have 70000 conversions for treatment in 80000 users.
I have 70100 converions for control in 800100 users.
My null hypothesis is that there is ...
0
votes
1
answer
25
views
Is two-sample Kolmogorov test working correctly?
I have two one-dimensional samples that I'm trying to quantifiably distinguish (or deny such distinction). I.e. the null-hypothesis is that they come from the same population (distribution?). The ...
- 45
1
vote
0
answers
86
views
Comparing means in Box-Cox transformed data
Deeply sorry as this wasn't really covered in my statistics classes. I am current comparing datasets for people divorcing vs. dissolving their marriages. I have two variables, length of marriage, and ...
- 11
1
vote
1
answer
94
views
Kolmogorov Smirnov Test Consistency
I was reading that the Kolmogorov Smirnov 2 sample test is consistent, that is Probability of rejection under $H_1$ is 1 for sample size going to infinity.
Say we have 2 random variables X and Y. K-S ...
- 37
1
vote
1
answer
268
views
When transforming a dataset using the square root (or any other procedure), does the standard deviation of the dataset get transformed too?
Taking an introductory statistics course, and I am confused about the impact of a transformation on the df for a two-sample t-test. The df is dependent on n of each sample and SD of each sample, but I'...
- 11
0
votes
1
answer
72
views
Compare two samples when each observation is a pair of dependent values
At an airport there is a company that offers a specific service for travellers (there are no competitors offering the same service at the location). They performed an experiment - they temporarily ...
- 157
1
vote
0
answers
16
views
Permutation Testing Using only Source Samples
Short Background on Permutation Testing
Suppose I have two sets of samples $P$ and $Q$ drawn iid from distributions $\mathcal{P}$ and $\mathcal{Q}$ over $X$.
I also have access to a test function $T: ...
- 160
2
votes
1
answer
177
views
How to compare multiple Two-Sample Hotelling's T-squared tests performed on different sample sizes
It's my first question, so please be patient with me.
I'm working on the comparison of multiple datasets of different lengths (60 to ~10000). They share a set of 9 features which I would like to use ...
- 21
0
votes
1
answer
113
views
two-sample proportions hypothesis test: 10 successes and failures rule
I asked this question on mathematics stack exchange, and reposting here as it did not get resolved.
Suppose the sample sizes and number of successes are ($n_1, y_1$) and ($n_2, y_2$), for the two ...
- 3
2
votes
0
answers
62
views
How does the following term happen when using the Wald Test on two-sample binomials?
I'm having trouble understanding one of the steps of the Wald test used in two-sample binomial proportions.
More specifically, when $X_1, ..., X_{n_1}$ are iid Binomial$(1, p_1$), $Y_1, ..., Y_{n_2}$ ...
- 21
3
votes
3
answers
1k
views
Two-sample Kolmogorov-Smirnov discrete distributions
Let's say that two Census takers interview a number of couples and ask for the number of children they have. I want to compare the two resulting samples to find out if they could have come from the ...
- 55
4
votes
0
answers
122
views
Do asymptotic statistics "solve" the Behrens-Fisher problem?
The Behrens-Fisher problem concerns comparing two means from independent (maybe multivariate) samples in a way robust to heteroskedasticity in the populations being compared. It seems that if one ...
- 1,373
1
vote
1
answer
258
views
Two Samples Test for Discrete Variable and Different Size Groups?
Hello there and apologies for my English and lack of statistic skills!
Two different groups of participants, one comprised of children and the other of adults, run once, at different times, the same ...
- 111
2
votes
0
answers
92
views
Test for distribution equality
This question touches Kolmogorov-Smirnov testing, but asks actually something different.
Consider independent random variables $X_1, \dots, X_n$. I want to test the following hypothesis:
$$
H_0: X_i\ ...
- 419
0
votes
0
answers
28
views
Test two samples with correlated observations within sample
I have repeated observations from individuals that have been randomly assigned to either Group 1 or Group 2. In detail, 64 subjects were randomly assigned to one of the two groups and performed a task ...
1
vote
1
answer
165
views
Paired t-test vs two-sample t-test - animal populations
From my understanding, a paired t-test is used when samples are dependent of each other.
I'm having trouble deciding whether a paired t-test should be used when comparing the average population of a ...
user355881
0
votes
0
answers
25
views
Two Sample test for two samples with different distribution
I have independent random samples of $R^d, d>1$ $X=\{X_1,..X_n\}$ and $Y=\{Y_1,.. Y_n\}$ with respective distributions $F_1$ and $F_2$. In a regular two sample problem we test $H_0:F_1=F_2$ vs $H_1:...
- 3
1
vote
0
answers
47
views
Testing difference of medians and stochastic ordering using paired samples
I have paired samples from skewed distributions with unequal variance (before and after treatment), and would like to test for
difference of means
difference of medians
stochastic ordering of treated ...
- 41
2
votes
0
answers
187
views
With 2-sample bootstrap testing, why pool treatment with control samples?
I have data from a randomized experiment. Members of the treatment group received a treatment, members of the control group did not.
I want to test differences in distributions of ordinal outcomes ...
- 21
2
votes
2
answers
774
views
Two-sample test with bootstrap and different sizes
I have two independent i.i.d., respectively $Y_1,\dots,Y_{n_Y} \sim F_Y$ of size $n_Y$ and $X_1,\dots,X_{n_X} \sim G_X$ of size $n_X$. I would like to test:
$$
H_0:\mu_X = \mu_Y
$$
versus any ...
- 333
1
vote
1
answer
633
views
How to perform F-test in R for checking equality-of-variance (homoscedasticity/heteroscedasticity) if raw data is not given?
In a typical two-sample problem scenario, I possess the sample mean, variance, and size, but not the raw data for samples drawn from two populations. I wish to perform the F-test to check equality of ...
- 193
1
vote
2
answers
82
views
What is the theoretical justification of testing for equal mean/distribution via resampling?
If we have two data sets $X_1,\ldots,X_m$ and $Y_1,\ldots,Y_n$, each i.i.d., and wanted to determine whether $\mathbb{E}[X_1] = \mathbb{E}[Y_1]$ or not using $\bar{X}_m - \bar{Y}_n = \hat{\Delta}_{m,n}...
- 1,373
1
vote
2
answers
2k
views
Bayesian : Comparing means of two posterior samples/ Help a Frequentist Out
UPDATE Thanks for the many thoughtful responses and questions! I've made edits here to clarify further. and also respond to each respondent individually.
Original Post
I have two sets of posterior ...
- 208
1
vote
0
answers
65
views
Two sample similarity analysis for non-independent daily time series data
What two sample or other relevant statistical tests can be performed in R between two samples which are independent of each other, but the observations within each ...
- 111
2
votes
1
answer
105
views
Doing a t-test on a normally distributed difference of two non normal distributions
I have two populations of donations (segment 2 people were subject to a different form):
...
6
votes
2
answers
3k
views
What difference between Mann-whintey U-test and Kolmogorov-Smirnov test on truncated log normal distributions?
I have two populations who have been exposed to two different websites that should bring them to donations: one with a progress bar that pushes them to give (B, segment 2) and the other not (A, ...
3
votes
3
answers
182
views
T-test states difference of donation is significant when Z-test claims not, what method to use?
I have two populations who have been exposed to two different websites that should bring them to donations: one with a progress bar that pushes them to give (B, segment 2) and the other not (A, ...
5
votes
1
answer
253
views
Why is the two-sample test giving me inconsistent results?
I am applying a two-sample t-test to determine whether we have software regressions on latency measurements.
Procedure
Run the test for build b1 and gather 60 latency measurements.
Run the test for ...
- 187
5
votes
1
answer
85
views
Why am I observing non-uniformly distributed (negatively skewed) p-values for two-sample tests of mixture distributions when the null is true?
I am interested in generating Gaussian mixture distributions as the null distributions for a series of two-sample test simulations. It is a well established fact that p-values follow a uniform ...
0
votes
1
answer
230
views
t-statistic varies widely when used for bootstrap sampling process
I have two samples of size 20 and I want to see if the difference between the geometric means is significant using two-sample t-statistic and bootstrap sampling.
Here is what I'm doing
...
- 145
4
votes
2
answers
2k
views
Testing for equal proportions when sample sizes are very small
Suppose I observe binary data for two samples (hopefully the notation below is obvious) and I wish to test the hypotheses:
$$H_0: p_1 = p_2$$
$$H_A: p_1 \neq p_2$$
I know there is a $z$-test for doing ...
- 1,373
8
votes
2
answers
36k
views
what is the difference between a two-sample t-test and a paired t-test
While I was glancing at hypothesis tests, I saw paired and two-sample t-test but couldn't understand the difference. For the explanation of these two tests, I saw the following sentence
" Two-...
- 505
3
votes
1
answer
312
views
Can I simulate a two-sample K-S test when I can't perform a one-sample test because a distribution parameter is unknown?
Let's say I have a signal which I want to test for normality. I know that the mean from the theoretical distribution is zero, but it has an undetermined (unknown) variance. If I knew mean and variance ...
- 33
5
votes
1
answer
2k
views
R function for weighted two-sample t-test **with Welch-adjusted t statistic**?
I'm conducting a hypothesis test for the difference between two groups. The file dat1 contains all observations of measure for ...
- 129
1
vote
0
answers
49
views
Determine distribution of a new test statistic
Question from my course project:
Suppose we are conducting a two-arm trial. For the control group, there are $n_c$ patients, and their outcome is denoted by X, which has distribution $N(\mu_{c},\sigma^...
- 21
0
votes
1
answer
552
views
Two-sample test + paired/independent samples
This might be an easy question but I got lost in my thinking so thank you for clarifying that for me. So I have some troubles with understanding the ideas of two-sample tests in addition to paired and ...
2
votes
3
answers
170
views
The mean of natural log of some data
I have the following data in natural log:
N = [0.929, -1.745, 1.677, 0.701, 0.128]
O = [2.233, -2.513, 1.204, 1.938, 2.533]
I want to calculate the mean ratio of ...
- 21
1
vote
1
answer
211
views
Confused about Two Sample Proportion Tests - am I performing this correctly?
I am dealing with an advertising dataset that includes hundreds of advertisements, how many clicks each ad receives, whether the amount of clicks received is considered "high" or "low&...
- 11
1
vote
0
answers
180
views
Two-sample test when samples are not completely independent?
Let's say I measure the height of 100 individuals every week in both Illinois and Texas while allowing for the same individual to be measured multiple times in the following weeks. This way, my ...
- 11
1
vote
2
answers
823
views
Performing a two sample t-test for equal means on subset of a dataset
If I have a dataset that, of among other variables, contains the gender and wealth for a sample of a population with a year variable that represents the year the data about the individual was ...
- 11
2
votes
0
answers
114
views
test whether two samples are from the same high dimensional multinomial distribution
I have two high dimensional vectors (~150 dims), assumed to be sampled from multinomial distributions. How can I test whether the two are from the same multinomial distribution or different ones? (...
- 41
-1
votes
1
answer
326
views
Kolmogorov-Smirnov two-sample test p-value evaluation
I'm using python's scpipy implementation of the Kolmogorov-Smirnov two-sample test (ks_2samp).
Q1: Why are my p-values in the examples below so low, specifically on the first example?
Q2: Can I ...
- 33
0
votes
1
answer
215
views
two-sample Kolmogorov-Smirnov test suggest that sample with the same data generating process from different populations
I have two different samples I need to test if they are drawn from the same population distribution. The two samples are of the volatility of an asset that I simulated. Since the data is simulated I ...
