All Questions
17
questions
0
votes
1
answer
68
views
Bootstrap P-value and confidence intervals with more than two samples
I have been trying to find a simple way to use the bootstrap for a hypothesis test that involves more than two samples. The motivation for using the bootstrap is for the usual reasons: the test ...
6
votes
1
answer
163
views
Geyer: nonparametric bootstrap hypothesis tests typically have no power. Why?
I came across this page and was surprised and confused by the following:
A naive person attempting to do a bootstrap test just calculates a P-value as something like
mean(tstat.star > tstat.hat)
...
- 201
0
votes
0
answers
81
views
Nonparametric hypothesis test for ratio of medians
I have 4 distributions:
A1, A2, B1, and B2.
I would like to test whether there is a significant difference between median(A1)/median(A2) and median(B1)/median(B2). While I can calculate propagated ...
- 1
0
votes
1
answer
84
views
Hypothesis Testing with a Bootstrap
I have a list of 1000 samples from a distribution. I can find the lower limit of the 95% CI as it is the 2.5% percentile. In this case, it is zero. I would like to also test the hypothesis that a ...
- 179
0
votes
1
answer
724
views
Mann Whitney and boostrap confidence intervals provide contradictory results
We are doing some a/b testing and have non-normal metrics so our typical t-test aren't appropriate. I typically use bootstrap CI and/or Mann whitney tests to determine the true effect. In this case ...
- 11
1
vote
0
answers
895
views
Understanding Sharpe Ratio Hypothesis Testing - Ledoit + Wolf
I've been poring over this paper written by Ledoit and Wolf regarding their approach to constructing hypothesis tests for Sharpe Ratios.
In short, they see that running circular block bootstrap ...
- 41
2
votes
1
answer
729
views
Appropriateness of nonparametric bootstrap methods to assess difference between two groups [closed]
This question is motivated by the discussion of this earlier question.
I have two samples $X$ and $Y$, where both samples have $n$ elements. Both samples represent optimal solutions returned from two ...
3
votes
0
answers
210
views
Can non-parametric tests, e.g. Mann-Whitney U, be used on non-normally distributed statistics off of bootstrap samples?
I have some return data from some different portfolios which I would like to compare using risk vs return ratios. The standard Sharpe ratio has a nice solution for calculating the significance of the ...
2
votes
0
answers
54
views
Suggest a (nonparametric?) test for the difference between two strictly-positive distributions
$\mathbf{X}$ is a $n\times p$ matrix of data. I'm treating $\mathbf{X}$ as having a multivariate normal distribution with some arbitrary covariance structure.
$\mathbf{Y}$ is a $(m<n)\times p$ ...
- 13.5k
1
vote
2
answers
1k
views
Non-parametric tests in big data scenario
Suppose I have two populations A and B , with sizes $n_1$ and $n_2$ respectively, where both $n_1$ and $n_2$ are large (say, above 500).
I want to test that the values $x_1, \dots, x_{n_1}$ of A ...
- 344
0
votes
0
answers
109
views
Is it possible to validate very small p-values via bootstrap?
Suppose I want to see whether the p-values generated by a certain parametric method are close to the “true” p-values. To avoid relying on any parametric assumptions, the latter are produced via ...
- 2,742
1
vote
1
answer
194
views
Bootstrap hypothesis testing with small sample sizes
I have 2 campaigns (a control and a test campaign), the data are like this:
...
6
votes
2
answers
3k
views
How to compare two non-normally distributed samples with very different sizes? (Mann-Whitney vs Randomization/Bootstrap)
Perhaps this is a very basic question, but I didn't find yet a simple solution for this simple problem:
I want to compare two samples (say X and Y) for a continuous variable which is non-normally ...
- 75
3
votes
1
answer
3k
views
Hypothesis testing - Wilcoxon test, bootstrapping, or something else?
A colleague has developed a treatment for to "prevent falls" in cognitively impaired, psychiatric patients. Since this would be very useful treatment in this population, we especially do not want to ...
- 165
3
votes
0
answers
1k
views
Kolmogorov-Smirnov and bootstrap
The KS test uses the statistic
$$
D_n=\sup_x |\hat{F}_n(x)-F_0(x)|
$$
where $F_0(x)$ is the distribution to be tested and $\hat{F}_n(x)$ the empirical distribution. Under the null hypothesis $D_n$ is ...
- 61