All Questions
Tagged with multiple-comparisons permutation-test
36
questions
1
vote
0
answers
34
views
Is a permutation test with max statistics appropriate to account for multiple comparisons when searching for correlations between two questionnaires?
I'm trying to understand if the max statistic method for permutation test is appropriate to use and how i should apply it in my situation:
I have a sample of 38 individuals who performed a task and we ...
0
votes
1
answer
126
views
How to implement multiple comparison correction after Wilcoxon rank-sum test with relativelty small sample sizes and many comparisons
I am comparing 2 groups, performing a total of 180 comparisons. When doing these 180 pair-wise comparisons using the Wilcoxon rank-sum test more than 40 show a significant (p<0.05) difference ...
- 23
4
votes
1
answer
248
views
Adjusting p-values in multiple regression using permutation test
I have run a multiple regression model using a set of 10 categorical and numerical explanatory variables. This model results in 10 coefficients and their associated p-values. I would like to report ...
- 43
0
votes
0
answers
26
views
Permutation test replications aggregation
Short version:
Is it right, and if yes, what is the right way to combine permutation tests applied on n randomly selected subsets of a bigger dataset? I was not able to find a publication that tackles ...
2
votes
1
answer
152
views
Is the FDR linear-step up procedure Benjamini and Hochberg valid for empircal p-values from non-parametric tests (e.g., bootstrap)?
I have empirical p-values from bootstrapped hypothesis tests with 10,000 resamples each. Because there are a finite number of resamples, some of the p-values are "0", i.e., < 0.0001 with ...
- 61
5
votes
2
answers
2k
views
Adjusting P values for multiple comparisons using permutation tests
I have a number of continuous predictors (biomarker measurements) which I would like to test for association with a binary outcome variable (disease status), adjusting for multiple comparisons. As ...
- 51
1
vote
0
answers
16
views
How to statistically analyse gene-environment interactions pairs?
I am new here so sorry I hope I am using the forum correctly. Also, my knowledge on statistics is very basic and is something I am working towards learning more.
I am studying gene-environment ...
- 11
3
votes
1
answer
2k
views
Testing for equality of variance for more than 2 groups
How do you test for equality of variance between, say, 3 groups? Is there an extension of the two-sample F-test? Or is it OK to just run multiple F-tests or permutation tests among 3 groups?
- 337
1
vote
1
answer
650
views
how should I adjust P values for multiple comparison from permutation test?
I have around 1000 permutation test and get a list of p values. Not sure if it is ok to use P.adjust(p_values) in R to directly adjust my permutation p values? I know in other settings(like t-test) I ...
- 445
2
votes
0
answers
584
views
How permutation based FDR control methods take into account the correlation structure?
There are a number of permutations-based False Discovery Rate (FDR) control methods, such as SAM [1], dChip [2], Ge et al [3], Meinshausen [4], and Efron [5]. It's been said that in these methods, ...
- 131
1
vote
0
answers
142
views
Correction for multiple comparisons after permutation
I've trained 17 regression models using a support vector regression machine providing correlations (r) between true and predicted labels and the mean squared error (MSE) for each model.
The target ...
- 71
0
votes
0
answers
1k
views
How to find the power of a permutation test?
I am new to the permutation test and has found some online resources to learn how to conduct the permutation test. But I have yet to find a good explanation of how to find the power of a permutation ...
- 1,544
1
vote
0
answers
459
views
Permutation test, probability and FDR
I encountered a problem to perform FDR control of my data, and am not sure how can I do.
I have hundreds to thousands of observations, each observation has two scores. So I have $\mathbf{s_1}=\{s_{11}...
- 185
3
votes
0
answers
2k
views
How to interpret permutation test p-values in the context of multiple testing
Suppose I want to test for a significant effect of an explanatory variable on five different binary outcomes (observed in the same subjects from the same population). The outcomes are not independent ...
- 443
2
votes
2
answers
572
views
Can I use a permutation test to determine whether overlap of 2 distributions is significant?
I am evaluating whether two distributions are different from each other. For example, histogram A (B) has most of its binned values between 10-15 (15-20). So, histogram B has a right-ward shift in ...
- 21