Questions tagged [multiple-comparisons]
Signals situations where one is concerned about achieving intended power and size when more than one hypothesis test is performed.
1,867
questions
0
votes
0
answers
15
views
Use of multiple comparison correction and composite DV
I am investigating the effect of a single X on a composite DV, Y. I want to also investigate the effect of X on the 36 sub-components of Y. Thus, I would have to run 37 multiple regression models, ...
- 1
0
votes
1
answer
10
views
Filter one-sided tests before FDR
In this blog post, the author shows one example of "weird" p-value histogram ("Scenario C"):
One provided explanation is that a one-sided test was run, and the tests where the ...
- 145
1
vote
0
answers
21
views
Recipes for combining multiple null-hypothesis tests
I have two questions about hypothesis testing:
Is there one or multiple generic recipes for combining tests together? Some procedure that would produce a new better test $T_\star$ from two tests $T_1$...
- 2,516
2
votes
1
answer
32
views
ANOVA finding a main effect prevents inflation of Type I error rate
In the (free) book Understanding Statistics and Experimental Design, section 6.3, they say
'...ANOVA offers a second trick. If we rejected the null hypothesis, it is appropriate to compare pairs of ...
- 123
0
votes
1
answer
24
views
What criterion to use to compare multiple correlations of binary variables?
I have $N$ definitions of certain properties of countries (for example, if the country is "democratic", "totalitarian" etc.), and want to test how consistently different people ...
- 101
0
votes
0
answers
26
views
Appropriate way to correct for multiple comparisons in this case
I am trying to compare the performance of an optimisation algorithm on different objective functions. The functions can vary in their form (4 different types) and dimension. I would like to first ...
- 3
6
votes
1
answer
418
views
Related low p-values that do not meet statistically significant thresholds
If I were to have multiple related measures, each with "low" p-values that do not quite meet the threshold of significance, could the fact that multiple measures produce a general trend with ...
0
votes
0
answers
21
views
Impact of variation in the total number of samples when performing multiple fisher exact tests
I have dataset of 2 time points. In each time point I have to compare the counts of different components at each position (number of positions =1000). I used fisher exact test to compare the if there ...
- 309
0
votes
0
answers
46
views
Steps to conduct a linear mixed model and post-hoc comparison test
I would like to ask for your help with the best way to analyze the following experiment within R.
Here is my design: there are 15 treatments, in four blocks. For each treatment, in each block, we have ...
2
votes
2
answers
44
views
Bonferroni correction necessary? regression to predict ONE outcome
I am working on a project with a small sample size where I have multiple predictors at baseline and one IV. I am trying to see if any of the DVs are good predictors for the score on the IV (continuous)...
- 21
0
votes
0
answers
12
views
Post-hoc tests for repeated measures ANOVA with aligned rank transformed(ART) data
I'm working on a 2x3 factorial design with two within-subject variables and I'd like to test for both main and interaction effects. Since the data is not normally distributed, I've performed an ...
- 1
4
votes
2
answers
67
views
Making 3 comparisons with a ternary variable
I have a study in which I want to study the effect of a ternary variable $x$ in interaction with a binary variable $y$. The ternary variable is categorical, and for some reasons, I want to do the $3$ ...
1
vote
1
answer
50
views
Multiple comparisons and explanation of two-factor experiment
I am doing investigation with temperature and accession on trait of plants with R.
Temperature: 25/20, 30/25, 35/30
Accession: V9,BTx623, 2-16-6, 1-10-4, 2-16-6, 19-22-12
The interaction and ...
- 123
2
votes
2
answers
194
views
How to compare two PMF generating functions?
I have two functions $f$ and $g$ that map an input vector $x_i$ to a probability mass function.
For example:
$$
x_i = [0,1,2] \\\\
f(x_i) \rightarrow [0.1, 0.3, 0.2, 0.2, 0.2] \\\\
g(x_i) \...
- 21
0
votes
0
answers
31
views
GLM Multiple Comparisons
I am performing several Generalized Linear Models in my analysis and I am wondering which method to use for adjusting p-values due to multiple comparisons.
I have 4 outcomes (judgement of intensity ...
- 11