Questions tagged [multiple-comparisons]
Signals situations where one is concerned about achieving intended power and size when more than one hypothesis test is performed.
1,860
questions
0
votes
0
answers
34
views
Steps to conduct a linear mixed model and post-hoc comparison test
I would like to ask for your help with the best way to analyze the following experiment within R.
Here is my design: there are 15 treatments, in four blocks. For each treatment, in each block, we have ...
2
votes
2
answers
41
views
Bonferroni correction necessary? regression to predict ONE outcome
I am working on a project with a small sample size where I have multiple predictors at baseline and one IV. I am trying to see if any of the DVs are good predictors for the score on the IV (continuous)...
- 21
0
votes
0
answers
10
views
Post-hoc tests for repeated measures ANOVA with aligned rank transformed(ART) data
I'm working on a 2x3 factorial design with two within-subject variables and I'd like to test for both main and interaction effects. Since the data is not normally distributed, I've performed an ...
- 1
2
votes
2
answers
63
views
Making 3 comparisons with a ternary variable
I have a study in which I want to study the effect of a ternary variable $x$ in interaction with a binary variable $y$. The ternary variable is categorical, and for some reasons, I want to do the $3$ ...
1
vote
1
answer
50
views
Multiple comparisons and explanation of two-factor experiment
I am doing investigation with temperature and accession on trait of plants with R.
Temperature: 25/20, 30/25, 35/30
Accession: V9,BTx623, 2-16-6, 1-10-4, 2-16-6, 19-22-12
The interaction and ...
- 123
2
votes
2
answers
193
views
How to compare two PMF generating functions?
I have two functions $f$ and $g$ that map an input vector $x_i$ to a probability mass function.
For example:
$$
x_i = [0,1,2] \\\\
f(x_i) \rightarrow [0.1, 0.3, 0.2, 0.2, 0.2] \\\\
g(x_i) \...
- 21
0
votes
0
answers
30
views
GLM Multiple Comparisons
I am performing several Generalized Linear Models in my analysis and I am wondering which method to use for adjusting p-values due to multiple comparisons.
I have 4 outcomes (judgement of intensity ...
- 11
2
votes
1
answer
35
views
"paired" comparison of more than two algorithms
When comparing two algorithms $A$ and $B$ on test data $x_1,\ldots,x_n$ by means of a performance measure that yields $A(x_i)$ and $B(x_i)$, there are two standard methods:
Check whether $\frac{1}{n}\...
- 5,362
0
votes
0
answers
12
views
When is Tukey post hoc test result identical to Games Howell post hoc test for Welch ANOVA?
In what situations is the Tukey post hoc test identical to the Games Howell post hoc test? I have an ANOVA analysis where HOV is violated and I have a statistically significant Welch ANOVA result. I ...
- 321
0
votes
0
answers
16
views
How to report few-shot accuracy for LLMs?
I am comparing three prompting techniques in LLMs to check which one is best. All prompting strategies include three examples for in-context learning (few-shot only, no fine-tuning).
If I do greedy ...
- 205
6
votes
2
answers
309
views
Is it multiple comparisons?
Our study hypothesis is that metabolic syndrome (MetS) has impact on some outcome. There are at least five definitions of MetS in the literature.
One of colleagues suggested using all five definitions ...
- 63
1
vote
1
answer
32
views
Mixed Effects with repeated measures design and covariate included
I have a dataset that consists of the oxygen consumption (MO2) values for 3 different species of fish. My experiment was a repeated measures design where I measured MO2 of the same 14 fish per species ...
- 31
3
votes
1
answer
34
views
Significant results from ANOVA post-hoc Tukey's, insignificant with welch-ANOVA post-hoc Dunnett's T3
I performed one-way ANOVA tests with post-hoc Tukey's to look at multiple comparisons and got highly significant results, but noticed the $F$-value on the ANOVA was high. The SDs in the groups were ...
- 31
2
votes
1
answer
62
views
Is it possible to do statistic analysis from only knowing my groups's means and sample sizes?
I am writing about a problem my colleagues (in human health field) consider impossible to solve, but I want to believe mathematics have an answer. I am being told it is impossible to do statistical ...
3
votes
1
answer
50
views
Identical SE values for different groups using emtrends
I'm using emtrends to extract slopes and do pairwise comparisons between the groups of my independent variable, controlled for the other variables in the model. ...
- 131