Questions tagged [multiplicity]
The multiplicity tag has no usage guidance.
14
questions
1
vote
0
answers
23
views
Combining Survey Weights
I have an annual survey that is somewhat complex in design. Sample frames are pulled quarterly and overlap. Each quarter's sample is removed from the subsequent quarter's frame. Samples are stratified ...
- 426
3
votes
1
answer
115
views
If we know that there were no type 1 errors in previous tests, should we still make corrections for multiplicity?
Suppose there's a study where an interim analysis was performed to evaluate efficacy with the intention of stopping the study. However, since the predefined success criterion wasn't achieved, the ...
- 2,207
2
votes
1
answer
54
views
Including Covariate P-Values in Multiple Comparison Corrections"
I'm currently working on a statistical analysis involving multiple comparisons and I'm uncertain about a specific aspect of the process. Specifically, my question is about the inclusion of p-values ...
- 53
1
vote
0
answers
114
views
Why are equally-wide confidence intervals used with the Benjamini-Hochberg correction?
This paper says:
*2. For any p-values that are significant after FDR correction, construct a CI for the corresponding test with coverage $1-\alpha^{'}$, where $\alpha^{'}$ is: $\alpha^{'}=\frac{k}{m}\...
- 1,252
0
votes
0
answers
50
views
Friedman test_multiplicity
I'm performing Friedman test to see if there is a difference between 8 treatments, then if it is significant I'll do Wilcoxon signed rank test to see which pair is significant. I wonder if I should ...
- 11
4
votes
2
answers
67
views
Does the Prospective-Alpha-Allocation-Sequence (PAAS) actually control the family-wise error rate?
I recently read about the PAAS as an alternative to Bonferroni. Unlike Bonferroni, where $\alpha$ is subdivided among multiple tests such that
$$ \alpha_1 + \alpha_2 + \ldots + \alpha_k = \alpha $$
...
- 63.7k
1
vote
1
answer
355
views
Equivalent to Dunnett's test for binomial endpoint?
I have a trial comparing 2 treatments vs a control group. Typically one would use Dunnett's test for this situation, but the outcome measure is binomial. It seems strange to me but I've looked for ...
- 559
3
votes
3
answers
672
views
Are methods of p-value adjustment only necessary when a universal/overall hypothesis is the target of inference?
I am trying to understand more clearly in which cases an adjustment of p-values is necessary. At the moment my reasoning about this can be summarized as follows:
Benjamini and Hochberg (1995) list ...
- 226
0
votes
1
answer
30
views
Should multiplicity be addressed when doing preliminary examination of missingness?
When dealing with missing data, one recommended initial step is to examine the effect of the missingness in each variable on the other variables. This entails doing multiple tests where a significant ...
- 48
7
votes
1
answer
213
views
Possible multiplicity issues in equivalence testing
I'm currently looking to compare 3 groups, trying to prove equivalence between them. For this, 5 specific parameters are measured for each group, and a pairwise comparison for each parameter among the ...
- 71
0
votes
1
answer
429
views
P value adjustment for pairwise comparisons of negative binomial
Question 1) Am I appropriately doing the pairwise comparisons?
Question 2) Why is the Sidak method used for the emmeans? Should I instead use adjust = "none" for these?
Context:
I have a ...
- 3
2
votes
0
answers
51
views
Multiplicity Correction for Coefficient Estimates in Linear Regression [duplicate]
When regressing a dependent variable y on some feature vector x with a standard linear regression, is there any correction in place for multiplicity or is this just not relevant in this case?
The ...
- 31
1
vote
1
answer
972
views
Multiplicity correction when using emtrends to see if slopes are different from 0
I have a relatively straightforward question but I haven't been able to find the answer. Consider this example:
...
- 165
8
votes
1
answer
311
views
Bayesian Inference in the presence of multiple hypotheses
"Because [Bayesian Inference] respects the forward flow of time or information, there's
no need for nor availability of methods for correcting for
multiplicity ... The evidence of one question
...
- 63.7k