Best practice for comparing data between completers and non-completers?

Question

I conducted a study which collected data on a brief one-time intervention at 3 timepoints (pre, post, and one-week follow-up). There were some participants who dropped out between these 3 timepoints. In a second version, I also compared 2 groups doing the same intervention, but under different conditions. Would the recommendations change in these circumstances?

I ideally want to only use data from participants who completed the surveys at all 3 stages, for the most 'accurate' idea on how effective the intervention was. So, what is considered best practice to ensure there were no significant differences between participants who dropped out and those who completed all 3 stages?

I searched around and the current plan is to use MW-U/independent t test to compare age, and chi-square for gender, SES, occupation to ensure there are no significant differences between dropouts and completers, and potentially a mixed ANOVA for the study using 2 conditions in addition to the 3 timepoints. Eyeballing the data at this stage, nothing looks drastically different, but worth noting in the results section perhaps. And if something does come up, would I just acknowledge in results section that certain demographics (for example) were more likely to not complete, then continue my analysis only using data from people who completed all stages?

Answer 1

A few quick ideas:

You can compare completers and non-completers using a range of statistical tests. Ultimately, the choice of test will typically depend most on the type of outcome variable. For instance, if it is binary or unordered categorical, you would most commonly use chi-square. If it is a typical numeric variable, you would most commonly use an independent groups t-test. That said, there are many other tools for looking at group differences, which might be justified in some cases. For instance, if the outcome variable was numeric but distributed non-normally, you might use non-parametric tests. If you had a bunch of numeric outcome variables, you could do an overall test of group differences using MANOVA.

You may also want to consider your alpha level. If you are comparing many different outcomes, you may want to correct your alpha. For instance, you could divide your alpha by the number of outcomes that you look at.

More generally, you may also want to look into "intention to treat" analysis.