So I've just come from a conference attended by psychologists who share an interest in concerns about the replicability of the field's findings, as well as considering what may be done to improve it.
Repeatedly, conference attendees highlighted the need for a Q&A resource where they would be able to ask questions, get answers, and learn about aspects of the replication "crisis", open-science, and meta-science, thereby generating a lot of methods-focused discussion.
Naturally, I thought, "this sounds perfect for a Stack site". After finding the Cognitive Sciences Stack though, I'm not quite sure what intended "target market" of users is these days, as from what I've seen in other posts on the meta Stack, it seems as though this may be a period of transition for the site.
I wanted to therefore check in and ask:
Are questions on these sorts of topics (replicability of psychological research, open-science methods, meta-science) considered as falling within the purview of the Cognitive Science site?
For example, someone might ask for an explanation of current methods used to quantify the replicability of psychological literatures, and how they differ from one another. A good answer might include accessible descriptions and comparisons of methods like p-curve (Simonsohn, Nelson, & Simmons, 2014), R-Index and TIVA (Schimmack, 2014), statcheck(Nuijten et al., 2015), and GRIM (Brown & Heathers, 2016), and might even point to a user-friendly resource like p-checker (http://shinyapps.org/apps/p-checker/) that helps psychologists use these methods themselves.
Some other hypothetical questions of this sort could include:
- Asking about the merits/concerns of pre-registration of hypotheses, materials, and data-analysis plans, as well as recommendations for pre-registration services.
- Asking for an explanation of how various meta-analytic methods attempt to assess and correct for publication bias in the psychological literature
- Asking for a summary of evidence supporting/challenging the impact of the adoption of open-science practices in the field.
Though some of these questions (like #2) and others might also be relevant on other Stacks sites (e.g., CrossValidated), the issues seem especially relevant to psychologists as our field seems preoccupied with concerns about replicability and building a cumulative science.
If questions like these are considered on topic, I suspect there a number of psychologists that would enjoy using this site as resource to learn about those issues (and I and others from the conference could help to draw attention to this site, for that purpose).
