"Flag" and "Recommend Deletion" options for answers that are merely unscientific opinions

Question

The following suggestions address the case where users treat the forum like a social media site (eg, reddit or slashdot), and post "opinion" answers, with no attempt to suggest that they have any grounds or basis for their assertions (example, example, example, example, example). Related to this, and possibly covered by the same option, are "data point" (anecdotal) answers that merely share personal experience (example, example, example).

Note: This case is different from the one mentioned here - the latter implying that answers make assertions based on unsourced / uncited evidence (and are not recommended for deletion), which is perhaps due to laziness rather than lack of scientific rigour, whereas the option I'm proposing is for answers that make assertions based on personal opinion / experience / belief, with no indication that supportive evidence even exists (and should be recommended for deletion).

I notice that psych.meta is heavily focused on question quality, and has virtually no discussion on answer quality. This is exemplified by @StevenJeuris's excellent comprehensive summary (and continuing discussion), that links to dozens of posts on questions. I suspect this focus is at least partly due to the way psych.SE's performance is tied to reducing the ratio of questions to answers, so there is incentive to reduce low quality / unanswered questions, but less incentive to reduce low quality / bad answers.

However, the purported benefit of reducing questions for attracting professionals to the community is only speculative. The science-minded of us know that to really find out how much it would benefit the community to tighten down on questions is to test it. I suggest that the quality of answers may also play a part in attracting or repelling professionals. That said, I hope this post might be useful for starting a discussion.

Note that the example answers I include below are still there as of this writing (several are upvoted and one of them is even an accepted answer!), so I don't think the current system of flagging, downvoting, and moderating is as effective as it could be. I myself have been discouraged from flagging or recommending for deletion non-scientific answers on the grounds that they don't violate any of the forum's policies, and would like to see that changed. This is my proposal:

---

The "flag" link for Answers has the following options:

1. spam
2. rude or abusive
3. not an answer
4. in need of moderator intervention

Additionally, when reviewing Low Quality Posts (ie, flagged answers), the "Recommend Deletion" button has the following options:

1. No comment needed
2. This is commentary on another post, not an answer
3. This is a “thank you” comment
4. This is an “I’m having this problem, too” comment
5. This is a different question posted as an answer
6. This is a link-only answer (and not spam)

My suggestions:

1. I would like to see a "flag" option: "not a science-based answer" - in addition to or in place of the current "not an answer" option - to discourage / remove unscientific opinion answers.
2. "Recommend Deletion" options 2, 3, and 4 are basically the same (this is a comment, not an answer - common for new users without sufficient reputation to comment), and should be merged as there is little reason to differentiate between them.
3. In their place, I would like to see the same option: "not a science-based answer" - to support the proposed "flag" option.

Thoughts?

Answer 1

I agree that this issue needs sorting in some way, but there is a problem using the low-quality flag approach when we have answers upvoted by others as the low-quality flag isn't given as an option.

The question arises whether a separate flag can be created in the first selection list for sites like us and MedicalSciences without affecting non-scientific sites which don't have the references requirements.

If that is possible, then I would go down that route and have that put in.

Answer 2

Unscientific Opinion:

The problem with opinion should be obvious: Everyone has one, and that doesn't help users get answers. We already ban questions that solicit opinions, so it stands to reason that we should ban opinion-based answers.

One question that has come up is: Why are the opinions of experts more valid than the opinions of lay people? The answer, I believe, is that strictly-speaking, they are not. A good answer relies on evidence. In practice, experts are more likely to be familiar with available evidence, and rely on it to construct their views, but any lay person who does the same can provide equally valid answers. In fact, this is exactly what allows a stack like this to exist, with contributions from both professionals and lay people.

Now, just having references or citations is neither necessary nor sufficient for good questions or answers, but evidence-based answers can at least be peer-reviewed and scrutinized for accuracy by other users, where opinion-based answers cannot. It is still important for users posting answers to vet their sources, for example to ensure that they actually support their claims, but even if they fail to do this, or fail to provide references at all, at least evidence-based answers can be validated.

Anecdotal Evidence:

One common argument is: Perhaps personal anecdotes are not useful on their own, but surely in aggregate they provide useful evidence upon which to base conclusions? The answer, I believe, is absolutely not. One only needs to look at the huge amount of medical quackery, and corresponding mental health quackery, to see why.

If anecdotes are bad evidence, then multiple anecdotes are not better evidence - in fact they are worse evidence - they simply multiply the bad effect. The result is the mistaken belief that good evidence has been collected to determine effect, when in fact, good scientific evidence shows that the effect is possibly the complete opposite.

There is a massive gap between scientific evidence and aggregated anecdotal evidence. Probably the most important distinction is the lack of controls in anecdotes. For example, a person evaluating a particular treatment has no comparison point to base their evaluation against - ie, the outcome of a no-treatment condition. For all they know, the no-treatment condition could have had better results. Good scientific evidence is controlled and randomized, and ideally double-blinded, multi-sited, and pre-registered. Additionally, the statistical analysis that researchers apply to their data is far more complex than just aggregating data points - it involves controlling for confounding variables and many other nuances.

Action Items:

Regardless of whether we add a new flag for this scenario or not, opinion-based and anecdotal answers can be downvoted, and/or flagged as "not an answer" or "very low quality", and be deleted accordingly.

Please vote up this answer if you would like to see the suggestion proposed in the question implemented; downvote if you don't.