Problem statement
Many new users are unaware of how to formulate a good question, particularly for the technical sites like Stack Overflow, Super User (OK, all of SE). This leads to many new questions being:
- Closed
- Downvoted
- Deleted
In addition, it leads to a fair bit of community work trying to determine what they are asking. This leads to bad experiences for everyone - askers receive "hostility" and the community wasting effort.
Proposal
My suggestion is to create an A/B study analyzing the effects of pre-populating the question box with text, indicating the parts of a question that are important to a good question. The goal is to increase the quality of questions initially.
Right now, much of the burden on pulling out the details of questions are on the community. If a question is vague, it will normally not be improved without comments. If it's not clear, similarly the community has to ask/etc.
I would like to push this burden onto question askers more directly by pre-populating the required information for lower reputation users.
This would look similar to the following (probably could be word smithed), when clicking "ask a new question"
Note that the intent is to make all this text visible in both the input field as well as the actual post body.
Depending on what site this is ultimately added on the specific wording may need to change, particularly the first line.
Desired outcomes
The goals of such a pre-formatted box are:
- Make it clear what information is expected
- If users don't fill this out it (or leave the pro format text) it is clear they did not bother to read the box they dumped questions into
- Help with rubber-duck debugging
- Questions which have information addressing those three questions are considerably more likely to be scoped/answerable questions
- Guide users unfamiliar with SE (but otherwise capable in asking good questions) in how to do so
- Often simple typo types of questions are useless on SE anyways
- Many people are more than capable of asking good questions but are just unfamiliar with SE
- Optimize for pearls, not sand
- Provide users who receive hostility (downvotes/close votes) a rationalized feedback, particularly if they blatantly ignore the pre-populated text
- If users ignore one or more of the sections they now are deliberately choosing to ignore how SE works
A/B study guidelines
I propose that this be implemented in an A/B study, with the following criteria and metrics.
- Target users - users who have less than 50 reputation (or are at risk of a question ban, regardless of reputation)
- Target site - while this fits on Stack Overflow most naturally, this could easily be adopted for a large number of sites
- Metrics
- Conversion rate of questions asked after clicking "ask a question." Lower % conversion --> good.
- Close rate of questions asked, lower --> good
- Median and quartile scores of questions asked, higher --> good
- Average/median time spent writing a question (if this information is available), higher --> good
- User retention, specifically how many users ask additional questions