120

Problem statement

Many new users are unaware of how to formulate a good question, particularly for the technical sites like Stack Overflow, Super User (OK, all of SE). This leads to many new questions being:

  • Closed
  • Downvoted
  • Deleted

In addition, it leads to a fair bit of community work trying to determine what they are asking. This leads to bad experiences for everyone - askers receive "hostility" and the community wasting effort.

Proposal

My suggestion is to create an A/B study analyzing the effects of pre-populating the question box with text, indicating the parts of a question that are important to a good question. The goal is to increase the quality of questions initially.

Right now, much of the burden on pulling out the details of questions are on the community. If a question is vague, it will normally not be improved without comments. If it's not clear, similarly the community has to ask/etc.

I would like to push this burden onto question askers more directly by pre-populating the required information for lower reputation users.

This would look similar to the following (probably could be word smithed), when clicking "ask a new question"

question template

Note that the intent is to make all this text visible in both the input field as well as the actual post body.

Depending on what site this is ultimately added on the specific wording may need to change, particularly the first line.

Desired outcomes

The goals of such a pre-formatted box are:

  • Make it clear what information is expected
    • If users don't fill this out it (or leave the pro format text) it is clear they did not bother to read the box they dumped questions into
  • Help with rubber-duck debugging
    • Questions which have information addressing those three questions are considerably more likely to be scoped/answerable questions
  • Guide users unfamiliar with SE (but otherwise capable in asking good questions) in how to do so
    • Often simple typo types of questions are useless on SE anyways
    • Many people are more than capable of asking good questions but are just unfamiliar with SE
    • Optimize for pearls, not sand
  • Provide users who receive hostility (downvotes/close votes) a rationalized feedback, particularly if they blatantly ignore the pre-populated text
    • If users ignore one or more of the sections they now are deliberately choosing to ignore how SE works

A/B study guidelines

I propose that this be implemented in an A/B study, with the following criteria and metrics.

  • Target users - users who have less than 50 reputation (or are at risk of a question ban, regardless of reputation)
  • Target site - while this fits on Stack Overflow most naturally, this could easily be adopted for a large number of sites
  • Metrics
    • Conversion rate of questions asked after clicking "ask a question." Lower % conversion --> good.
    • Close rate of questions asked, lower --> good
    • Median and quartile scores of questions asked, higher --> good
    • Average/median time spent writing a question (if this information is available), higher --> good
    • User retention, specifically how many users ask additional questions
Improve this question
21
  • 5
    Good news: it's in review. Bad news: it's in review for a year, meaning most likely it won't happen. (At least not in the near 6-8 years future)
    – Shadow Wizard
    Commented Apr 18, 2016 at 14:13
  • 4
    I'm not conviced this is an exact duplicate. The idea – providing a question outline to guide new users – is shared, but the ideas differ in two important points: (1) The almost-dup suggests a sophisticated template system, possibly extending the question editor. This question is not, and is trivial to implement. (2) This question suggests measurable metrics to find out whether such an outline is an actual good idea. Evidence-based action is the best kind of action. Such ideas are absent from the almost-dup. Ideally, both suggestions would be merged… @enderland maybe make it an answer?
    – amon
    Commented Apr 18, 2016 at 14:24
  • 7
    I really like how this proposal is presented, and am in 100% accord with its stated motivations. But the cynic in me suspects we can already predict the outcome of the proposed test: a marginal improvement for the small population of OPs who really care how their questions are received, and formulaic disregard by the overwhelming majority whose fundamental motivation is to get others to do their work for them, with the minimum possible effort on their own part. You can't cure lazy. Unless, of course, SE decides to refuse to display any questions that don't live up to some minimum standard.
    – Dan Bron
    Commented Apr 18, 2016 at 14:36
  • 7
    @DanBron that's an ideal outcome, though. I would much rather help those who are capable of actually writing good questions do so and make it clear when people are being lazy. I don't really mind downvoting/closing lazy questions. If someone leaves the pro-forma text without answering in a code dump I will ruthlessly downvote and vote to close that question.
    – enderland
    Commented Apr 18, 2016 at 14:47
  • 4
    @DanBron additionally, if it is the case that people refuse to address the pre-populated guiding questions - that would be a great case for dumping those questions into a review queue of some sort, etc. I didn't include any sort of analysis on the text body of the questions as that is much more difficult to easily get as far as the A/B study goes (presumably?) but I would imagine you could do some pretty interesting textual analysis for how the questions with the added text were received. I would guess unaddressed prepopulated text would be received far worse than otherwise.
    – enderland
    Commented Apr 18, 2016 at 14:54
  • 3
    I really like this idea. It would be perfect for a site like Hardware Recommendations, where we get a lot of off-topic questions and questions that don't have enough detail.
    – ArtOfCode
    Commented Apr 18, 2016 at 15:06
  • 2
    With a caveat-if any part of the template remains in the answer when submitted, the user is permabanned. And something else, but I can't figure out what's bad enough.
    – user1228
    Commented Apr 18, 2016 at 15:42
  • 2
    This is a pretty good idea, with that text being available for editing by mods. I imagine the team can't come up with a specialized message for each site.
    – M.A.R.
    Commented Apr 18, 2016 at 19:50
  • 3
    I love the idea, but what happens if one of the questions isn't applicable or if the users question doesn't fit the template entirely? They may be discouraged from asking.
    – angussidney
    Commented Apr 19, 2016 at 6:40
  • Don't forget to move title to bottom and change placeholder to "What is the problem you are trying to solve?" or something along these lines.
    – user138231
    Commented Apr 20, 2016 at 22:18
  • 4
    Whilst in principle I agree that something like this is very much needed, this template covers only a narrow subset of questions suitable for Stack Overflow - debugging questions that to my mind are not even the most interesting or useful. How would you fill in this template for a question that asks how to solve a particular problem for which you don't yet have any code (for example, you can't find a suitable library or API call)? Or what about a 'best practice' question?
    – Ian Goldby
    Commented Oct 19, 2016 at 7:50
  • 2
    @IanGoldby you realize that there is a specific close reason on Stack Overflow for "recommend a library/tool" right? If people don't ask that because they can't fit it into the template, perfect.
    – enderland
    Commented Oct 19, 2016 at 12:08
  • 4
    @enderland You miss my point. SO isn't just for "help me debug my code" questions. It is perfectly reasonable to ask how, for example, to obtain the month name in the local language in C#. There are thousands of high quality questions of this type. Finding answers to these kinds of questions is the main reason I come to SO.
    – Ian Goldby
    Commented Oct 19, 2016 at 13:11
  • 1
    To address Ian Goldby's point, you could present the asker with a small (ordered) range of templates to choose from, as well as a "free-form" template for those who believe that their question is still a valid fit for the site, with a stern warning that the question will get quickly down-voted and probably closed (and perhaps deleted) if it actually isn't an acceptable question.
    – PM 2Ring
    Commented Jul 26, 2017 at 17:12
  • 2
    Currently being tested on Stack Overflow: meta.stackoverflow.com/questions/360260/…
    – Shog9
    Commented Dec 7, 2017 at 4:36

2 Answers 2

Reset to default
11

Here's my take, for what it's worth.

Initially we should indeed probably start with something simple, as to gauge whether this even actually helps at all (in typical A/B testing fashion) before potentially overwhelming the user with a bunch of text, which could be off-putting.

I'm a bit worried that going too simple might actually hurt question quality (more so than an empty box), as users might include less details because they only fill in the relevant sections.

We should definitely initially (try to) avoid optional or misleading sections (or any sort of choice regarding sections), as this might potentially confuse or dissuade users (more than we might like or expect).

My initial proposal would probably be something like this:

Description of the problem:

Give a brief description of your problem here.

Also include (1) your input, (2) the output (or exact error) you're getting and (3) the output you want.

What I have tried:

Briefly tell us how you've tried to solve this problem. Did you debug? Did you try Google?

My code:

Put your code or code showing how you tried to solve the problem here. For posting advice, see https://stackoverflow.com/help/mcve.

This is what (I think) we should be striving towards: (more extensive with optional sections)

High-level description of the problem:

Give a brief description of the problem you're trying to solve here.

My question is:

Which specific part of this do you need help with or how do you want us to help you?

Put a one-sentence question here so we know exactly how to help you.

What research I've done:

Did you do any debugging? Did you Google your problem?

Include some links and explain why they did not help - this will make sure we don't tell you what you already know.

My code:

Put your code or code showing how you tried to solve the problem here.

If you take some time to construct a Minimal, Complete and Verifiable example (https://stackoverflow.com/help/mcve), this will make it a lot easier for us to help you.

Do NOT just copy a part of your program here, since the problem might be in another part of your program and then we won't be able to help you.

Example input:

Put the exact input you used for your code here.

Output I want:

Put the exact output you want to get for your input here.

Output (or error) I'm getting:

Put the exact output you're getting for your input here.

If an error message, make sure to copy the EXACT, full error message here.

Improve this answer
0
10

The template suggested in the question is problematic for a number of reasons:

  • Encourages users to treat SO as a "fix my code for me" service
  • Does not encourage research
  • Focuses too narrowly on issues where the user has existing code. (Some good questions don't even involve code. Many good code questions are too simple to need an MCVE and only need an expected input and output.)
  • Largely excludes tool questions (e.g., git usage)

Dukeling's answer suggests a short alternative that shares the same problems and a long alternative that is too verbose. (Users are unlikely to read through the long one.)

For this reason, I'd like to propose a different template based on a comment by amon:

# How can I.../What is...
Explain what you're trying to do or find out.

# I tried.../I found out...
Explain what you've tried and what information you found during your research.

# But it did this instead.../But my understanding seems wrong because.../But I'm confused because...
Explain what happened that you didn't expect or what makes it difficult for you to understand.

I'm certain this could use some tweaking, but I think this template is a vastly better starting point for the following reasons:

  • Encourages user to do research first and demonstrate it
  • Discourages "Here is my code, now fix it," code dump type questions
  • Encourages the user to include information about their own understanding of the situation, which makes it simpler to correct where they've gone wrong
  • Sets the correct tone that the user should be seeking information/knowledge
  • Is easier to generalize to more StackExchanges.

It does feel incomplete, and I think that's largely the reason Dukeling ended up with a too verbose template. If we feel like this isn't enough, then I think we should look more into something more in the vein of Petter Friberg's suggestion to have a sort of "wizard" or walk-through that prompts to the user to include specific information. If the system just dumps a wall of text at them, most users are just going to skip it.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .