How can we improve the quality of new users' challenges?

Question

_{All stats given here are correct at the time of writing. I can't promise that they'll stay exactly correct for a significant amount of time (and they most likely won't), but I'd be surprised if the pattern challenges significantly}

Of the last 50 questions that were posted to main (and that are still undeleted):

• 33 were posted by an inexperienced challenge author (\$\le3\$ open, positively scored questions)
• Of those, 18 are negatively scored
• And, 20 are closed (7 as duplicates)

Looking at the deletion stats (10k+ only) for the same period, we can change those numbers to

• 41 were posted by inexperienced challenge authors
• 25 have a negative score
• 28 are closed (10 as duplicate)
• 8 are deleted (6 deleted by Roomba)

How can we improve these stats?

We have the Sandbox specifically to avoid this - somewhere that newer users, less experienced with challenge writing, can go in order to learn how to post high quality posts. Clearly, however, it isn't working, and in fact, some users aren't even aware of it before posting their first question.

In order to see exactly what it's like to post a question as a new user, I logged into one of my sockpuppets and went through the motions of posting a question. My observations:

• Our info box is obviously visible, but doesn't stand out in a page full of links and more. In fact, visiting my latest question, more space was given to ads than the info box

• When I click the "Ask Question" button, I was greeted by our custom modal, which, as I've said before, isn't bad, it's just not great.

• I then drafted a low quality question, and hit "Review your question". Something that immediately caught my eye:

  Our automated system checked for ways to improve your question and found none.

  Granted, we shouldn't expect an automated system to be our absolute filter for low quality questions, but I can think of a few automated checks that could be helpful, such as checking for winning criteria tags, or looking for patterns that may indicate the post is off-topic or low quality.

  Additionally, as noted by Unrelated String, for users who are less familiar with the SE system, this could be seen as an "endorsement" that their question is good to go, rather than the system telling them there are no "system" errors (e.g. missing title, no tags etc.)

• During this process, there was only ever a single time where I was directed to our Tour page and our how to ask pages, both of which are supposed to be there to help newer users ask high quality questions, and that was in the Step 1 while asking my question, along with a lot of information in a fairly small box. Even if I had visited the Tour/asking pages, though, I'm not sure it would've helped:

  You should only ask practical, answerable questions based on actual problems that you face. Chatty, open-ended questions diminish the usefulness of our site and push other questions off the front page.

  _{What types of questions should I avoid asking?}

  Have you thoroughly searched for an answer before asking your question? Sharing your research helps everyone. Tell us what you found and why it didn’t meet your needs. This demonstrates that you’ve taken the time to try to help yourself, it saves us from reiterating obvious answers, and above all, it helps you get a more specific and relevant answer!

  _{How do I ask a good question?}

  Focus on questions about an actual problem you have faced. Include details about what you have tried and exactly what you are trying to do.

  _{The Tour page}

  We've asked hundreds of times about our misleading Tour/help pages, so consider this one the latest in a long series of similar questions. That doesn't change the fact that the pages are misleading at best, and downright wrong at worst.

  Furthermore, despite it being the go-to, number one piece of advice offered by countless users across the site, our how to ask page only mentions the Sandbox once, and just doesn't mention other useful resources such as our faq or pages such as Things to avoid when writing challenges

• "This site is for programming contests and challenges. General programming questions are off-topic here. You may be able to get help on Stack Overflow." is one of our primary close reasons (19.05 % of closed questions in the last 90 days), behind "Needs details or clarity" at 44.44 % and "Duplicate" at 23.81 %. This suggests that a lot of users post here without fully understanding what the site's actually about.

• Finally, the Sandbox isn't even fully accessible for new users, as due to a bug that has gone unfixed since September 2020 (going on 7 months now), users require 5 reputation to post on Meta.

---

In short, the systems we have in place in order to help newer users post high quality questions are not working. New users do not get directed to helpful pages until after they post here, they don't get adequately informed about the purpose of the site, and they don't have access to, or the knowledge of, valuable and helpful resources.

So what can we do about it?

What suggestions do you have to make it easier for new users to post good quality questions? What can/should we change (or try to change) in order to decrease the number of low quality posts?

Answer 1

As far as I can see, there are two big issues here that both need addressing:

• Visibility of useful resources
• How helpful those resources are

Visibility

No one likes having a whole bunch of links to a whole bunch of posts thrust into their face as soon as they do anything. That said, some links are close to essential for new users in order to properly understand the site.

To avoid having a massive standard comment, or heavily cluttered info box or similar, I suggest we collate Meta posts which we believe are all but necessary for new users to read, and tag them with something like new-users-guide to allow newer users to be pointed to the tag and able to browse through the posts.

This guide can then be constantly linked to every step of the way for new users. Link to it from the Tour page, in the sidebar when writing a question, in the Info Box at the side of the site, in the modal that pops up when you ask a question. We can't force people to read something, but by encouraging and linking it enough, we can make sure that the vast majority of people are aware of what the site is about before ever posting anything.

As I said in the question, there's nothing that stands out about places to improve your challenge before posting. When new users answer a question, they get a big yellow info box. Having one of these for challenges could be very helpful, directing new users to resources such as the Sandbox, the Tour/help pages or a new-users-guide.

Helpfulness

As noted in a lot of meta discussions, the Tour and Help pages aren't fully relevant and/or helpful to newer users.

There have also been recent examples of users not understanding how long to leave a question in the Sandbox, suggesting that the purpose of the Sandbox ("A place to leave a draft so that others can suggest feedback until it's ready to be posted") isn't fully understood by all.

And, of course, new users can't post in the Sandbox.

Improving our "automated systems" would be a significant first step. Escalating the reputation issue to SE and emphasising it's importance, customising our asking modals and tag warnings, and correcting our Tour and help pages would hopefully lead to fewer off-topic and low quality questions. And, even if nothing changed, fixing these would be an improvement to the site

Finally, updating our faq pages, promoting some of the faq-proposed pages, and linking new users to the FAQ would help those users better integrate into the site.

---

TL;DR: If I had to sum this answer up in one sentence, it'd be this:

Collect all our helpful information, links, pages and more into a single location and promote links to the location at every step of posting a question

Answer 2

Warnings

From what I can tell, the review screen is somewhat customizable. It doesn't seem like it can test for not having a tag (i.e., not having an objective winning criterion), but we can add regex based warnings for the body or tags.

I suggest the following:

Language tags

Some of the more common of these, like javascript, java, c, etc., with warnings against language restrictions

Restricted source

Add a warning for Do X without Y, with an appropriate link

Other possible options

Later today, I'm going to download every undeleted challenge on CGCC, and look for the tags and substrings that are most common in low quality questions. Strategically adding warnings prompted by these could catch a fair number of challenges, if it works.