[Update: June 01, 2021]
First and foremost, thanks again to the community members for providing their thoughts and feedback on this phase of the project. We read through the community responses and took some more time to think about our approach. Now, we’re back to share an update.
We will work on this in Q3 2021. Closer to launch, we will publish a more detailed breakdown post on Meta.
Planned changes
First posts → New user question AND New user answer queues
- To avoid context switching, we are creating two separate queues to handle the individual post types.
- The purpose of these queues remains the same – helping new users learn how to use the site. With that in mind, reviewers will be able to choose from three actions: Looks OK, Edit, and Share feedback.
- Reviewers are encouraged to leave comments for the post-author, but now they can also choose from a few options of canned feedback which will appear as a comment from the Community account.
- Additionally, if a user’s first post is not successful (i.e. receives negative votes), the next couple of subsequent posts will enter the appropriate queue.
Deprecate Help & improvement
- Questions that would have ended up in this queue from Triage will now enter the New user questions queue.
New actions for Late answers queue
- As previously mentioned, the Late answers queue has been successful in identifying and removing low quality posts, repeat answers, and spam.
- We’re adding definitive actions to address those exact problems: Looks OK, Edit, and Delete.
Keep Triage queue
- After some back and forth, we’ve decided to keep the Triage queue. There wasn't a strong indication on either side of the argument to get rid of it. Triage does get in front of poor questions and as mentioned earlier in this post, we’ve already addressed the stickier issues with this queue.
- Triage will continue to handle questions only.
- The Triage queue will also be available on all sites but only turned on by default on Stack Overflow. Have a meta discussion and ask a moderator to status-review if your site wishes to add the Triage queue.
Name change: Low quality posts → Low quality answers
- This queue will continue to handle answers only on Stack Overflow.
Other
- We’ll retire badges associated with deprecated queues, but this won’t have any effect on badges already earned in these respective queues.
- We still plan on prioritizing aging review tasks
- No planned changes to the Suggested edits or Close votes queue
What about the Reopen votes queue?
Any edit, regardless of its substance, currently sends a closed question for review in the Reopen votes queue. With this logic in place, well-meaning, grammatical edits could prohibit further reopen-worthy edits from being reviewed.
This issue is a bit bigger than the review queues workflow itself, but we’re working on a proposal. Look forward to a separate post on this matter.
This time last year, the Public Platform team announced an initiative to improve the review queues. Our goal was to explore new ways to better support our curator and moderator communities by revisiting the review queues used for community moderation.
With the help of the community, we have been able to accomplish a significant amount of work and made multiple releases including:
- Improvements to the review suspension experience
- New review queue help center articles
- Visual design changes to the queues
- Review queue onboarding
We’ve made a lot of progress with this project, but there is still work to be done. Today, we’re coming back to the community to discuss one of our last objectives: improving queue workflows and pathways. In parallel to our other releases, we’ve been taking more time to carefully consider what changes would make the most impact on the review queues. We’d like to share some new proposals and get your feedback.
The original plan
Our initial research helped us identify the following pain points in review queue workflows:
Problem #1: The Requires editing action in Triage was sending a surplus of unsalvageable posts to the Help & Improvement (H&I). H&I reviewers are then limited to either Skip or mark the task as Very low quality, having it return to the Triage queue and creating a frustrating endless loop.
Solution (implemented): We updated the actions available in the Triage queue and redirected tasks to where they may be better addressed.
- Needs community edit → Help & improvement queue
- Needs author edit → Close votes queues
- New flag action for other problematic posts
Problem #2: First Posts and Late Answers queues and combining those tasks into Triage. These queues were intended to give good posts a head start toward success and filter out bad posts, but tasks could see better, faster intervention.
Solution (not implemented): Initially, we considered deprecating these queues. When we got into technical discovery, we discovered that we couldn't simply move First Posts and Late Answers tasks into Triage because the Triage queue only accommodates question tasks.
New considerations
We wanted to look at these problems again with fresh eyes. We revisited our internal conversations, reviewed some data, and reread the Meta feedback from previous posts to ideate on new solutions.
Deprecate Triage
- Triage is a Stack Overflow exclusive queue and was created to quickly assess and provide feedback on questions that had been flagged low quality. This queue is often the root cause of other review queue issues. It lacks strong guidelines for reviewers to follow and assumes that they can identify good questions, both leading to inconsistent reviews.
- Currently, the Low quality post queue on Stack Overflow only handles answers. By deprecating Triage, we’d like to adopt how the Low quality posts queues on other SE sites function and route all posts (questions and answers) that have either been flagged as low quality or fall below the system-generated quality score threshold to Low quality posts.
Deprecate Help & improvement
- H&I hasn’t been successful in its intended purpose with issues from Triage diminishing the queue’s usefulness. Improvements to other queues could still satisfy the same need.
- H&I is our least productive queue. Over the last 90 days, only 7 tasks went into the queue each day on average, compared with 2,458 for First Posts (our busiest queue) and 219 for Reopen Votes (our second-to-last busiest).
Create separate First posts - questions and First posts - answers queues
- The First posts queue had good intentions in helping new users with their first posts, but the queue itself has no clear goal. Reviewers can take any or no action against a review task and is often not impactful or useful to the post.
- We want to separate questions from answer tasks and identify specific actions that may help new posts fare better as well as maintain content quality on the site.
Keep the Late answers queue
- The Late answers queue has been effective in removing low quality posts, repeat answers, and spam.
- Similar to First posts, we want to identify specific actions that help in resolving problematic answers.
Other: Prioritize aging review tasks
- Review tasks age out of the queues if they are not acted upon, so we’d like to ensure posts get the attention they need in a timely manner.
Next steps
We will not be conducting 1:1 user tests or interviews in this release. Instead, we would like to open this up for discussion on Meta to gather a number of opinions. Please share your thoughts until April 28, 2021. Afterwards, we’ll analyze your responses and share new designs. We’re excited to read your responses and look forward to learning from your insights.