TLDR: Many questions are not reviewed by enough users to accumulate sufficient votes to close, and as a result, what few close votes are cast, end up aging away and are effectively wasted.
I expect that it can be frustrating for users who take the time to cast close votes to find that questions don't get closed simply because no one else looks at them. It wouldn't surprise me if some users have stopped using the review queue because of this, compounding the problem. This concern has been raised before, so my intention with this post is to provide much more detail, and concrete options for moving forward.
Background
I did not know much about how the review queue works before looking into this, so here are some details:
- Users with at least 500 rep can vote to close or reopen questions.
- In general, 5 close votes are required to get a question closed, except for moderator votes, which are binding.
- The first close vote adds a review task to the close vote queue, and it remains pending for up to 14 days.
- If the review task expires, then the close votes also "age away" (invalidate), which means that any future close vote starts the process anew. The reason for this is because if close votes did not age away, then eventually all questions would get closed - given infinite time, they would eventually accumulate sufficient close votes.
- Note that task expiry happens silently.
Statistics
We can get a rough idea of the health of the review queue by looking at the average number of users that go through each review task:
 |
Note that averages are below 5 partly because some percentage of tasks are completed by moderators, while others expire. Assuming that moderator intervention is more or less constant, the differences are largely due to user activity. Averages over the past 2-3 years indicate that most review tasks do not have sufficient reviewers to complete without moderator intervention.
Options
There are several options available to low-volume sites like ours:
- We can ask to have our site require only 3 votes to close questions instead of 5. Since the close vote privilege is based on rep, rather than the knowledge or trust needed for this task, there is a risk that question closing may be influenced by a small group of unqualified users. Moreover, many review tasks currently don't get even 3 reviews before aging away.
- We can ask to change the grey circles to red. Currently, review queues get a grey circle when there are tasks pending, and turn red when there are many tasks pending. Making the queue red whenever there are any pending tasks has apparently helped some other low-volume sites get more attention to their review queues.
- Moderators can manually monitor for tasks about to age away using the close votes tool, or search for tasks that have already expired using the data explorer, and cast their binding vote if they agree with existing votes. I personally have started doing both of these tasks, but hammering questions with aged away votes has caused problems before, so the trick is to close just before votes expire...
- I have posted a proposal on MSE to warn before close votes expire, giving both users and moderators a chance to rescue these tasks, if appropriate. My preference is to let users have their say before hammering a question, but I also would be happy to intervene with my binding vote if I knew it was needed. While I think this solution would be ideal, even if it was accepted, it may not be implemented for years.
If you like any of these options, or have your own to suggest, then post an answer accordingly, and we'll see how the community votes.
