4

There's been a sudden, massive spike in the number of posts in the Late Answers review queue:

326 posts in the late answers review queue

Seems like there's usually only a handful per day, certainly not 300. Most of these answers seem to have been posted several years ago. Some are even community wiki now:

old community wiki answer

Is this a bug, or is it an intentional effort to clean up old posts that were never reviewed?

1 Answer 1

6

This is a network-wide change. See this Meta.SE question and this answer to it:

Queue growth

Review queues lose effectiveness if posts aren't regularly (and accurately) cleared. A queue such as close votes on Stack Overflow that never seems to get to 0 fails to provide time-sensitive feedback that aids learning. So if we raise the bar for late answers which increases the number of tasks going into the queue, we risk making the queue less useful for people answering questions late.

As I write, the Late Answer queue on Stack Overflow is empty. In the last 24 hours the number (non-audit) review tasks of each type added to the queues:

Review Tasks Type           
------------ ----------------- 
2467         Triage            
2428         First Post        
2333         Close Votes       
1846         Suggested Edit    
 883         Low Quality Posts 
 619         Late Answer       
 594         Helper            
 141         Reopen Vote        These two factors suggest that there's room to add more tasks to the Late Answer queue. Assuming we

raised the reputation limit to 100, we'd add about 300 tasks:

Rep Bin Tasks 
------- ----- 
 10       417   
 20       160   
 30        85    
 40        76    
 50        44    
 60        41    
 70        37    
 80        21    
 90        20    
100        20   

You might expect the number of tasks associated with the 1 to 10 reputation bin would equal the number of late answer tasks in the last day. The discrepancy stems from reputation earned since answers were added to the queue. But the takeaway is that raising the reputation limit to 100 would add half again as much work to the queue. That doesn't seem particularly onerous.

Not surprisingly, Stack Overflow would see the most added Late Answer reviews by far. Other large sites, such as Ubuntu, Server Fault, Super User and Mathematics, would get 10 or so extra reviews a day.

Justifying 100 rather than 50 reputation

Discovering answers as comments certainly would suggest 50 as a good limit for this queue. But this queue is as much about granting exposure to potentially upvote-worthy answers as it is about finding problems. Especially on large sites, it's pretty easy for a good answer to a very old question to be lost in the noise. As a strategy for new users answering old questions that already have good, but not excellent, answers seems productive if not that few people will notice those answers. At 10 reputation, a knowledgeable new user has one shot at earning upvotes from excellent answers to aged questions.

A potential complication

The one concern I have is that answers are dequeued once the author has reached 100 reputation. If we raise the maximum reputation for going into the queue, we should also raise the minimum for coming out of it. Unfortunately, I don't have an easy way to gather numbers to set an appropriate level. One of the reasons I'm only looking back a day in the numbers I showed above is that it minimizes the effect of gained reputation. It's possible, but messy to estimate reputation at the time a user answered a question. So while my instinct would be to raise the deque minimum to 200, but I can't really predict if that's appropriate or not.

Proposal

Raising the bar to 100 would require changing two variables at once. Since that would make it harder to evaluate each individual change, I'd like to try raising the reputation for getting enqueued to the Late Answer reviews to 50 and reconsider upping it to 100 at some point in the future.

1
  • 1
    Ah, a network wide change, that would explain why I didn't see anything in meta.programmers. Thanks!
    – 8bittree
    Commented Sep 29, 2015 at 19:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .