25

We all know the problem with the close votes queue, but it seems the problem is people not reviewing enough. Whenever I had time, I would go through all 40 that I was given, but by then I wanted to do more. Would raising that limit of 40 and not counting Close Votes queue votes towards your daily close vote allowance help?

16
  • 3
    I'm not sure it would do any good to the quality of reviews.
    – Bart
    Commented Jul 26, 2013 at 8:11
  • 7
    That would be a few hundred gold badges and an empty review queue, I guess.
    – Rob W
    Commented Jul 26, 2013 at 8:21
  • 16
    I don't know about removing it, but raising it might help a little. I get pretty bored after about 50-60 reviews, and I have a higher than normal tolerance for boring, repetitive tasks. I see a full page of people max out on reviews at 40 every day. It would be interesting to see how long that continued if the maximum were doubled. Commented Jul 26, 2013 at 12:14
  • 2
    @Bill But you have to also consider the maximum number of close votes. Can't you omly even get up to 50. I feel that a review limit above the close vote limit would just encourage people to use Leave Open so they can maximize their reviews.
    – animuson StaffMod
    Commented Jul 28, 2013 at 4:02
  • Is there a Meta Q where its been definitively settled that the size of the close vote review queue is actually a problem that is causing real harm?
    – joran
    Commented Jul 28, 2013 at 4:22
  • @animuson you bring up a good argument. However, the close votes queue is only open to 3k+. If you have that much reputation, you should be trusted enough to close correctly.
    – Cole Tobin
    Commented Jul 28, 2013 at 5:41
  • @animuson Not counting close vote review queue votes as part of the normal daily allowance was mentioned in the question already. Commented Jul 28, 2013 at 13:07
  • @ColeJohnson No, none of those Q's address my question. The only thing in there that addresses whether the size of the close vote queue is an actual concern is this which is Shog9 basically saying that we should worry less about it. So I'll ask again, where has it been discussed and established that the size of the review queue is actually causing harm? What harm is it causing?
    – joran
    Commented Jul 28, 2013 at 13:09
  • @joran harm? None that I've seen. Other than the big number detracts people from actually wanting to use it.
    – Cole Tobin
    Commented Jul 28, 2013 at 15:13
  • @animuson We're already trusting 2k+ users to edit without needing to be reviewed. I'm sure we could trust a 3k user more than a 2k one.
    – Cole Tobin
    Commented Jul 28, 2013 at 15:30
  • @ColeJohnson Well, that seems like the sort of thing that actually needs to be established, rather than just asserted, before we start tinkering with the system too much. Because as far as I can tell, the close vote review queue sees quite a lot of reviewing action.
    – joran
    Commented Jul 28, 2013 at 18:15
  • 2
    @joran I can tell you that it detracts me knowing that after my 40 reviews, all I see is the same number and a notice saying I can't review anymore.
    – Cole Tobin
    Commented Jul 28, 2013 at 19:08
  • the problem has been (at last) acknowledged and addresed by SE team: Enough fuzzying: let's let everything into the close queue and age out questions that don't reach a threshold
    – gnat
    Commented Apr 29, 2014 at 22:03
  • 1
    There is ongoing work on the subject of the close vote queue so I'm voting to not reopen this question as to keep focus on recent events.
    – bad_coder
    Commented Oct 18, 2022 at 18:14
  • 1
    @Joachim this was originally on meta.SO (before the SO/SE meta split). So the migration pulled it here and closed it.
    – Cole Tobin
    Commented Oct 19, 2022 at 18:35

3 Answers 3

29

Instead of increasing the review limit for everyone, I suggest incrementally increase it based on current rep, somewhat similar to the system of number of post flags per day based on rep. This would be the best balance.

  • 40 for > 3k rep
  • 50 for > 5k rep
  • 60 for > 10k rep
  • 80 for > 20k rep
  • Unlimited for trusted users (25k rep, after all they are trusted)
7
  • 4
    as a side-note, it would save us from threads like these popping up requesting for increasing reivew limits, when users can earn that privilege themselves. Commented Aug 3, 2013 at 16:01
  • Nope, I currently have 40 close votes per day at 8.8k. With this suggested system, I would have 10 more review queue close votes per day. Commented Aug 3, 2013 at 16:23
  • That's confusing. I have 50 a day, but the queue only allows me to use 40. The other 10 I have to use on my own.
    – Cole Tobin
    Commented Aug 3, 2013 at 16:25
  • 3
    those numbers are just for illustration. Commented Aug 4, 2013 at 1:56
  • I have 7.8k rep, and a limit of 40 close vote reviews per day as well.
    – Flimzy
    Commented Nov 12, 2013 at 17:13
  • Maybe instead of rep, base it on how many questions you've reviewed? Because my rep is just inching (due to the barrage of low quality questions in the queue drowning out good questions), but my review count keeps going up. It'll be a long time before I hit 4k, but I'm already past 1000 reviews IIRC. Why can't I get a review allowance bump?
    – Cole Tobin
    Commented Jan 27, 2014 at 17:57
  • @ColeJohnson, that could work, though some might prefer something more than "you can vote because you have voted". Reputation points is used for all other privileges on SO. Commented Feb 21, 2014 at 4:33
12

The real problem is that too many questions that probably should be closed are on the site and need to be dealt with by up to 5 users - and then they get edited and enter yet another queue when the OP tries to get it re-opened. This ends up taking up a lot of reviewers' time. We also have the problem of so, so, so many duplicate questions getting introduced, and in a lot of cases people just answer them because that is easier than finding a duplicate (I am certainly guilty of that).

What if we had an additional queue to prevent bad questions from appearing in the first place? Let's say a "new question" queue where posts meeting any of the following criteria are placed before they appear on the site, or at least before you can answer:

  • low quality filter (obviously)
  • posts from users with < x rep
  • posts from users that have had more than x% of their questions closed
  • posts from users who have never asked a question

Perhaps they only appear to users with certain rep until they are approved, like deleted posts are only visible to 10K. But if they meet any of these criteria, you should be able to comment on them but not answer, until they are approved. For folks wanting to answer the question and earn rep, this is motivation to approve them.

This spreads out these questions into two review queues: bad questions that were posted and then community members marked them as bad, and potentially good questions that need to be approved. This buys reviewers time to find duplicates, too, before anyone wastes any time posting answers. Now, how will this decrease the workload? Perhaps it only requires three certain-rep users to approve a question. How will we motivate this queue to be processed as quickly as it is populated? Give more rep and/or additional badges, and make its daily limit much higher. Preventing answering the question (just as if it were closed or locked) would be motivation as well.

8
  • 2
    this would probably not happen but i like the idea that each question needs to PASS(get accepted by X amount of at least 1K+ users for example) a review queue before appearing on site!
    – user221081
    Commented Oct 9, 2013 at 14:02
  • 3
    The Reopen Quee is nearly always empty however. That shows how few questions are actually being tried to be reopened.
    – Cole Tobin
    Commented Oct 9, 2013 at 15:15
  • @ColeJohnson sure, and I would expect that, but it's still a lot of extra work in those cases. And it just reinforces that, if most questions end up getting and staying closed, the less work it takes to do that, the better.
    – Aaron Bertrand Staff
    Commented Oct 9, 2013 at 15:18
  • 1
    Maybe we could lower the amount of reviewers needed to close a question? Why do we need 3 for edits, but 5 for close and reopen? There are some situations where more reviewers are needed for suggested edits (bad reviewers) and a lower close requirement would lower the number. Maybe 4 on both?
    – Cole Tobin
    Commented Oct 9, 2013 at 15:20
  • -1. I think this proposal moves the problem rather than solves it. It's almost like sweeping bad questions under the rug during a period of time. Commented Oct 9, 2013 at 16:10
  • 1
    @Simon great, so what is your proposal? And did you notice that my proposal moves the problem and in many cases will reduce the amount of work required to deal with it?
    – Aaron Bertrand Staff
    Commented Oct 9, 2013 at 16:37
  • This proposal would frustrate new users to the point of them not returning if points 2 and 4 were to be implemented. Most people who ask a question usually want as speedy a reply as possible, and points 2 and 4 would just hinder that for new users.
    – RobH
    Commented Dec 2, 2013 at 17:40
  • 4
    New users who write garbage questions are the reason we have to close so many. Maybe we want them to be frustrated. Commented Dec 2, 2013 at 19:52
1

I firmly believe that raising the cap would help tremendously. I frequently max out the cap, and so do dozens of other folks. It isn't that hard to do, especially if you filter the queue to streamline your review -- most close votes are either so easy I can decide them in 15 seconds or less or are far enough outside my expertise that the obvious move is to skip the question and move to the next one. I would review more if I didn't get capped at 40.

Is there really a problem? If so, what?

I have only been active on SO for a few months, and I obviously have only been able to access the review tools for part of that time. During that time I have nonetheless watched the queue grow from about 90k to more than 110k. That kind of number verges on the realm of the ain't-gonna-happen. But having 100k+ potentially bad questions on here drags down the quality of the site, so it needs to get fixed.

Why raise the cap?

The fundamental question here is: why cap it? Nobody wants to see a bunch of reviewers get trigger-happy and close tons of good questions. But the point here is this: these questions have already been flagged for closure. We're not talking about allowing newbs to go all Lone Ranger and unilaterally close the most famous questions on SO; we're talking about letting trusted users cast one of five votes necessary to close a question that has already gotten at least one of those votes or been flagged by the system as probable junk.

[Edit: Cole's comments raise the opposite concern, that of badge-whores auto-voting to leave bad questions open. This is a legitimate concern, too, but that's the entire purpose of the audit system; it puts the brakes on when it becomes clear that people are on autopilot and not paying attention. Besides, no one (non-moderator) voter can keep the question open. Again, the point here is that these are supposed to be trusted users. If we can't trust users to review responsibly, the entire SO/SE model is broken. I don't think it is, and I don't think others who are bothering looking at meta in the first place think it's broken, either.]

Other suggestions

I agree with Samuel that tying the cap to reputation makes sense. This helps limit the Lone Ranger problem and ensures that reviewers have significant experience with the site before they can go too crazy with close votes. But Samuel's point and mine are the same in this respect: the cap should be raised.

Aaron's suggestion is appealing at first glance, and an approval queue for new questions by new users may become necessary. That said, I agree with the comments on his answer that such a queue would frustrate new users and reduce or eliminate one of the best values of this site: the ability to get high-quality answers very quickly. An approval queue would also lead to gamesmanship: a reputation-seeking reviewer has an incentive to craft a great answer, then vote to approve the question, then post the answer within seconds of the question going "live." New users -- who lack access to the approval queue -- would find it nearly impossible to compete with this except on the most difficult questions, further driving down their incentive to participate actively on the site.

One more suggestion

There is no reason the number of votes required to close needs to be five in all circumstances. That number should be lower if, for example, the question already has significant downvotes. Likewise, if it is flagged by multiple users as a duplicate and the system can detect substantial similarities between the question and its duplicate with some simple statistics, that should reduce the number of votes required. There are probably other examples that have not come to mind.

7
  • 2
    I've watched it grow from 50k during my time here. It's unacceptable!
    – Cole Tobin
    Commented Jan 27, 2014 at 17:52
  • +1 except that the problem with robo reviewers is that they click through with "leave open", and if a question gets too many "leave open", it leaves the queue
    – Cole Tobin
    Commented Jan 27, 2014 at 17:55
  • @ColeJohnson I'm not sure I agree with your comment on robo reviewers -- I think at least as many people get in a habit of clicking "close" instinctively. I know I fight that tendency myself, sometimes, especially after reading a bunch of incoherent questions in a row. This is the problem with any kind of human review: decision fatigue eventually wipes us out. But, that 110k-question queue isn't going away on its own, so I think we have to run the risk of some bad robo-reviews, at least until we knock it down a lot.
    – elixenide
    Commented Jan 27, 2014 at 18:02
  • 1
    I'm not saying all robo reviewers do that, but badge whores do. I agree with you that us, the good reviewers, can fall into the habit of just "closing" however.
    – Cole Tobin
    Commented Jan 27, 2014 at 20:12
  • @ColeJohnson Fair enough -- I agree with that completely.
    – elixenide
    Commented Jan 27, 2014 at 20:33
  • trigger happy? have you ever been in the close votes queue? I think I vote close 90% of the, because they are truly crappy questions... Commented Mar 20, 2014 at 13:07
  • @NickyDeMaeyer I assume that question was directed to me; correct me if I'm wrong. Yes, I have been in the close votes queue -- I've completed about 1600 actions there so far. I have voted to close about 95% of the time; based on the filters I usually use, very few good questions make it in there. I'm just saying that it's possible to get into an auto-close mode and not read carefully enough. I've definitely been guilty of that a couple of times.
    – elixenide
    Commented Mar 20, 2014 at 13:46

Not the answer you're looking for? Browse other questions tagged .