124

We need a hard rate limit on suggested edits, to limit the damage someone can do with blasting the review queue with minor edit suggestions.

Case in point: this user peppered the site with a whopping 133 tag-only edit suggestions in 2 hours and 15 minutes. That's one edit every minute on average, but towards the end he was entering a new suggestion every 20 seconds. He only stopped because a call to action here on meta drew enough rejections to have him banned from entering more.

If there was a limit to the number of suggested edits you can enter in a time period we could have stopped this guy earlier. No <2k user needs to edit hundreds of tags across that many questions in 2 hours time.

Considerations

  • There are cases where a retag drive or tag burninate justifies editing a larger number of posts, but I'd be fine if such tasks were limited to people with 2k+ reputation. A retag or burninate job requires a minimal engagement with the community to know when they are justified anyway.

  • The suggested edit review process, by itself, was clearly not enough to stop this barrage. It cannot be seen as a rate limiting measure, at least not as a sufficiently efficient one.

    The vast majority of the edits made in the above example were too minor, leaving spelling, grammar and formatting mistakes in the post untouched. The user was going too fast to pay attention to such details anyway. Yet this user has 98 approved edit suggestions, but all but 6 or 7 I'd have rejected as too minor, so the review process itself is broken.

What should be limited

  • The number of review votes reviewers have is already limited; I burned through 20 reviews in short order just reviewing the suggestions from just this one user. Certainly the number of unreviewed suggestions should be limited to let reviewers catch up with a user like this. I'd say having 20 outstanding suggested edits pending review is plenty.

  • Suggested edits should be made with some consideration; we don't want minor edits, so if you are suggesting more than 20 in an hour (that's 1 every 3 minutes) you are not improving posts anymore, you are gaming the system to gain reputation.

Improve this question
21
  • 7
    Why not have a daily rep limit of 10 for edits? Cf. 200 upvote daily rep limit.
    – Bathsheba
    Commented Dec 6, 2013 at 14:43
  • 20
    Why not give the editor -1 rep for a rejected edit?
    – Bathsheba
    Commented Dec 6, 2013 at 14:44
  • 3
    That would not necessarily stop such a barrage. It would take away an important incentive, sure.
    – Martijn Pieters
    Commented Dec 6, 2013 at 14:44
  • 1
    There are other, related, feature requests that suggest a penalty for rejected suggested edits.
    – Martijn Pieters
    Commented Dec 6, 2013 at 14:45
  • 2
    Hmmm, almost two years ago a similar question popped up. meta.stackexchange.com/questions/116636/… No definitive answers though or a status-declined
    – rene
    Commented Dec 6, 2013 at 14:45
  • 14
    @Bathsheba I don't think we should punish users for attempting to help in good faith.
    – Stijn
    Commented Dec 6, 2013 at 14:46
  • 3
    Indeed I'm sure this is not a new phenomenon but being able to acquire 1000 reputation just by adding tags etc. doesn't sit well.
    – Bathsheba
    Commented Dec 6, 2013 at 14:46
  • 1
    Were the proposed edits appropriate except for their size? Or do you think the additional tags didn't belong there?
    – Peter Alfvin
    Commented Dec 6, 2013 at 14:46
  • @Stijn: you mean the -1 for a rejected edit? The rate limit is not a punishment, just a natural break in how fast they do this.
    – Martijn Pieters
    Commented Dec 6, 2013 at 14:47
  • @PeterAlfvin: See the other post for the specific details on that user. I am only addressing the frequency here.
    – Martijn Pieters
    Commented Dec 6, 2013 at 14:47
  • @MartijnPieters Yeah I meant the -1 for a rejected edit. (I've actually upvoted this question)
    – Stijn
    Commented Dec 6, 2013 at 14:49
  • 1
    Actually, it might be beneficial to ratelimit >2ks too. Tag wikis should not be edited lightly.
    – Manishearth
    Commented Dec 17, 2013 at 9:49
  • 1
    Related: Do not allow edits on different posts within 5 minutes and Should the amount of edit suggestions per day be limited.
    – S.L. Barth is on codidact.com
    Commented Dec 24, 2013 at 9:47
  • 1
    @S.L.Barth you can always flag an edited post and ask a moderator to suspend the user.
    – Shadow Wizard
    Commented May 18, 2016 at 12:06
  • 1
    @PythonMaster202: because we don't want people to be making 100 tag edits a day. Seriously, if you need to retag that many posts, post on Meta to discuss this.
    – Martijn Pieters
    Commented Jan 3, 2020 at 20:34

5 Answers 5

Reset to default
75

As of a few seconds ago, this limit is now active network-wide:

  • at-most 20 pending suggested edits per editor on beta sites
  • at-most 5 pending suggested edits per editor on graduated sites

These limits are based on my analysis of the speed at which edits are generally approved and the number of edits (and editors) that would have been affected over the past few months. They can be adjusted if it becomes necessary. For future reference, here's a query that estimates who would be blocked at a given threshold and another that calculates the median time to review edits for a given site, in minutes.

Kudos to Michael Stum for implementing this!

Please report any bugs here on meta.

Improve this answer
19
  • 3
    What happens if you try to edit the wiki and excerpt on three tag wikis? The 5th and 6th edits would be submitted with the same click.
    – yellowantphil
    Commented Jul 13, 2016 at 3:51
  • 6
    I haven't done a thorough review, but I believe it would go through without complaint (and then you'd be blocked on the next wiki you tried to edit), @yellowantphil - the check runs before you're shown the editor, not as the edit is submitted (because that'd be cruel and frustrating).
    – Shog9
    Commented Jul 13, 2016 at 3:55
  • 4
    Michael Stum is on the Stack Overflow team? Sweet.
    – BoltClock's a Unicorn
    Commented Jul 13, 2016 at 4:13
  • 4
    Couple of questions: first, is there any daily limit? Or can users still submit 200 suggested edits in a day? Second, what about anonymous suggested edits?
    – Shadow Wizard
    Commented Jul 13, 2016 at 7:31
  • @ShadowWizard Going by the information available, I would say that it's a window, not a hard limit. So, no, limits per-volume aren't implemented.
    – Braiam
    Commented Jul 13, 2016 at 13:03
  • @ShadowWizard: given the current median time of 28 minutes on SO, on that site you should be able to have about 250 suggestions approved per 24 hours, if you time them just right (e.g. submit a new suggestion every 5 minutes and 36 seconds). In practice the time can vary greatly of course, so my money is on mass editors giving up earlier than that.
    – Martijn Pieters
    Commented Jul 13, 2016 at 13:09
  • @MartijnPieters true, no doubt the new limit is good and welcome, just wonder if it's accompanied by any other limits.
    – Shadow Wizard
    Commented Jul 13, 2016 at 14:20
  • 1
    Keep in mind, the goal here is not to say anything akin to, "> X edits / period is bad and you should feel bad" - it's merely to make folks aware that, while suggesting, they're using a shared resource... And they can't use it all.
    – Shog9
    Commented Jul 13, 2016 at 18:30
  • at-most 5 pending suggested edits per editor on graduated sites I hope no Suggested Edits queue has thousands of suggestions waiting just like SO's close votes...
    – SE is dead
    Commented Jul 13, 2016 at 19:34
  • 4
    No. There's a hard limit of 200 pending edits, at which point no one can suggest any more, @dorukayhan. Currently SO is at 168.
    – Shog9
    Commented Jul 13, 2016 at 19:37
  • The pending edit queue size for SO was experimentally increased from 200 to 500 in June. meta.stackoverflow.com/a/351728/149428
    – Taylor D. Edmiston
    Commented Aug 27, 2017 at 19:04
  • 1
    Yes, that's temporary @TaylorEdmiston - we're currently working on a better solution: meta.stackoverflow.com/questions/355233/…
    – Shog9
    Commented Aug 28, 2017 at 16:03
  • Actually, the check does happen when submitting the edit - I tested this by opening the editor on more than five posts (which the system allowed me to do since I had zero pending edits), then submitting the edit on five of them. On trying to submit the sixth I was prevented from doing so.
    – Sonic the Anonymous Hedgehog
    Commented Dec 10, 2017 at 7:42
  • @MartijnPieters 28 minutes for tag edits? Unbelievable: last year, my tag edit suggestions were always in the queue for 2-10 days on SO.
    – Cœur
    Commented May 13, 2018 at 2:01
  • 1
    @Cœur: right, see this updated query; tag wiki edit suggestions have a median time to approval of 1054 minutes, so 17.5 hours (!!!).
    – Martijn Pieters
    Commented May 13, 2018 at 10:48
23
+125

There are 2 main problems that I see that would be resolved by rate-limiting of some fashion:

  1. As mentioned, this would prevent someone flooding the review queue
  2. The second is someone repeatedly making the same minor edit for a large number of posts.

As Martijn mentions in the question, limiting the number of pending suggested edits would go a long way to resolve #1, but it would only partially address #2 as it would allow someone to resume editing as soon as an edit is reviewed (whether it be accepted or rejected), and in sites with fast moving review queues (i.e. Stack Overflow), this could be almost immediately.

We know that an edit ban kicks in when enough of the users edits in the last 7 days have been rejected, but this is sometimes not enough when someone is editing en masse. They can usually garner enough accepted edits to raise the threshold for an edit ban. If you are able to maintain a 3-to-1 accept to reject ratio, you'll never trigger the threshold. Enough so-so looking edits mixed in with a handful of robo reviewers and you'll be able to maintain that threshold.

To avoid this, I think further rating limiting based on edit rejections would be a fair solution. The exact numbers can be fleshed out with further discussion, but the idea would work like this:

  • User is rate limited to 20 pending suggested edits
  • This limit is reduced by 5 for every rejected edit in the last hour (the reason for the time limit is to prevent this from being a complete ban and just a way to slow them down until the community can handle the situation).
  • So if a user has 20 pending suggested edits and gets a rejected edit, they are now limited to 15 pending suggested edits. This means they can't propose any new edits until they have fewer than 15 pending edits. And this number would be further reduced if they pick up any additional rejects in the preceding 60 minutes.

A bonus of this is 4 reject edits in any 60 minute period effectively bans the user for the remainder of the hour. It would be a very temporary ban, but should be a sufficient period of time to allow the community to catch up and/or leave him a message explaining why their edits are not acceptable.

This could have an additional benefit on sites that have very strict tag burnination processes (Stack Overflow, I am looking at you). It's not uncommon to end up finding some "helpful" soul trying to clean up a tag that hasn't been completely resolved (or even if it has been resolved, just starting a mass tag-only editing spree). By both limiting edits in the queue and further limiting by rejections, it helps the community to limit the damage and allow the user to be educated on the process and proper editing.

Improve this answer
4
  • 1
    +1 for gradually reducing the available edit slots. This way, we can educate the editors in an early stage.
    – S.L. Barth is on codidact.com
    Commented Dec 24, 2013 at 9:56
  • Good point about the tag burninations. But the editor who made me raise the bounty was not just retagging. (If only they were!)
    – S.L. Barth is on codidact.com
    Commented May 18, 2016 at 12:05
  • I was curious as to how much of an effect this would have, so I put together a little query to help guess at it.
    – Shog9
    Commented Jun 24, 2016 at 1:27
  • I'd greatly appreciate this system over the one we now have (only 5 suggested edits)
    – ejderuby
    Commented Aug 9, 2019 at 18:33
11

To counter the answer given by Lance Roberts:

At the onset I dislike the idea of rate limiting the number of edits we can suggest in a day. I have been a prolific editor myself and it was the privilege of suggesting edits which encouraged me to increase my participation on sites in the Stack Exchange Network. Any rate-limitation of the likes that I can only suggest x edits in a day (where x is too small) would have irked me for sure. I have easily crossed 20 edit-suggestions-a-day on numerous occasions - at times the similar number of my edits were pending review.

Moreover, an anonymous user is also able to suggest edits - how would you deal with it? I don't consider this as spamming if done in good faith, so using the IP level ban/restriction doesn't seem appropriate to me if it's not spam.

However, what Martijn Pieters proposes:

Suggested edits should be made with some consideration; we don't want minor edits, so if you are suggesting more than 20 in an hour (that's 1 every 3 minutes) you are not improving posts anymore, you are gaming the system to gain reputation.

should be fine in my opinion and provides a good balance for rate-limitation and quality of suggested edits.

I am sort of neutral on his other suggestion:

The number of review votes reviewers have is already limited; I burned to 20 reviews short order just reviewing the suggestions from just this one user. Certainly the number of unreviewed suggestions should be limited to let reviewers catch up with a user like this. I'd say having 20 outstanding suggested edits pending review is plenty.

As I mentioned in my answer earlier, there have been times when there have been more than 20 of my edit suggestions pending in review queue. However, once I reach there I am sure I won't feel bad that I can't review more for the time being - as I know that I can edit more once the community catches up and starts trusting me more. So, I am neutral on that suggestion.

Improve this answer
1
  • 2
    Note that my other suggestion limits your unreviewed suggestions. On site with an actively managed suggested edits review queue, having 20 reviews still outstanding is going to be almost impossible. But on a quieter site, pushing 20 reviews into the queue would slow you down in proportion to the reviewing capacity of the site.
    – Martijn Pieters
    Commented Dec 19, 2013 at 19:26
4

Another option is to limit the rep mining in these cases -

  • Option 1: Allow users to edit as fast as they want but +2 rep eligibility is replenished only once a minute or more
  • Option 2: Allow users to edit as much and as fast as they want, but limit the rep from edits to overall 20 a day (or any other value). To sweeten that a bit you can add a badge saying the user have reached X daily edits on Y days.
Improve this answer
8
  • No, a badge for a number of edits is only going to lead to more gaming. I think that not handing out rep for edits is going to lead to a lot of confusion, complicating the rules too much.
    – Martijn Pieters
    Commented Dec 22, 2013 at 14:10
  • 1
    @MartijnPieters, handing out almost unlimited rep is just as bad. Ok scratch the badge, why not limit the rep rate? if someone invested 5 seconds in an edit (like the guy above who probably didn't even read the questions), does he deserve the rep for it?
    – Leeor
    Commented Dec 22, 2013 at 14:26
  • 1
    There is already a cap on suggested edit reputation. You cannot earn more than 1000 points that way.
    – Martijn Pieters
    Commented Dec 22, 2013 at 14:53
  • I vote for option #2.
    – Victor Zakharov
    Commented Jun 12, 2014 at 14:00
  • @MartijnPieters Huh, I didn't know about that. Why is that? It seems odd that no matter how many good edits you have, you'll never earn the ability to skip the suggested edit approval process at 2k. (It would take a pretty long time to get 1000 good edits, but still...)
    – jrh
    Commented May 11, 2017 at 14:48
  • @jrh: at 2 points per edit, you'd only need 500 edits to get there. We want you to do other things besides editing to gain reputation.
    – Martijn Pieters
    Commented May 11, 2017 at 15:00
  • 1
    But I enjoy editing posts, and tons of posts need it...
    – ejderuby
    Commented Aug 9, 2019 at 18:50
  • @ejderuby, that's great, and you can continue doing so, but the reputation award should be limited. Editing is not efficiently peer-reviewed so, by itself, it should not give you enough rep to get advanced privileges. You need to combine it with answering and receiving votes on your answers. The +2 award is just a good way to attract beginner participation.
    – Leeor
    Commented Aug 11, 2019 at 17:19
1

The limit for approving suggested edits is 20 a day, so I would think that would be a good limit for suggesting those edits.

Improve this answer
1
  • 10
    I don't agree, actually. There is more than one reviewer on any given site. This is why I proposed a limit to the number of unreviewed suggestions you can push into the queue. 20 in a day is very low, really. That is about 1 every half hour, in a 10 hour period.
    – Martijn Pieters
    Commented Dec 19, 2013 at 19:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .