51

We un-gamified flagging too early by cutting off the gamified incentive after the first 500 flags; we need to provide more incentive. Let’s encourage more participation in policing the seemingly unstoppable flood of AI-generated plagiarism posted all over the network, especially but not uniquely on Stack Overflow. This would also help with fighting spam.

We should therefore have the system re-award another Marshal badge for every 500 helpful flags, not just for the first batch of 500 alone.

Culpa mea

I initially wrote:

For a brief period seven years ago, Marshal badges were awarded over and over again, once per 500 helpful flags. Unfortunately this accidental feature was an unintended one rather than one made after due consideration of a feature request, and so the code that let it happen was "fixed".

In a comment, animuson kindly corrected my misunderstanding of the previous incident:

Just as a note: the issue you cited has absolutely nothing to do with this. The bug was not that it was awarding badges for each 500 flags. The bug was that the query was not detecting users already had the badge and re-awarding it to them. Every user eligible for the badge was receiving duplicates regardless of flag count.

Sorry about that. I still think we should re-award the Marshal badge for each successive 500 helpful flags earned.

Improve this question
11
  • 20
    We do also have a spam problem at the moment that might help with.
    – Journeyman Geek
    Commented Jul 11, 2023 at 3:24
  • I'd have two badges this month already, just flagging spam ;)
    – Tetsujin
    Commented Jul 11, 2023 at 7:21
  • 3
    One concern though is that users with <3k reputation have it a lot easier to earn this badge, since they can cast recommend closure flags to increase their count while users with close-voting privileges can't.
    – Sonic the Anonymous Hedgehog
    Commented Jul 11, 2023 at 8:21
  • 1
    @SonictheAnonymousHedgehog I think one can argue that encouraging these users cast correct (i.e. confirmed by closure) close flags would be a benefit, not a concern
    – gnat
    Commented Jul 11, 2023 at 8:24
  • 3
    @gnat-onstrike- The point I was trying to make is that it's much easier for less-privileged users to earn those badges than those with more privileges. Badges should be just as easy to earn regardless of privilege level, to the extent practical. Case in point: a suggested edit being approved too late results in users being denied edit and answer badges through no fault of their own.
    – Sonic the Anonymous Hedgehog
    Commented Jul 11, 2023 at 8:31
  • 3
    Maybe it's just me, but badges don't mean much to me. I see them fall from the sky at, for me, random moments and their elusive names don't immediately explain wat they're for. So I never check why I seemed to have earned them. In fact, I hardly even notice them. It just makes me wonder if more badges would motivate people? I know what discourages me though and that is a company that doesn't support community consensus. It makes me feel our efforts are futile in the end. For me that can't be compensated by badges.
    – Gert Arnold
    Commented Jul 11, 2023 at 8:32
  • 3
    Just a reminder that ChatGPT answers are not banned network-wide; please check first. I get occasional flags from people who don't realize.
    – Rebecca J. Stones
    Commented Jul 11, 2023 at 9:07
  • 1
    @SonictheAnonymousHedgehog your logic makes me feel that it is rather in favor of adding this badge. Thing is, 3K users already have rep-dependent advantage in getting Steward badges by doing reviews in close queue (especially at SO) and from this perspective badges that can be achieved via close flags can be considered as a balancing measure that levels playing field for users under 3K
    – gnat
    Commented Jul 11, 2023 at 10:07
  • @gnat-onstrike- Well, I did say "wherever practical". That problem is unique to Stack Overflow and doesn't apply much for the rest of the network. Keep in mind also that 3k+ users are limited to 24 votes per day on most sites, 50 on SO, while <3k users can cast up to 100 flags per day based on their flagging history.
    – Sonic the Anonymous Hedgehog
    Commented Jul 11, 2023 at 15:52
  • @SonictheAnonymousHedgehog I don't see how this is unique to SO because relative balance between close flags and reviews is about the same at any site. Smaller sites have less close reviews and respectively less amount of awarded close flag badges. As for the difference in the amount of flags and reviews, it is unlikely to make Marshal easier to obtain than Steward because the latter involves only a reviewer while close flag has to be unanimously confirmed by 3 to 5 higher rep users (Leave Open reviews resolve such flags into disputed)
    – gnat
    Commented Jul 11, 2023 at 17:42
  • 1
    @SonictheAnonymousHedgehog <3k rep users' closure flags need to go through the queue, and in my past experience on SO, many of those age away. It's not efficient. I would even say that higher rep users (>=10k) have an advantage (at least on SO) with access to mod tools like new answers, where it's easy to find non-answers to flag (except for the fact that <10k users can just follow various chatrooms like Dharman's or the Natty bot's)
    – starball
    Commented Jul 11, 2023 at 22:53

3 Answers 3

Reset to default
18

I'm all for it. I had a similar idea and floated it a while ago in the Tavern:

floating a feature-request idea: award number of copy editor / marshal badges as a log function of total (Ex. total 500 helpful flags / edits => 1 marshal, 1k => 2 marshal, 2k => 3, 4k => 4 marshal, 8k => 5 marshal, etc) or a root function. thoughts?

I went for a log or root function to stave off arguments that it would be too easy to drown in gold badges: If you're really prolific, you can do 200 flags on a single site per day. If you use a linear function, that'd be the ability to earn two gold badges every five days, at which point I think it kind of loses its meaning as a gold badge (not that I'd personally have any problem with drowning in gold badges for what I've been spending my free time doing).

Improve this answer
1
  • 7
    I mean if you are really spending so much time on a site to get 200 flags in per day, I would say that you do deserve to drown in gold badges. However I do agree that it would lose its meaning at some point.
    – bobeyt6
    Commented Jul 11, 2023 at 12:54
12

I am in support of this idea, since it encourages lower-rep users to participate in moderation activities and high-rep users to flag low-quality answers. I would also suggest adding a separate ChatGPT detection badge to encourage weeding out GPT spam (after the current AI policy issues have been resolved). This would need a new flag such as the one that previously existed on SO, but I surmise it wouldn't be too hard to implement network-wide.

Improve this answer
18
  • 15
    It's far, far more frequently the answers that are AI-generated than it is the questions or comments. And while all three types do occur, I'd guess the ratio is 2 to 3, perhaps even 4, orders of magnitude more answers than questions. If you think through why people are doing this and how they are doing it, you'll probably see why it works out this way.
    – tchrist
    Commented Jul 11, 2023 at 2:55
  • 1
    @tchrist I'm not sure what you mean. My answer doesn't say anything about questions/answers.
    – bobeyt6
    Commented Jul 11, 2023 at 3:07
  • 11
    You said that people stop flagging once they gain close-vote privileges. Surely that only applies to flagging a question for closure before they gain close-vote privileges, right? You should still be flagging answers for spam or abuse or plagiarism or not an answer or ai-generated content, and so on and so forth, even after you can close-vote a question.
    – tchrist
    Commented Jul 11, 2023 at 3:11
  • 1
    @tchrist Oh, I see. I will edit my answer
    – bobeyt6
    Commented Jul 11, 2023 at 3:19
  • 17
    It doesn't make sense to have a ChatGPT detection badge unless and until there's a separate flag reason for this.
    – Cody Gray - on strike
    Commented Jul 11, 2023 at 4:32
  • 1
    How many lower-rep users do already have the first Marshal badge and thus would be encouraged by this? On U&L I see only two users with a reputation below 1000 with Marshal badge. Your argument does not convince me.
    – Philippos
    Commented Jul 11, 2023 at 5:56
  • 1
    I know it's popular to hate ChatGPT for the moment, but c'mon, let's not get carried away. If Wikipedia came out today, would we create an anti-Wikipedia badge? It was considered laughably unreliable at first, after all, anyone in the world could edit it.
    – Rebecca J. Stones
    Commented Jul 11, 2023 at 9:17
  • 4
    @RebeccaJ.Stones wikipedia is written by people and encourages citations. It has peer review and editing.
    – starball
    Commented Jul 11, 2023 at 9:18
  • 7
    @RebeccaJ.Stones If users were blindly plagiarizing anything, but even worse so, unreliable and often wrong Wikipedia info, onto SE sites in the hopes of grabbing a few quick points here, without having to put in any effort themselves... Yeah, some user would probably ask for a badge for cleaning up that crap too.
    – Tinkeringbell Mod
    Commented Jul 11, 2023 at 9:21
  • 1
    Somewhat related: New flag for ChatGPT answers
    – Elements In Space
    Commented Jul 11, 2023 at 10:29
  • 1
    @Philippos-prostrike- I don't have 3k on mathematics, but I have over 1.2k helpful flags, and I know a couple of other similar users as well. But I guess a second marshall badge would be mainly targeted at higher rep users.
    – bobeyt6
    Commented Jul 11, 2023 at 12:51
  • 2
    @Tinkeringbell The point is as AI improves and people calm down and become more familiar, and the problems we're having are resolved, an "anti-ChatGPT" badge will seem rather short-sighted. (Besides, there's not even a separate "spam" badge, and I didn't find a relevant feature request.) This piling on is alienating users, like me, who wish to use AI responsibly. Sure, plagiarism is a problem, but how can a user responsibly cite ChatGPT in their answer when they know how other users will react?
    – Rebecca J. Stones
    Commented Jul 11, 2023 at 14:44
  • 4
    @RebeccaJ.Stones How can you responsibly cite a tool that plagiarizes (as ChatGPT doesn't refer to the sources it uses to come up with its texts)? I don't think I want to know why you would even want to do that. IMO, it's a very, very, very good thing that users think thrice about citing the tool they used to plagiarize other sources, and end up not using the tool at all.
    – Tinkeringbell Mod
    Commented Jul 11, 2023 at 14:47
  • 2
    @Tinkeringbell Here are two (upvoted) answers of mine where I use and cite GenAI: 1 and 2. I used GenAI for help writing my answers, so I feel the responsible thing to do is to cite it and mention how I used it. Is that unreasonable in some way? Without even looking, you had already formed a strongly negative opinion---this kind of blind "ChatGPT bad" attitude, and not just this instance but from many users piling on in general, is alienating, and an "anti-ChatGPT" badge would make things worse.
    – Rebecca J. Stones
    Commented Jul 11, 2023 at 15:51
  • 7
    @RebeccaJ.Stones yes, I do find that unreasonable. Those language models learn their things somewhere, yet I have never seen one give proper attribution to the sources it got its information from. So you basically plagiarized something from somewhere, and only referenced a "middle-man" to make it seem like attribution is given. And my sample size of not seeing AI providing attribution to its source material (or even worse, just making something up) is larger than 2 examples, so it's nowhere near a blind attitude. I have looked, I have seen more than enough crap to warrant my judgment.
    – Tinkeringbell Mod
    Commented Jul 11, 2023 at 16:00
3
+50

We should therefore unfix the fix and have the system re-award another Marshal badge for every 500 helpful flags, not just for the first batch of 500 alone.

Downside: some users may game the system by overzealously flagging comments. E.g., it is easy to quickly flag 500 not-so-relevant comments. In fact, I wouldn't be surprised if one could use GPT to flag such comments (= using GPT to gain badges made to fight against GPT). So if people really want to accumulate Marshal gold badges, I'd suggest only counting flags on questions or answers, and not flags on comments.

Improve this answer
10
  • 4
    If the flags are judged "helpful", what difference does it make?
    – tchrist
    Commented Jul 12, 2023 at 1:01
  • 1
    @tchrist No difference, I am favor of using GPT when it works. I was just enjoying the irony.
    – Franck Dernoncourt
    Commented Jul 12, 2023 at 1:03
  • 1
    if you're concerned about load on mods for non-auto-delete-comments, see meta.stackexchange.com/a/384181/997587
    – starball
    Commented Jul 12, 2023 at 6:41
  • 6
    flag-removal of 500 useless comments would certainly be a useful achievement worthy of a gold badge and here I think your concern is not quite well presented. The real danger seems to be that folks can start coordinated comments flagging exploiting auto-removal feature with sole purpose of getting next Marshal badge with indiscriminate removals (CC @tchrist). System can't currently track this and even if it could, I can not imagine a sensible way to handle such a misuse / abuse. Probably the only realistic way to tame this issue is as you suggest - to limit repeat badges to only post flags
    – gnat
    Commented Jul 12, 2023 at 8:38
  • @gnat-onstrike- Some people may start to flag more borderline comments. I've seen plenty of comments that I found useful but that got deleted anyway.
    – Franck Dernoncourt
    Commented Jul 12, 2023 at 9:51
  • 1
    yes, and with repeat Marshal badges for comments flags this will likely get much worse than now
    – gnat
    Commented Jul 12, 2023 at 10:07
  • 2
    Flagging lots of comments seems like it would hardly be a problem; mods would immediately notice & react if needed. (On my sites, the user who seems to flag comments the most only flags 1–2 every few days.) Also, the GPT suggestion seems ridiculous — it's a fast track to being flag banned, plus there are better ways (using ML too nonetheless!) to decide what to flag (but probably still a fast track to a flag ban: exhibit A & exhibit B).
    – Laurel
    Commented Jul 12, 2023 at 12:22
  • 1
    @Laurel at SO this would be much harder than on smaller sites, especially if auto-removal of comments turns out into popular "team sport" driven by badge hunting. Add a difficulty of justifying such removals as abuse worthy of penalising - one can argue that there is nothing wrong in removing content that is officially declared as "ephemeral", "second class citizen". Add that unlike it is with deleted posts, such removals can be reviewed only by moderators, so whole burden of handling possible increase in mis-using their flag deletion will fall only on moderators...
    – gnat
    Commented Jul 12, 2023 at 13:43
  • 1
    ...really, repeat badges on comment flags are concerning
    – gnat
    Commented Jul 12, 2023 at 13:43
  • 4
    Yes, I agree with @gnat and Franck; awarding the badge for flags on comments, which are easily deleted with absolutely no independent oversight, is entirely too risky. Specifically, it risks incentivizing bad behavior. Posts do not have the visibility limitations that comments do, so I don't see the downside there.
    – Cody Gray - on strike
    Commented Jul 13, 2023 at 4:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .