9

Staff recently completed a test of a banner to inform users of a site's policy regarding the use of generative AI, finding that it doesn't significantly impact answer rates but that it does reduce the number of moderator flags for AI-generated content. As a result of the test, this feature is now available for all sites in the network as an optional feature.

The network-wide policy is that content created with generative AI must be cited as such. Software Engineering's current stance is that the use of generative AI to create content is prohibited, even with citation.

This brings us to two questions:

  1. Is our current policy of removing all generative AI content, even when cited, still the desired wish of the community?
  2. Do we want a banner informing users of our policy? The banner choice will depend on the answer to question 1.
8
  • Are there any statistics on the number of posts with AI content that gets removed?
    – Bart van Ingen Schenau
    Commented Jan 5 at 13:01
  • 1
    @BartvanIngenSchenau No. It's not a flag type. In my experience, it's a mix of custom flags, not an answer flags, and abusive flags. I don't know a good way to get the number of flags. And that would exclude anything handled on sight without a flag. I will say it's not immensely high - we don't have a ton of flags to begin with.
    – Thomas Owens Mod
    Commented Jan 5 at 14:47
  • As long as the content is cited I do not see an issue with that. I think we allow AI content and that it is cited and then allow this banner mentioning this.
    – JonH
    Commented Jan 5 at 18:05
  • 2
    @JonH You should bring this to an answer. However, as of right now, we do not allow any AI generated content here. Even if cited, it will be deleted. This is a good opportunity to explain why you think that allowing AI content should be permitted.
    – Thomas Owens Mod
    Commented Jan 5 at 21:34
  • Did you notice: you asked two questions in one, where both could be answered independently with yes or no. That takes voters the option of signaling agreement or disagreement individually (or we need four different answers to vote on for/against AI content, for/against a banner). Unfortunately, since you have added only one answer which collected certain votes, we cannot find out any more if those votes are for the AI policy or for the banner or both.
    – Doc Brown
    Commented Jan 10 at 6:12
  • 1
    @DocBrown Yes, but I don't see how that matters. There are only four possible answers to this question: yes/banner, no/banner, yes/no banner, no/no banner. The answer to question 1 dictates which banner is applied. I would interpret any up vote on my answer to be in support of yes/banner and a down vote to be against yes/banner and ideally someone would propose one of the remaining three options.
    – Thomas Owens Mod
    Commented Jan 10 at 12:39
  • 2
    @ThomasOwens: fair enough. Since you asked "Is our current policy of removing all generative AI content, even when cited, still the desired wish of the community?", I was wondering how far this rule of "removing all generative AI content" should go. What about content where only phrasing and grammar was improved? I asked this question on Meta.SE, but from the reactions there is seems clear this can only be answered individually on each SE site. I think it might worth trying to discuss this here.
    – Doc Brown
    Commented Jan 10 at 14:34
  • ... @ThomasOwens: or what is your opinion on machine translations by tools like "DeepL" or Google Translate? Today, they are surely AI tools Does their output count as "generative AI content"? There is this Meta.SO question, where the top answer says "no", but does our community agree on this?
    – Doc Brown
    Commented Jan 11 at 12:45

4 Answers 4

Reset to default
2

The banner for no AI content being allowed on Software engineering has been enabled. Apologies for the long turnaround time.

12

Is our current policy of removing all generative AI content, even when cited, still the desired wish of the community?

I believe the current policy that prohibits all Generative AI content is still the best policy. Moderators, other experts, and representatives of the company continue to refine heuristics and guidance for detecting and removing such content, as outlined in this Meta Stack Exchange post.

Given that many questions on Software Engineering are slightly more subjective than on other sites and there are no hard right or wrong answers, it's imperative that answers are based on the experiences of the people writing them. There are also known risks with allowing AI to be trained on AI-generated content, and as Stack Exchange data is open source, it's likely to continue to end up in training material for a very long time. From a purely quality standpoint - both ensuring that users can fully trust our content to ensuring the quality of the data as it may be used for other purposes - removing generated content is the best course of action.

Do we want a banner informing users of our policy? The banner choice will depend on the answer to question 1.

Aboslutely yes. Although regardless of if we allow cited AI content or not, the banner is helpful to make sure that contributors are aware of our expectations.

3

Let me first say I agree to all the arguments presented in Thomas Owen's post. Question's and answers on this site should be based on people's experience. However, I think we need to be a little more specific about what we mean by AI generated content, by based on and how much space this leaves for an acceptable and responsible use of AI. The current version of the AI policy on Stackoverflow is not specific enough for my taste, so as long as this policies wording does not get an update, I think we should not use this as a template.

To my understanding, "based on an author's experience" does not necessarily mean "literally written word by word by the post author". For example, machine translation services like Google Translate or DeepL are using artificial neural networks today. Hence translations generated by them can be seen as "AI generated content". Still, I think that is a perfectly acceptable use of AI, as long as authors give their own ideas as input, and proofread the result to validate the translation is accurate. Of course, I expect a user of such a service to be able to understand English to the degree they can reliably do the latter. This is in line with this Q&A on Meta.Stackoverflow.

Another case would be the usage of an LLM (and not just an arbitrary AI) for improving a post linguistically, in terms of phrasing, grammar and spelling. A few days ago, I asked a question on Meta.SE about this. My conclusion from that Q&A is that it is not a good idea to run a post through an LLM and copy/pasting the result right here - that would definitely count as "prohibited AI generated content", and it should be prohibited for good reasons. If, however, one interprets the AI generated (or "improved") text as a list of possible suggestions for changes, and then goes through that list one-by-one, picking the suggestions which really look as an improvement, whilst ignoring the others, that would be IMHO ok.

Still, I am a little bit unsure at what point such a post should be marked with a sentence "this post was written by the OP and linguistically improved using ChatGPT". This is surely a grey area and may have to be decided on a by-case basis. On the other hand, if one feels having to add this sentence to a post, they probably have already gone too far with using the LLM.

About the banner: in general, I think that's a good idea (as long as we don't get too many of such banners at the same time). Ideally, it should contain a link to a page which describes our expectations and what we mean (or not mean) precisely by "prohibited generative AI".

However, thinking a while over the banner, I am not convinced this banner will be more important than the other banner we often see as a reminder to the CoC when answering to a new users. If I understood this right, only one of the two banners will be shown, so we have to choose which one we will prefer over the other.

Looking at the current AI policy page of Stackoverflow (which is is the link on the banner on Stackoverflow), what I don't like about it is that it talks about generative AI in general, but then only mentions LLMs as example and is completely silent about other kind of generative AI in the broad sense, like translation services. In case we get a banner, I hope our policy page will be less vague.

4
  • 2
    I've been thinking about it. There are heuristics being developed by staff, moderators, and a small number of non-staff/non-mod experts. Looking through the heuristics and thinking about this, I'm not seeing many that would be triggered by a translation tool or someone using a tool (ranging from Microsoft Editor or Grammarly to something more powerful like ChatGPT or Bard) to improve their posts. These heuristics are designed to catch people taking all or large parts of generated content verbatim.
    – Thomas Owens Mod
    Commented Jan 14 at 19:14
  • @ThomasOwens: In between, I think I am not convinced the "AI banner" is really more important than the "new users" banner. I think from your moderator's POV you have a probably better experience how many answers you had to close or delete because they violated the CoC, and how many because they violated the AI ban. So I think the decision should be made on what you think happens more frequently.
    – Doc Brown
    Commented Jan 28 at 12:36
  • I can't speak for the other mods, but I've deleted very few things for CoC violations. In fact, I tend to decline more flags for things like rudeness. I tend to use a measure of what I consider professional - things said in an office, for example in Slack or Teams, emails, code reviews, code comments. There are some blunt comments, but very few get deleted for rudeness. The AI posting also appears to have dropped off, but I am more worried about that than CoC violations.
    – Thomas Owens Mod
    Commented Jan 28 at 12:51
  • @ThomasOwens: "I am more worried about that than CoC violations" - there is your answer.
    – Doc Brown
    Commented Jan 28 at 12:54
1

Seems like only two banners are offered, neither matches the current policy.

  • Reminder: Answers generated by artificial intelligence tools are not allowed on [Site Name]. Learn more

  • Reminder: Answers generated by artificial intelligence tools must be cited on [Site Name]. Learn more

I suggest the policy is modified to allow a degree of flexibility and human judgement.

"Answers must be primarily/substantively/mainly? written by the author"

This allows translations, or examples, or answers about Generative AI where deemed appropriate by humans.

Using the banner seems pointless, will it really stop someone writing a bot that answers questions and is deceptive about it? If a poster is citing their answer as AI generated then it can be deleted easily same as any other rules violation.

I can see why you might want to use a bot to hack a massive SO score, but a SE score?

EDIT -

Example of answers which I think should be allowed, even though they have used GenAI

Questions About AI

Q. How can I stop extra fingers being added to my GAI generated images?

A. you can improve your model with X, here is a before example [generated pic] and after example [generated pic2]

Here the answer includes GAI generated content but the answer has been created by a human and is useful.

Code examples written by humans with GAI help

Q. Here is an example of a big If statement, how can I refactor more betterer?

A. Big if statements can be replaced with blah [example written by human using copilot] here are the pros and cons etc etc

Again, here the answer is human written but contains GAI content

GenAI Translation tools used to make answer

Q. in this german paper on software development it says X, but why not Y?

A. on page six it goes into X vs Y and says that [google translated excerpt] ...

Again, human written answer, but made with the help of GAI

27
  • 1
    Our current policy is to delete all answers that match enough of the heuristics for generative AI. These policies and heuristics already exclude most, if not all, translations, since that's not generating new content. And nothing is said about examples of generative AI used to support existing answers, since that is not an answer itself. It should also be pointed out that SE has reviewed and all sites will be getting a banner. So it becomes a matter of which banner, and the only banner that matches our policy is the first one stating that generated answers are not allowed.
    – Thomas Owens Mod
    Commented Feb 3 at 12:48
  • I guess i was reading the summary in your question "the use of generative AI to create content is prohibited" rather than the linked "posting machine-generated answers is highly discouraged". I'm imagining an answer such as, "blah blah an example of machine generated content which proves this point might be : "machine generated content"", or the kind of things doc is talking about eg "....or as they say in french 'ce la vie!' (google translate used)" where the answer is human written, but contains machine generated parts
    – Ewan
    Commented Feb 3 at 13:07
  • A key thing in future might be the use of copilot or similar tools to write example code which is then included in answers. I don't think anyone would want to exclude that kind of thing, it's just better autocomplete
    – Ewan
    Commented Feb 3 at 13:11
  • There's absolutely nothing wrong with using translation, grammar editing tools, and things that help a human write better content. No reputable tool that performs these functions should fall under any of the currently valid heuristics to detect generated content. Copilot - when used to generate large amounts of code - does fall into prohibited content, though. The biggest issue with generative AI is that it is incredibly easy to produce garbage. (1/2)
    – Thomas Owens Mod
    Commented Feb 3 at 13:16
  • Moderators and users have consistently, across the Network, rejected GenAI because it floods the site with content that needs to be closely curated and can't easily be rejected. When content, on the surface, looks good and correct, but has subtle flaws, it's much easier to detect the class of content and reject it. That approach is less harmful. It reduces the risk of people being mislead by low quality or incorrect content and is more respectful of the time of the unpaid contributors and moderators since it's faster to detect generative AI than to validate the flood of generative AI content.
    – Thomas Owens Mod
    Commented Feb 3 at 13:18
  • I don't think anyone disagrees with that, it's just wording the policy to allow (for example) use of copilot/autocomplete in human written example code, but not 100%/90%/80%? generated answers. If you just say "No AI content!" well there is spell check! if you say "NO AI answers" well I put 'The AI says this:' at the start of the answer! You want something short and sweet which allows some flexibilty
    – Ewan
    Commented Feb 3 at 13:22
  • But the policy doesn't allow for Copilot. And there are fundamental differences between spell check or translators and Generative AI. We aren't saying "no AI". We are saying "no Generative AI". And even if you attribute AI, that's not permitted here.
    – Thomas Owens Mod
    Commented Feb 3 at 13:47
  • sorry, which policy doesn't allow copilot?
    – Ewan
    Commented Feb 3 at 14:22
  • 1
    This is our policy for Software Engineering. It prohibits the use of generative AI. Single instances of using generative AI will result in deletion. Abuse of generative AI - such as continuing to post generative AI after being warned or flooding the site with generative AI content quickly - is ground for suspension.. (1/2)
    – Thomas Owens Mod
    Commented Feb 3 at 14:33
  • 1
    We have a set of heuristics, created by staff, moderators, and other select users used to detect Generative AI. Any post that meets sufficient heuristics can be deleted. Having multiple posts deleted for matching these heuristics warrants a suspension.
    – Thomas Owens Mod
    Commented Feb 3 at 14:34
  • 1
    What i'm saying is we need to have a clear policy which allows those type of examples while banning the things you listed. I don't think the language in your linked meta question, or the language used in this question is good enough. Which I why I have made the suggested wording here
    – Ewan
    Commented Feb 3 at 14:44
  • 1
    I want more precision. and have given a suggestion.
    – Ewan
    Commented Feb 3 at 14:49
  • 1
    TBH I think in the near future all coding tools will have GenAI autocomplete built in. which means all the little example code snippets in answers will be generated using genAI, the policy should be written with that kind of thing in mind
    – Ewan
    Commented Feb 3 at 15:23
  • 1
    There is no "primarily/substantively/mainly". There are heuristics for code and heuristics for English. If you make a post - a question or an answer - and sufficient heuristics for a strong enough claim that it is generated content, it gets removed. It doesn't matter if all those markers are in 1 paragraph out of 8 or spread across the whole post. If you keep posting generated content, you get suspended. The heuristics are designed to allow the use of spelling, grammar, translation tools to help people craft higher quality answers while prevent the use of tools that pose risks to users.
    – Thomas Owens Mod
    Commented Feb 3 at 16:38
  • 1
    What "machine"? The heuristics are applied by people. If enough of the heuristics match, the post gets deleted. If people claim otherwise, that will be the answer - it looks so much like generated content that it is likely to be generated content and doesn't belong here.
    – Thomas Owens Mod
    Commented Feb 3 at 23:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .