9
$\begingroup$

We have a meta post about what questions are on-topic. It addresses the issue by what topics the questions address.

I believe we need to make it clear that questions and answers generated by AI are not allowed on this site. I also think there should be no more than one warning for doing this, and then the user should be banned from this site. AI has been very destructive of the information quality available online, and I want to nip it here as effectively as possible.

There may be other guidelines we'd like to see here, to improve the quality of questions asked.

Question 1: Should AI questions and answers be forbidden on this site?

Question 2: Are there any other guidelines you'd like to see added?

$\endgroup$
5
  • $\begingroup$ I would say that the main question is not whether or not A.I. generated answers are to be deleted. The main question is: is StackExchange working on a tool for detecting A.I. generated answers and is this done for all sites (StackOverflow, SuperUser, Math (educators), ...)? $\endgroup$
    – Dominique
    Commented Nov 28, 2023 at 10:26
  • 1
    $\begingroup$ Dominique, I hope @NotTheDr01ds can answer that question for you. But it is not "the main question". I am a volunteer, and have no say about what StackExchange does. What you and I can control is the guidelines of this group. Hence, my question. $\endgroup$
    – Sue VanHattum Mod
    Commented Nov 28, 2023 at 16:36
  • 2
    $\begingroup$ @Dominique I am a volunteer as well, but as of my last knowledge cutoff (an AI-answer joke) SE itself doesn't have any such tool currently. Moderators and community users are working with the SE staff on methods that we can use to reliably identify AI-content, but any such methods need to have a fairly high level of reliability so that we don't risk deleting actual answers ... (continuing in next comment) $\endgroup$
    – NotTheDr01ds
    Commented Nov 29, 2023 at 20:05
  • 1
    $\begingroup$ One of the main problems with AI is that it often looks so much like actual user-written content that it is difficult to detect. This conversely means that well-written human content can be (and worse, has been) mistaken for AI. SE Staff would like to achieve somewhere around a 98% accuracy level for removing AI content, meaning no more than 1 in 50 false positives. At present, there's no "automated" way of doing this that would come anywhere close to that for most SE answers. $\endgroup$
    – NotTheDr01ds
    Commented Nov 29, 2023 at 20:07
  • 1
    $\begingroup$ Adding on slightly, I'd say that many of the methods we use to detect AI content on Stack Overflow just won't work for some other sites - The questions are (of course) very different depending on the site, meaning the prompt that goes into ChatGPT (et. al.) is different, meaning the response that comes back is also quite different. Something that indicates a probably-AI answer on SO often won't even come up in ME answers. $\endgroup$
    – NotTheDr01ds
    Commented Nov 29, 2023 at 20:10

3 Answers 3

Reset to default
8
$\begingroup$

I'm not a regular here, and I just joined today to help @SueVanHattum with some AI-flagging of an answer here. Based on our chat, I suggested she post this, since there didn't seem to be a Community discussion on this topic.

I started out spotting some AI answers on Ask Ubuntu about a year back, but when I found out just how bad the problem was on Stack Overflow, I transitioned mostly over there. I'm part of a small community of users across the Stack Exchange sites who invest (way too much) time finding, proving (to a reasonable level), and flagging AI answers for the Moderators on sites to then verify and take action on.

There are several important facets to consider when crafting an AI policy for any site, and I'd like to share a bit about what other sites, especially Stack Overflow, have done, in hope that it helps here:

  • Is content that has been generated by an AI allowed without attribution?

    No, never. As @SueVanHattum (just) mentioned as I was typing this up, unattributed use of AI to answer a question is against Stack Exchange policy on any site, as it is considered plagiarism.

  • What if the user rewrites the AI content?

    I'm assuming it's quite obvious to most of the ME audience, but that's still a no-go. I'm assuming you'd still consider it plagiarism if a student rewrote something they read on Wikipedia, and the same rules would apply here. It can be tougher to prove, but the act of rewriting content that the user didn't come up with on their own (without attribution) would still be considered plagiarism.

  • Is content that has been generated by an AI allowed if the user properly attributes it to the LLM?

    • This is up to the community to decide. Many communities have decided that it is not acceptable. You can read some of the reasons in the Stack Overflow Meta Post, but I'd also add (especially for this community):

      • Users here, I believe, are typically looking for personal experience in answers. LLM's may be able to synthesize expertise on which it is trained, but never the experience that the community here can bring to the discussion.

      • If a user wanted an AI response, they could ask the LLM in the first place.

      • Reputation earned should reflect the user's experience, which does not happen when AI answers (which can often be mistaken for human writing) are upvoted.

  • Is a user-written answer that has been rewritten by an AI (for formatting or grammar purposes) allowed (either attributed or unattributed)?

    • Also up to the community to decide. Stack Overflow has decided that this is not acceptable. The reason is that multiple curators and moderators have observed AI's adding information that was not originally present when simply being asked to reword/rewrite the content. If you trust your users to review the rewrite closely and edit out anything they didn't write? Well, maybe. You'll need to decide.

    You (some members of the community) could also try asking ChatGPT (or other AI/LLM) to rewrite some of your own answers or popular answers here and see how the results look.

  • Is a user-written question that has been rewritten or aided by an AI (for formatting or grammar purposes) allowed (either attributed or unattributed)?

    • Typically, yes, although your community can decide otherwise. There seems to be less danger in this, than in rewriting an answer. There have been cases where this has been abused, but those situations are typically dealt with using other existing rules. I don't think you are going to face this issue here.

  • Once the policy is decided for your community, what happens when something is posted that you suspect may violate the AI-policy?

    • If you, as a community user, see something that you suspect may be written by AI, Flag it "In Need of Moderator Intervention". Provide as much information as you can. See this Meta.SE post for more information on how to write a helpful flag for AI content.

    • Since this is a smaller community with (hopefully) a small amount of AI content, I wouldn't expect the Moderators to keep up with the latest rules around how AI is handled (once identified). What happened today is what will likely happen in the future - A Moderator will join the Teacher's Lounge Chat (exclusive to Moderators), and if needed, they'll be referred to a separate chat where the "smaller community of users" I mentioned above can try to help out. Thanks to comments by other ME members on the answer under review, we were (relatively) easily able to verify that it was generated (in whole or in part) by an LLM. I have a suspicion it was not ChatGPT in this case, but rather Google Bard.

    • Once the Moderator has a reasonable level of verification that the content is AI-generated, they can delete the content, and warn the user about the policy. Future rule violation can result in escalating action, such as suspension from the site.

    • If the Moderator cannot validate that the post was AI, then no action will be taken on that post. Many Moderators will still mark a Flag "Helpful" in this case, as long as they believe the Flag was in Good Faith. The Moderators also have the ability to provide a short-reply when responding to the flag. E.g. "Helpful - But not enough to go on at the moment."

$\endgroup$
4
  • 2
    $\begingroup$ This is so helpful. Thank you. On "Is a user-written question that has been rewritten or aided by an AI (for formatting or grammar purposes) allowed (either attributed or unattributed)?" I would vote no, just so that someone can't claim some AI junk came from a question they had of their own. $\endgroup$
    – Sue VanHattum Mod
    Commented Nov 27, 2023 at 22:26
  • $\begingroup$ @SueVanHattum Understood - Just be aware that there are very few heuristics to "validate" that a question is AI generated. $\endgroup$
    – NotTheDr01ds
    Commented Nov 27, 2023 at 22:31
  • $\begingroup$ @NotTheDr01ds: is StackExchange working on a tool for detecting A.I. generated content and will this be used throughout all StackExchange or is this to be handled for every individual site (StackOverflow, Superuser, ...)? $\endgroup$
    – Dominique
    Commented Nov 29, 2023 at 11:38
  • 2
    $\begingroup$ @Dominique There is a group of moderators, community moderators, and users from AI Domination (an AI-focused SE chatroom associated to an SE sub-site on AI) working on LLM detection. This will likely never be automated, as the technology changes too quickly, but we are working on developing rough heuristics which outline when moderators can take action. $\endgroup$
    – Xander Henderson Mod
    Commented Nov 29, 2023 at 12:52
3
$\begingroup$

Note: It is already against the Stack Exchange Code of Conduct to post unattributed material created by AI.

Is uncited LLM usage considered a CoC violation under the Inauthentic Usage policy?

$\endgroup$
1
  • 2
    $\begingroup$ Good point, and I was just typing that up as part of my contribution. That saves me some time in not having to explain that part ;-) $\endgroup$
    – NotTheDr01ds
    Commented Nov 27, 2023 at 21:37
2
$\begingroup$

To be transparent, I'll preface this answer by stating that I'm an outsider to this site and community. Sue VanHattum (mod) suggested it might help if I posted giving some ideas, which is why I'm posting here. While I have experience working with and addressing LLM-generated content on the Stack Exchange network, I'm unfamiliar with the Math Educators SE site beyond taking the tour and briefly interacting with one of your (diamond) moderators in chat.

Consequently, these are ideas based on what I've observed on other sites, not based on your specific community. What works well on one site doesn't necessarily help another. I also don't have hard/conclusive data to give you that demonstrates these will work or help. It's also not guaranteed that even if you agree with the suggestion, Stack Exchange (the company) can or will do it for your site. Still, I hope these are helpful, but please ensure they meet the needs of your site.

Tour page warning

Diamond moderators could, after reaching consensus via Meta, edit your tour page to include a note such as the following (taken from Ask Ubuntu). You'd want to change the link and site name. I also don't know how many users actually read (and follow) the guidence in the tour.

Please also note our policy on AI generated content on Ask Ubuntu.

A dedicated help center page

Stack Overflow has a Help Center page explaining why LLMs (like ChatGPT) are prohibited. If y'all agreed on what to put there,you could request something similar. Similar to idea #1, this relies on people reading a page.

Answer box reminder

The above two options are more useful for people that "read the friendly manual", so to speak. Another option is that when a user uses the answer box, a warning is added to the answer box reminding them of the policy. See this Meta Stack Overflow post for what Stack Overflow did.

Featured Meta post

Some sites (Ask Ubuntu, Stack Overflow, and possibly others) have a featured Meta post (i.e., with the featured tag) reminding users about it. This means it appears in the right sidebar (or the bottom, on mobile). This also requires a diamond moderator to reapply the featured tag, as it will disappear after a month (30 days?).

I hope some of these help. Feel free to leave a comment here or ping me in chat with further questions or if you'd like clarifications.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .