17

I recently was alerted that an answer had been submitted for one of my unanswered questions (archive link), which I asked a few months ago. I took a look, and found an answer which immediately struck me as being very similar in tone to the sorts of responses I got when I was playing around with ChatGPT just a couple of weeks ago. So, I inspected the answer a bit more closely, and noticed a few things:

  • the name for similar-looking kanji suggested in the answer is「形似文字」, a term which I could not find anywhere online,
  • the answer gives a reading of this compound which is incorrect (けいじぶんじ rather than the more likely けいじもじ), and
  • the answer goes off on a tangent about how I should practice writing kanji(!)

These are the three main things that lead me to believe that this is an AI-generated response.

Looking at the profile of the person who posted it, they recently submitted around ten answers like this in the short space of an hour or so. They all read like ChatGPT-generated responses, and indeed, it seems some others have noticed this as well.

So, please can we ban (and remove any existing) AI-generated responses? They seem to be little help and basically constitute spam in my opinion.

11
  • 1
    AI-generated answers are already banned on Stack Ovewflow (I don’t know if this automatically applies to this site): meta.stackoverflow.com/q/421831
    – naruto
    Commented Jan 2, 2023 at 6:02
  • @naruto That's an SO-only policy, so it doesn't apply here. AFAIK StackExchange hasn't enacted a site-wide policy on it yet.
    – user19642323
    Commented Jan 2, 2023 at 18:02
  • 3
    There's already a policy against plagiarism, and that's effectively what someone posting ChatGPT answers with no editing or referencing is doing. But I imagine SE should make a clear decision specifically on it.
    – Leebo
    Commented Jan 3, 2023 at 0:19
  • 1
    I would strongly advocate banning AI-generated answers in general, and especially any that are posted without any attribution or explanation that the answer is AI-generated, as in this answer from 2023-01-01. In addition to the problems introduced by AI-generated text that might read well but could also be wrong, as @Leebo notes, unattributed AI posts are essentially plagiarism.
    – Eiríkr Útlendi
    Commented Jan 3, 2023 at 17:30
  • 1
    I think I found the first instance of someone using ChatGPT as part of the basis of asking a question here. At least I think it's the first person to say they used it when researching a question. I would support designating questions that are based on information produced by AI as off-topic, since there's little reason to expect that the AI will be correct in any assertions it makes.
    – Leebo
    Commented Jan 9, 2023 at 13:36
  • 1
    @Leebo yes, I noticed that. I don't think it was the basis of them asking the question, though, but I find it interesting (and baffling) that people already trust it enough to use it as a resource...
    – user19642323
    Commented Jan 9, 2023 at 14:22
  • 1
    @Leebo I was thinking about that question earlier, and I imagined a situation where a user posted a question which was based on curiosity driven by false information that was not generated by ChatGPT. The current site rules wouldn't bar such a question (unless it fell under "caused by a simple typo"), but otherwise such a question is not necessarily that informative. Should the site rules be changed?
    – jogloran
    Commented Jan 10, 2023 at 1:05
  • While this is an important topic I'm almost beginning to suspect the user in question is not an AI, I've seen their types before who have weird preconceptions and refuse to budge
    – Angelos
    Commented Jan 14, 2023 at 15:29
  • 1
    @Angelos I think their recent answer is not AI-generated, though may perhaps be AI-assisted. As for whether they were using AI to generate the answers from 1 January: I am without doubt that these were AI-generated. There is overwhelming evidence for this; in particular, please see their activity across the network (page 2) - they were active in a discussion at StackOverflow meta, taking a rather vehement pro-AI/ChatGPT stance. Quote: "SO reaction to ban this marvelous technological breakthrough will only hasten SO's decline."
    – user19642323
    Commented Jan 14, 2023 at 16:08
  • 2
    @Angelos Some of the (now-deleted) answers contained bizarre "facts" invented out of whole cloth that seem unlikely to come from a human, even a human who is mistaken about Japanese in some ways. It's not that "they" are an AI, just that they seem to have definitely used AI for some answers.
    – Leebo
    Commented Jan 14, 2023 at 16:08
  • 2
    ...and if there wasn't already overwhelming evidence, they've just changed their name to 'ChatGPT'.
    – user19642323
    Commented Jan 15, 2023 at 10:36

1 Answer 1

Reset to default
7

The question

Are AI-generated answers banned?

Apparently not.

Comment discussion so far

Currently listed in the comments on the question post (as of the morning of 2023-01-04, US West Coast time, paraphrasing here):

  • As noted in the comments by @naruto, Stack Overflow now has a temporary policy banning answers generated by ChatGPT.
    This appears to target just that one chatbot, rather than AI in general. I'm not sure why the policy is so super-specific, when it can be difficult sometimes to tell which particular chatbot might have generated a given chunk of text.

  • A reply comment by @user19642323 explains that this policy only applies to Stack Overflow, with no group-wide StackExchange policy that we are yet aware of.
    Given the lack of anything for the SE group, each Stack is left to its own devices. We should come up with something for the JSE, especially now that we are getting quite a few low-quality (and sometimes flat-out wrong) posts that appear to be from a chatbot -- so far, almost everything from @Max Hodges.

  • @Leebo then points out that we do have a policy banning plagiarism, and anyone pasting ChatGPT content into an answer post as-is, and without explaining what it is and referencing the source, is essentially plagiarising the bot.

  • @I concur with the previous commenters.
    I'll add now that chatbot posts are also dead ends -- so far, I have not seen any replies from Max Hodges to any comments on their posts. Follow-up questions, pointers, or corrections are ignored. This is not conducive to good-quality information.

Proposal

The Japanese Stack Exchange should document a policy of banning AI-generated answers.

  • Copy-pastes of AI-generated content without any editing or proper attribution effectively constitute plagiarism.
    (Update: This is less of a concern from a legal standpoint, as content generated by AI systems generally doesn't fall under any copyright protections. However, this is a concern from a usability and usefulness standpoint, as a user posting AI-generated content as their own is misrepresenting their own degree of knowledge, and also impeding the questioners and other readers from directly utilizing that same AI system.)
  • AI-generated posts preclude any productive engagement with the poster.
  • AI-generated content often reads well, but contains inaccuracies (such as mistaking the intended sense of "explicit", or bad romanizations and confusion about verb conjugations), self-contradictions (had a good example last night, but it has since been deleted), or complete fabrications (such as inventing the word おた and proclaiming that it means "fun").

In addition, any user who posts AI-generated content should receive a warning, and if they persist, we should consider banning them. JSE is not intended to be a training ground for AI developers.

I hope this may be useful in prompting further discussion.

15
  • Personally, I am against banning AI-based contents on the grounds of plagiarism. Very few AI systems available now claim intellectual rights of their productions. There is usually no legal or ethical problem per se in copying AI's productions without explicit citation or permission.
    – naruto
    Commented Jan 5, 2023 at 8:37
  • 2
    @naruto I don't think the legal aspect is as important as the fact that it's against the spirit of stack exchange in some sense to post something as your own that you didn't create. People who want answers from AI can go ask AI themselves. Passing off AI answers as human answers is unhelpful to questioners.
    – Leebo
    Commented Jan 5, 2023 at 14:27
  • @naruto, thank you for bringing that up. I'll clarify the post -- my intention was more as Leebo describes it, in that it is unhelpful and misleading to questioners and other readers.
    – Eiríkr Útlendi
    Commented Jan 5, 2023 at 19:37
  • @Leebo SO's policy says nothing about plagiarism, and there's a good reason for that. My opinion is that we should ban AI-generated contents solely because they're unhelpful and harmful, not because they lead to plagiarism.
    – naruto
    Commented Jan 5, 2023 at 21:26
  • @naruto, if you search that page for the string "plagiar" (to allow for both noun and verb forms), it shows up quite a bit in the comments. This comment even explicitly references 'an existing "plagiarism" ban reason', although without linking to it, so apparently there must be a ban on posting copy-pasted content as one's own. That said, I don't care too much what we decide is a relevant justification for banning AI-generated content, just so long as we do create such a ban policy.
    – Eiríkr Útlendi
    Commented Jan 5, 2023 at 21:47
  • @EiríkrÚtlendi Yes, but most are unofficial comments. I found only one comment from a staff member that contains "plagiar" (this). He does mention the danger of accidentally plagiarizing existing (human-generated) material if you don't check the content, but he is not saying that using an AI-generated content in and of itself constitutes plagiarism.
    – naruto
    Commented Jan 5, 2023 at 22:12
  • Besides, I see no need to indiscriminately ban all sorts of AI-generated content. My answers do often contain sentences generated by DeepL, which is, technically speaking, very close to ChatGPT. This does not mean DeepL is mostly unhelpful or I was involved in plagiarism. For now, I think SO's policy of only targeting ChatGPT is the most reasonable.
    – naruto
    Commented Jan 5, 2023 at 22:13
  • @naruto: My concern is not that AI might have been used. My concern is when a post is mostly, or even entirely, generated by an AI. The bizarre おた post seems like one such example. When a whole post is AI-generated and the poster doesn't mention that, the poster is misrepresenting their own expertise -- and any attempt at conversing with them about the post is likely to fall on deaf ears, since they may in fact be ignorant about the actual content. This is bad for any Stack site.
    – Eiríkr Útlendi
    Commented Jan 5, 2023 at 22:19
  • I don't see any reason to specify any single text-generation AI. There are many of them (and more as time goes), and it can be hard to tell the output of one from another. This is why I don't agree with the SO policy's approach of only targeting ChatGPT. I agree with you that there is a place for AI-generated content. My intention with this policy is to come up with a reasonable approach to banning AI-generated posts, where the post itself is basically entirely copy-pasted content. A sentence here or there is no problem. An entire post would be.
    – Eiríkr Útlendi
    Commented Jan 5, 2023 at 22:24
  • 1
    It has been always possible to delete answers or suspend users based on the "extremely low quality posts" reason. There are actually several humans who was suspended for this, and our community has the right to (subjectively) decide which are low quality answers. So no new rule is necessary to suspend Max Hodges. In my opinion, if we are to add a new rule, it has to be minimally inclusive, and based on a perfectly correct reason. In SO's case, AI-based answers actually overwhelmed the ecosystem, so a new "temporary" policy was absolutely necessary. The same thing has not happened here (yet).
    – naruto
    Commented Jan 5, 2023 at 22:45
  • @naruto, you make sense. 😄 Shall we shelve this policy idea then? We can always revisit this in future if needed.
    – Eiríkr Útlendi
    Commented Jan 5, 2023 at 22:56
  • 1
    Well, I think it's still fine to specifically ban ChatGPT-based answers now, but the reason should be because they are almost always of low quality, not because they're related to plagiarism. And please don't stop me from using DeepL :)
    – naruto
    Commented Jan 5, 2023 at 23:13
  • @naruto Are you saying that if the AI-generated answers were high quality and correct you would be okay with people posting them as if they wrote them on their own? Just trying to fully understand your position. Reasonable people can disagree on it.
    – Leebo
    Commented Jan 6, 2023 at 2:23
  • @Leebo Basically yes, as long as the poster can take responsibility about the content. Like it or not, AI is already used as such in many business fields including art and professional translation. Except in exams and competitions where you yourself are supposed to be the sole author, of course.
    – naruto
    Commented Jan 6, 2023 at 2:32
  • 3
    @naruto I guess my image of Stack Exchange, where we have a reputation score attached to an individual, is closer to the end of the spectrum where the individual's authorship matters. I understand that a high reputation score does not equal trustworthiness or expertise on its own, but it still feels strange to me that someone with effectively no Japanese knowledge would be able to amass a trusted role on the site just by doing what anyone else with access to an AI chatbot could do. But if that's the way the site ends up going, I guess users who disagree can move elsewhere.
    – Leebo
    Commented Jan 7, 2023 at 4:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .