10

A new Internet phenomenon has reached our own corner of the Stack Exchange network. Today, we had for the first time an answer which we suspect has been generated by ChatGPT.

The network has been plagued by this kind of answers since the tool came out a few weeks ago. They tend to be at the same time convincingly written and of dubious veracity. That is, it is a coin toss as to whether they are right or wrong, but they come across as written by somebody who knows what they're talking about.

The main issues that stem from these answers are:

  • reduction of content quality (due to many wrong and/or vacuous answers)
  • interference with the vote usefulness (due to many misguided upvotes for impressively written wrong answers)
  • rep farming (due to the same upvotes and the ability to post large volumes of answers)
  • unmanageable increase in moderation load (only on SO so far)
  • copyright and plagiarism (the situation is quite muddy there)

Stackexchange-the-company has decided to leave it to each site to decide how to deal with these answers. Many sites, notably Stackoverflow itself, have decided to ban ChatGPT generated answers, but this is not a default policy network-wide.

What should we do with these answers, as a community?

9
  • 1
    I wouldn’t limit a policy to answers, but include questions and comments as well.
    – Stephie Mod
    Commented Dec 28, 2022 at 20:01
  • oh I just stumbled upon the answer and came here to ask about it
    – Luciano
    Commented Dec 29, 2022 at 9:16
  • That answer comes back as "According to Hugging Face, the chance that this post was generated by OpenAI is 0%."
    – DavidPostill
    Commented Dec 30, 2022 at 10:27
  • Are you sure that the Chatgpt terms and conditions require attribution? I see nothing of the sort on openai.com/terms , and on the contrary it seems that we are assigned all rights on its output.
    – Federico Poloni
    Commented Jan 1, 2023 at 12:21
  • @FedericoPoloni Kind of a moot point - it's not clear those rights are even OpenAI's to assign. The most restrictive licenses are the completely unknown licenses of its completely unknown inputs, and while it may generally sufficiently recombine and modify so that its output doesn't plagiarize or infringe, it also may copy important snippets verbatim. We really have no way to know if/when this happens, and even OpenAI wouldn't know unless they actively analyze for it.
    – Cascabel Mod
    Commented Jan 1, 2023 at 17:25
  • @Cascabel I was referring to the part in the question where OP writes "ChatGPT terms and conditions require an attribution to their bot". This statement seems dubious/false to me. I am aware that Chatgpt has other originality issues, but these are a different matter (and indeed they are mentioned separately by OP in the question). It is better not to use false claims to support otherwise valid points.
    – Federico Poloni
    Commented Jan 1, 2023 at 17:29
  • 1
    @FedericoPoloni interesting find. I remember reading that point somewhere on the Meta SE or Meta SO discussions, but I cannot find it again right now. The closest I could find in the actual T&C is that they forbid users to "represent that output from the Services was human-generated when it is not" which I take to mean that, if you post their output under your name, it would be a violation. But also, non-attributed use of foreign-produced text is a violation of SE's T&C. I also suspect that the whole is legally unclear - there have been lawsuits about AI inventors in ...
    – rumtscho Mod
    Commented Jan 1, 2023 at 17:59
  • ... patent law, and no clear judgements so far. I haven't heard about copyright lawsuits yet, but suspect that the legal machines will chew on it for years to come. So, I changed my wording to the much more vague "muddy" - it could be unsatisfying, but at least I'm not making confident claims in an area where I'm no expert!
    – rumtscho Mod
    Commented Jan 1, 2023 at 18:01
  • Speaking as someone who, as a non-lawyer, regularly deals with copyright lawyers at work -- I wouldn't base anything on expecting a legal decision about ChatGPT. The situation is, at best, complicated.
    – FuzzyChef
    Commented Jan 4, 2023 at 17:54

3 Answers 3

Reset to default
15

My suggestion is to follow the example of SO and ban chatGPT for the time being.

Apart from the reasons listed above (especially the risk of quality decline and that we have and enforce a clear policy against plagiarism), there have been enough twitter posts and similar, where users described how they used chatGPT to mass-generate content that they posted without even the slightest regard for quality, even without reading the tool’s output, in a clearly malicious intent. (Will add references later, I forgot to screenshot them.)

As long as we can’t estimate how this will ultimately impact our site and the network, and as long as there’s no final decision from SE (understandable), I would rather ban them now instead of opening the gates and then doing a major cleanup later.

Note that each individual post needs to be handled manually, and we don’t want to run into the same situation as SO. So rather nip it in the bud now, while most posts are still a “testing the waters”, limited by the thresholds keeping low-rep users from posting too many questions and answers.

10

I don't even think we needed to ask. We've always expected that people write their own answers, and that they answer in good faith, with information they are confident is correct. Machine-generated answers violate both of those principles.

The main reason there's not a network-wide ban is that staff is concerned about how enforceable it is. I share that concern, but it doesn't make any particular answer okay. If someone gets away with something, they get away with something, but that doesn't change the fact that it's against the rules and spirit of the site.

6

I'll give one more reason we should ban these:

ChatGPT answers suffer from the "gramatically correct nonsense" problem that often plagues text generators. Questions on cooking would be particularly prone to being gibberish, given that cooking answers are very dependent on an intuitive understanding of the physical world. There's a few "AI recipes" sites that are among the funniest blogs on the web.

The last thing SA needs is more low-quality answers.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .