10

On StackOverflow, the use of ChatGPT is not allowed.

I was wondering if we shouldn't do the same here.

I came across these and although the text is nicely written, I would think a simple "yes" would be the same.

Generic answer 1? Generic answer 2? (user is removed by mod and answers aren't visible anymore)

4
  • 2
    How would we determine if the text is human generated or AI generated?.. Poor answers are already downvoted and removed all the time.
    – JonathanReez Mod
    Commented Dec 12, 2022 at 14:32
  • That being said I’ve removed the user in question as their answers did seem computer generated. But this is a problem that existed long before ChatGPT.
    – JonathanReez Mod
    Commented Dec 12, 2022 at 14:34
  • 3
    @JonathanReez The problem becomes worse with ChatGPT because the answers look closer to what a human would write. But it's not passing the Turing Test quite yet :)
    – gerrit
    Commented Dec 12, 2022 at 17:05
  • 1
    There is a discussion about a site-wide ban meta.stackexchange.com/questions/384396/…
    – mdewey
    Commented Dec 13, 2022 at 13:08

2 Answers 2

Reset to default
15

We should not normally allow ChatGPT answers.

According to the terms and conditions users sign up to before they can use ChatGPT, OpenAI requires that text generated by ChatGPT is marked as being written by ChatGPT, so if it's not marked, we can't keep it in any case.

I have a hard time thinking of any circumstances in which a ChatGPT-generated answer would be a good answer, but in those rare cases, it should at the very least be clearly marked as such.

10
  • OpenAI is unlikely to own copyright on text generated by ChatGPT as modern laws are commonly interpreted to state that machine generated text is not subject to copyright. So what they have to say on this matter isn’t legally binding, at least in the US.
    – JonathanReez Mod
    Commented Dec 12, 2022 at 17:06
  • 2
    That being said, even if OpenAI does own the copyright, there is currently no feasible way to prove that the text in question is AI generated, unless you know the exact prompt used to derive it.
    – JonathanReez Mod
    Commented Dec 12, 2022 at 17:08
  • 5
    @JonathanReez It's not a matter of copyright, it's a matter of the terms and conditions that users sign up to when they use ChatGPT. Apart from that, claiming a ChatGPT text as ones own is plagiarism.
    – gerrit
    Commented Dec 12, 2022 at 17:50
  • Terms and conditions are only binding to the person who accepted them. They cannot be used to prevent the distribution of public domain works by other individuals. There’s existing case law on this from a museum that asked viewers not to distribute their photos of 19th century art work - a court has ruled that uploading these works to Wikimedia is still legal. OpenAI may of course sue the person in question but they can’t sue SE.
    – JonathanReez Mod
    Commented Dec 12, 2022 at 17:57
  • As for plagiarism - AFAIK there’s nothing in the terms of SE that forbids plagiarism of public domain works. Attribution is certainly a good practice but I’m not able to find anything that mandates it for works not protected by copyright.
    – JonathanReez Mod
    Commented Dec 12, 2022 at 17:59
  • 6
    @JonathanReez: Per the Help Center page "How to reference material written by others", plagiarism is defined as "posting the work of others with no indication that it is not your own". Plagiarism of public-domain works may not be copyright infringement, but it's still considered plagiarism if you're posting others' work without indicating that you didn't write it yourself. The SE network's plagiarism policies are not dependent on copyright. (See also: this FAQ on Meta Stack Exchange.)
    – V2Blast
    Commented Dec 12, 2022 at 19:04
  • (That said, the sources that ChatGPT-generated posts are drawn from can be hard to determine, so it can be tricky to determine whether certain parts of such a post are plagiarized. And of course, I don't personally know how frequently ChatGPT-generated posts are being posted on this site.)
    – V2Blast
    Commented Dec 12, 2022 at 19:09
  • @V2Blast By plagiarizing, you steal someone else's work and pass it as your own (from the Meta.SE link) => my reading of this is that it's bad to steal from humans. I.e. it's not expected that people using Google Translate to help them translate their answers provide an attribution to Google. Plagiarism is normally bad because you don't give credit to the person who deserves it. In this case there is no "person", so there's no one to "steal" from. Of course, SE is free to clarify their T&Cs but IMO the current text is a bit vague.
    – JonathanReez Mod
    Commented Dec 12, 2022 at 19:11
  • @JonathanReez: I think whether copying/pasting ChatGPT-generated content itself is plagiarism is a bit murky, but I was talking more about the fact that ChatGPT is trained on certain content, so it may end up reproducing portions of that content when generating text – and it doesn't attribute the sources it's drawing from when it does so. As a result, portions of the generated text may effectively be plagiarized from other sources. Anyway, my point was just that copyright infringement and plagiarism are separate (albeit somewhat related) issues.
    – V2Blast
    Commented Dec 12, 2022 at 19:55
  • @V2Blast well, humans are trained on human-generated text too and end up accidentally reproducing it all the time. The lawsuit for the "My Sweet Lord" song is a classic example of this happening. Based on my understanding it is exceedingly unlikely for more than 1 sentence to be plagiarized by ChatGPT verbatim.
    – JonathanReez Mod
    Commented Dec 12, 2022 at 19:59
7

I have been looking for information on ChatGPT and what I found makes me think that all answers generated in this way will be poor in quality and likely full of mistakes.

I was pointed to this answer and I think we now have to find ways to recognize and delete these answers.

11
  • 3
    The answers from ChatGPT are actually excellent and often without mistakes. That also means it’s much harder to figure out what was written by humans and what was written by AI.
    – JonathanReez Mod
    Commented Dec 12, 2022 at 17:22
  • 3
    @JonathanReez, do you mean without language mistakes or without mistakes in the information they hold?
    – Willeke Mod
    Commented Dec 12, 2022 at 17:44
  • 1
    Both. It’s both factually correct (not on all topics) and able to write coherent human-like text. You can try it for free yourself.
    – JonathanReez Mod
    Commented Dec 12, 2022 at 17:53
  • 2
    @JonathanReez, it is possible that answers by a bot work out to be without mistakes, the samples I have seen in the discussion show that the bot does include many mistakes, as they do not understand the words they string together. And as such we do not want that kind of answers.
    – Willeke Mod
    Commented Dec 12, 2022 at 19:03
  • 3
    @JonathanReez I did try it myself and in my experience, more often than not, it's factually incorrect. It might depend on what questions one asks, of course.
    – gerrit
    Commented Dec 13, 2022 at 7:15
  • 4
    @JonathanReez It's factually correct sometimes. Quite a lot when asked for general knowledge or trivially-looked up information. Once a question needs a bit more research, it becomes more of a lottery. Ask how to get from an airport to the centre of the nearby city and it'll mostly do fine. Ask it how to get from Manchester Airport to Leeds? I tried and it confidently informed me that Manchester airport is about 27 miles south-east of Leeds (it's actually about 55 miles south-west of Leeds) then suggested I take the X6 bus line from Manchester to Leeds. Which doesn't exist.
    – Chris H
    Commented Dec 14, 2022 at 9:22
  • 6
    Asking how to get from the Fernsehturm to Jungfernstieg (intentionally choosing places that could be either Berlin or Hamburg - Berlin's Fernsehturm is more well-known than Hamburg's, Hamburg's Jungfernstieg better known than Berlin's), it suggested taking the U2 towards Pankow. So far we seem to be in Berlin. Except then we're supposed to change at Heinrich-Heine-Straße (still Berlin, though the U2 doesn't stop there) and take an U3 in the direction of Uhlandstraße. Berlin has an Uhlandstr., but the U3 doesn't go there.
    – Chris H
    Commented Dec 14, 2022 at 9:35
  • 5
    now, Hamburg's U3 does connect Uhlandstraße with Jungfernstieg. There are parts of this answer that make sense. But that's the issue. When faced with an ambiguous question, rather than saying "I don't know" it confidently gives directions which have a hint of reality to them but can't be followed. If somebody asked that because they were near the Berlin Fernsehturm and wanted to visit their friend in Jungfernstieg (Berlin), there's no obvious indicator that it's wrong unless that person just happens to know the Berlin U-Bahn well enough that they wouldn't have to ask in the first place.
    – Chris H
    Commented Dec 14, 2022 at 9:39
  • 1
    @ChrisH There's a very good reason for that. GPT doesn't know anything except which words go together. It knows that "Fernsehturm" is associated with "U2" and "U2" with "Pankow," one can "change trains" at "Heinrich-Heine-Str." and take the "U3" to "Jungfernstieg." But it doesn't know what any of that means, and thus it can't possibly hope to synthesize real instructions out of those connections.
    – OpenAI was the last straw
    Commented Dec 15, 2022 at 20:35
  • 3
    @AndrewRay I know that - Jonathan is the trying to argue ChatGPT might be of value and provide factually correct answers here (“actually excellent and often without mistakes”) I’m providing demonstration of how it gets confused as soon as it has to do anything other than regurgitate commonly known facts or trivially looked up information.
    – Chris H
    Commented Dec 16, 2022 at 7:01
  • 3
    @ChrisH I'm not trying to disagree with you. You have demonstrated that it can't coherently synthesize information, and I'm trying to explain why it's not capable of synthesis and never will be. It's a fundamental limitation of the model.
    – OpenAI was the last straw
    Commented Dec 16, 2022 at 12:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .