With a featured post on Main Meta allowing sites to request warning banners about AI-generated content, I was wondering if Literature would implement that feature. However, I noticed a lack of posts discussing if we DO or DO NOT allow AI content in the first place...
So, what is our site's thoughts on allowing AI-generated content?
(Discussing answers only here.) I have seen one AI-generated answer on the site:
What kinds of drugs are mentioned and/or described in de Sade's stories? The answer (containing output from ChatGPT) was deleted, so only high-rep users can see it, but I spent a bit of time looking at the answer and can report on the experience. ChatGPT was able to produce a plausible-sounding answer to the question, in that it lists some of de Sade's works and asserts that drugs are mentioned in them, with plausible-sounding references including section numbers and page numbers. This is just what you'd expect an answer to look like, but on inspection nearly all of it was wrong.
Here I'll just quote the first paragraph of the ChatGPT material in the answer and list the mistakes, but the other paragraphs are of a similar quality.
"Justine" (1791): In this novel, opium is mentioned in several instances. For example, in Part Two, Section One (pages 93-94), the character Clairwil consumes opium to alleviate her suffering. Opium is also referenced in Part Two, Section Four (page 137), where another character experiences the effects of opium-induced dreams.
(i) ChatGPT gives page numbers, but any human would know that when a work has multiple editions or translations, the page number by itself is useless, because you don't know which edition it refers to. (ii) ChatGPT refers to "Part Two, Section One" and "Part Two, Section Four", but Justine is divided only into tomes ("books" or "volumes", but sometimes "parts" in English translation), and not further divided into sections, so these references are bogus. (iii) ChatGPT says that Justine has a character called Clairwil, but this character is from Juliette and does not appear in Justine. (iv) ChatGPT says that opium is mentioned several times in Justine, but the word does not appear once in the text. (v) Note the vague description "another character"—if this had been a real reference, it would have been just as easy to name the character. (vi) The actual answer to the question, easily discernable from the text of Justine, but not mentioned by ChatGPT, is "alcohol and caffeine", for example, "Il but douze bouteilles de vin, quatre de Bourgogne, en commençant, quatre de Champagne au rôti; le Tokai, le Mulseau, l'Hermitage et le Madère furent avalés au fruit. Il termina par deux bouteilles de liqueurs des Isles, et dix tasses de café." (1792 edition, tome 2, p. 96).
So in this case the ChatGPT answer had negative value: its claims were mostly incorrect, but presented in a superficially plausible way so that substantial effort was required to determine that they were false.
As long as this remains the state of the art of language models, my opinion is that AI-generated answers to literary questions are a waste of everyone's time, and we should reject them by policy.
However, if we adopt this policy, that does not necessarily mean that we need to also turn on the warning banner immediately. According to the meta post, the banner only applies to answers, and we are not currently receiving many AI-generated answers, so existing moderation mechanisms suffice for now. We can always turn on the banner at a later date if the situation changes.
(Discussing questions only here.) We have seen a few questions come in that consist either entirely or largely of "ChatGPT says X, is ChatGPT correct?" For example:
There was another such example recently but I don't remember it precisely.
It does not help askers to post questions that discuss at great length what ChatGPT has to say about the topic at hand. Instead of getting answers, they get downvotes. That is unwelcoming.
Nor does it help regular contributors to have to reiterate each time that the asker needs to revise the question to remove references to ChatGPT as they are a distraction.
So I propose that we have a custom close reason for questions. It could be worded something like:
This question relies on the output of an AI large language model (LLM) such as ChatGPT, Google Bard, or Bing Chat. These tools are designed to emit plausible language, not to provide factual information. Asking for verification or analysis of the output of LLMs is off-topic for Literature Stack Exchange. If you can ask your question without the LLM content, please edit it to do so. The community can then vote to reopen the question.
The custom close reason will carry more weight than the comments of individuals. I had a long exchange, frustrating to both parties, with the asker of the second question as he kept interpreting my statements about his question as value-judgments rather than statements of policy. To be fair, in the absence of a stated policy, they were my opinions. Adopting a custom close reason will make clear that this is not one individual's opinion, but our policy.
We might also want to update What types of questions should I avoid asking? to include some variation of this close reason.
AI generated content should be treated like any other content. If it's good you upvote it, if it's bad you downvote it, if it doesn't answer the question you delete it, if it's an off-topic question you close it.
AI might sometimes provide useful content, and will probably provide some wrong answers as well. So will regular users.
We should not police the tools that participants use to construct questions and answers. Some tools are better than others, but there are probably none that are perfect. We don't have rules that you can't use Wikipedia even though it might contain incorrect information; we don't have rules that you can't use a translation of a book that might be flawed; we don't have rules that you can't ask your friend a question and post their answer.
The only appropriate way to judge a post is by its content. An answer generated by AI might be a great answer. It shouldn't be deleted just because of its origin. A question generated by AI might be interesting, thought-provoking, and lead to answers that provide new insight on a topic. It shouldn't be deleted because it was created by AI.
Furthermore, there is usually no foolproof way to be sure if a post was generated by AI. By having a no-AI policy, we risk falsely identifying posts as AI content, and/or getting into protracted debates about a post's origins. Accusing users of such tactics may lead to unwanted drama, and discouragement of some participants.
As of now, we do not seem to have a problem of being inundated with heaps of bad AI posts. It is not such a big deal if there are a few posts here and there with AI-generated bad content. And if there is a particular user that continually posts such answers, they can be dealt with the same way any other user would be dealt with for similar infractions unrelated to AI.
Prohibiting AI generated answers sets a precedent in the realm of censorship. The site should be a place for free and open engagement with all people and all ideas that meet the site's scope. It is not a huge leap from deleting AI content to deleting content from sources we deem invalid, to deleting answers that contradict an author's statements, to eventually deleting anything we don't agree with.
