AI-generated answers are no longer allowed on Writing.SE

Question

Recently, AI-generated answers have become a major problem across the Stack Exchange network, and Writing.SE is no exception. While it's been great to see the site become more active again over the past few weeks, this has unfortunately also led to a number of users posting AI-generated answers. This is a problem, for a number of reasons:

• For starters, coming to a site about writing and posting things that you didn't write is (IMHO) audacious beyond belief. It's also plagiarism - and it's not simply a matter of crediting the AI, because AIs don't actually write things, they just mash together bits of text that other people have written.
• While AI-generated answers usually look correct, they may not actually be correct. This carries the obvious risk that people viewing the answers might not understand what's wrong with them either and blindly follow their advice.
• Similarly, the people posting the answers might not know if they're correct or not -- for example, because they lack the subject matter knowledge to write an answer without using AI. This isn't really fair to those who do have the subject matter knowledge to answer, and leaves them unable to correct their answers even if any problems are pointed out to them.

As a result, my fellow moderators and I have decided that all AI-generated answers are banned on Writing.SE. We have already deleted several such answers, and will continue to do so. If you see any answers you suspect are AI-generated, flag them and we'll look into it.

Answer 1

I support the policy, but I think there is something missing from the reasoning.

As it stands, the reasons given apply much more generally. Basically any text you copy from another website is text you did not write, that might have been copied from yet somewhere else (including an AI), may be wrong, and you might not recognize it as wrong.

As an additional reason for banning AI, I would add that it allows people to do all of the above much too easily. It's become a problem because effort and cost is no longer a barrier. If it was still difficult to get an AI to generate answers, then it would happen so rarely that a ban wouldn't be necessary.

(As an aside, I think it's a mischaracterization to say AIs "just mash together bits of text that other people have written". That is not how they work. They don't pick and choose words and phrases from the sources they've been trained on and then mash it together into a response.

To oversimplify it, an AI model only contains probabilities of words given a context. So the text-generation process is like having a monkey hitting a typewriter, and all the keys of the typewriter have different sizes so some are more likely to be hit than others. And each time the monkey hits a key the key-sizes change to reflect the new probabilities.)

In any case, I hope that simply instating the ban will help deter most people from breaking it. But I do have some worries that if the ban needs to be enforced it can lead to false accusations against which its hard to defend yourself. There are no fool-proof tools yet for distinguishing human from AI-generated text. And that distinction will just become harder in the future unless AI companies build in some sort of watermark.