Let me back this up and look at the broader picture here. We're about to enter a time that Star Trek once hinted at, but is now here and ready for your use: artificially generated content. The Internet has now reached maturity and search engines can run as much training data down your throat as you can handle. Quite literally we're watching the true next iteration of the Internet be born right now. Google can show you what already exists. AIs can generate almost anything your mind can dream up.
The problem there is ownership. The US helped the Internet in its infancy by making an environment where you can "Fair Use" just about everything.
[O]nline intermediaries that host or republish speech are protected against a range of laws that might otherwise be used to hold them legally responsible for what others say and do.
If you post a meme on SE that's using a copyrighted image, all SE has to do is take it down to avoid liability. With such a low bar to clear, it's allowed Fair Use to thrive. But... what do we do when these AIs start generating content that is wholly based on the works of others?
Allen's victory prompted lively discussions on Twitter, Reddit, and the Midjourney Discord server about the nature of art and what it means to be an artist. Some commenters think human artistry is doomed thanks to AI and that all artists are destined to be replaced by machines. Others think art will evolve and adapt with new technologies that come along, citing synthesizers in music. It's a hot debate that Wired covered in July.
And will these tools drown out actual users?
Established artist communities are at a tough crossroads because they fear non-AI artwork getting drowned out by an unlimited supply of AI-generated art, and yet the tools have also become notably popular among some of their members.
These are from September, involving art communities, but all ChatGPT is doing is basically a fancier search than Google can serve up. And this problem isn't going to go away because we're using machine learning everywhere. If an AI can't do it now, just wait.
For my fellow mods and I on Stack Overflow, the root problem boils down to two issues
ChatGPT is a parrot
Parrots are very smart birds and they can mimic sounds very well. But parrots cannot talk. They emulate the sounds they hear but they do not comprehend what they're saying.
ChatGPT is better than any chatbot we've seen. It writes in natural language, not the stilted text that typifies such systems. It generates what appears at first blush to be quality content. But we've noted that ChatGPT is doing what a lot of inexperienced users on Stack Overflow do: try to be the best-sounding parrot. Someone asked ChatGPT if it should be allowed to answer Stack Overflow questions and posted it in a now-deleted answer on our rule. I do have to admit it's amusing
I am writing to express my extreme disapproval of the idea of allowing ChatGPT answers on Stack Overflow. This would be an irresponsible move and would fundamentally undermine the integrity of the platform.
Polly want a cracker? (fascinating English.SE etymology lesson there)
ChatGPT is not a write or programmer, it's just copying other smart-sounding sources that look highly relevant. It can (to its inventors' credit) write simple, passable code. But ChatGPT doesn't know what SQL injection is and another mod experimenting with it found it will merrily give you code suggestions using it. Why? For too long that was how a lot of people did it on the Internet, and you can still find that poor advice everywhere. Hence why a lot of folks on Stack Overflow will incessantly warn you about not doing that.
The various AI image generators can afford to be parrots. In fact, that's what the users want. "Paint me a picture of a cat riding a unicorn carrying a shotgun in the style of Vincent Van Gogh". That doesn't work so well when you need something like "How do I create a contact form on my Wordpress blog?" That's not something I would trust ChatGPT to answer. It might give you workable code that someone will use to gain control of said blog.
Dishonesty
I won't link the person who said this, but this is a real Twitter post
So I started a new stackoverflow account and I am plugging random questions without answers into https://chat.openai.com/chat and pasting the answer.
So far, after 9 answers in 1.5 hours it has 1 accepted and 3 upvotes and a reputation of 62...
And
6 accepted answers, of the 26, 11 upvotes, 5 downvotes. I am not checking the answers in any way. I'll give it a rest for now and see how we do tomorrow...😄
I'm sure ChatGPT (or Google if you wanna go old school) can tell you where to find this user, but that second statement is downright scary. They're not checking the answers in any way. In other words they also are not a programmer. I've already gone over why that part is a problem, but there's another one here: This user admits they didn't write the code. From the Meta Stack Overflow FAQ on non-English content
Translating a question for a non-English speaker sets them and all participants up for a poor experience, due to the OP not being able to follow and respond to feedback from comments, understand answers, or get assistance from the Help Center.
The inverse of that is true here. A user comes in and asks a question. Someone posts an AI-generated answer without understanding anything about the question. What if the questioner posts a comment asking a clarifying question? What if the AI did something confusing in their answer? This user cannot interact in a meaningful way with said questioner. And, going back to the art site debacle, who owns it if it's been copied? I mean, we don't use Creative Commons for no reason. This butts up against plagiarism in a most uncomfortable way.
TL;DR What do we do about it?
Answers generated by an AI should be considered as being written by the AI. That means you can quote them like any other source, but you must attribute them to the AI, just like any other source, and not use a bulk-copied AI answer quote as an answer. This way, we're avoiding the thorny issues of people running to the latest AI to get answers so they can copy-paste them as their own. We have plagiarism tools (current and forthcoming) in this wheelhouse so we don't need to reinvent any wheels.
If the AI gives a bad answer, we have votes for that.
