Although I'm a moderator on two network sites who has had to deal with GenAI answers, I'm writing this from the perspective of a lower (<2k reputation, with limited access to moderation tools) user with expertise in the subject matter of the site in question.
I was recently checking on some recent answers to questions that I've answered across the network. I check in to see if new answers to the question have been posted, if an answer other than mine was accepted, vote (if necessary) on new answers, or leave comments on the question or any answers. I noticed that one question that I answered had a different accepted answer, so I read it. Several aspects of that answer stood out as suspicious, so I dug deeper.
Several aspects of the answer, although poorly formatted, had several hallmarks of GenAI posts. I don't want to get into the heuristics that I used here, since they were ones previously shared among moderators under the previous GenAI policy to determine if a post is suspicious. The first part of the answer was created using something like the question as a prompt and the second part of the answer using something like a comment left by the question asker.
When I probed more, I noticed that the user had been answering a lot of questions recently. I opened several of them and saw a pattern of characteristics that I tend to associate with GenAI. I also observed a distinct change in writing patterns from early 2021 and before. Although the user has a history of decent to good quality posts, there was a distinct change after tools like ChatGPT and Bard became available to the public. I did notice that there was a huge change in answer volume in late 2022 through today.
I used the Stack Exchange Data Explorer to investigate the change in answer volume. Prior to 2022, the user posted less than 10 answers in a day at a maximum. There were also days, weeks, and in one case, years, between posting. However, since the availability of GenAI tools, the user was posting several answers most days and peaked at nearly 40 answers in a day, with several days at over 20 answers per day.
Part of the reason that I, as an expert, use the SE platform, is to build a portfolio of answers that demonstrate my expertise. I've used this "portfolio" in more than a couple interviews in the past, to demonstrate my ability to write as well as my knowledge in relevant topics. Being able to demonstrate that my answers were up-voted and accepted goes to that demonstration of knowledge, expertise, and experiences.
But now, I am "competing" with GenAI answers. I have concerns as to what the follow-on effects will be. If GenAI posts are created quickly, and especially if they are up voted or accepted, will anyone bother to read and up vote later answers written by humans? Will new experts bother to follow up posts with new answers over time? There has been past discussion that reputation is seen as expertise - will this rep inflation for people who are not truly experts prevent experts from being seen, voted, and accepted? Will people who are using GenAI be wrongly seen as experts because they can write convincing answers?
Experiencing the GenAI problem from an expert and non-curator/non-moderator perspective has opened my eyes to a whole new side of the problem. Do experts want to participate in a venue where their answers get buried by GenAI? The thought crossed my mind, and if it's crossing my mind, it's probably crossing the minds of people who are even more expert than me who have better things to do with their time and knowledge.
If the communities on the network are to thrive, they need experts. And this expert has no desire to compete with GenAI. For me, this experience underscores the need to have strong policies and low thresholds for removing content that is likely to be produced by GenAI and ensure that the Stack Exchange network is a place for human experts to produce content that shows off their knowledge and expertise.
In the comments, Chindraba_on_strike brought up a good point about engagement that I wanted to elevate.
Engagement metrics are important for businesses that have platforms like the SE network. Driving away experts will have a negative impact on a lot of these engagement metrics. Consider what is likely to happen if GenAI answers are not handled quickly and decisively:
- Experts, like myself, who do not want to compete with GenAI answers leave the network. This reduces the number of people who can successfully perform actions like asking for the appropriate clarifications on questions, making assertions about the subjectivity or answerability of questions, vetting the usefulness of questions to the general population and voting on them, and vetting answers and voting on them.
- As experts leave, a number of engagement metrics go down. Experts who regularly visit stop visiting, so page view and active user metrics go down. Since experts provide answers, new answers are likely to decrease. Depending on the volume of questions, the recurring user metrics also go down.
- As experts leave, the "learner" class also leaves. If getting answers on the network is no better than getting answers from ChatGPT, Bard, Copilot or other Generative AI solutions, why shouldn't the people with questions just ask those things directly? Page view and user metrics continue to drop. New question metrics drop. As both questions and answers drop, votes also drop since there's less to vote on.
- A continued cycle of fewer questions to keep experts engaged and fewer experts continue to decrease engagement until communities are dead.
Holding on to experts should be a primary focus, since catering to that class of user is paramount to keeping other classes of users active and engaged.
