How to detect use of multiple AI-based tools by student?

Question

It blew my mind when I found students can use multiple AI tools to overcome the possibility of detecting the use of AI to answer assignments and test questions.

I was on a family visit and sitting with several guests. One of the people I distantly knew was doing their assignment. That personally was literally copying the question from the assignment and pasting it to ChatGPT to find an answer. After receiving an answer from chatGPT, the person again copied the answer from chatGPT and pasted it to the premium version of another AI tool to expand or shrink the length of the answer and change wording, tone, and such. After using 2-3 AI-based tools to write an answer, the person then uses another AI-based tool to plagiarism checking software to check plagiarism. After ensuring there is no plagiarism as suggested by AI tool, the person submitted an answer with scientific references. The references cited were actually available online when checked.

This gave me a scary picture of how academia might be operating. For me, it's not that students used AI to answer the question. The major concern is students may not know what they were answering about or what their answer actually means in the real world. What they would do if they had to intellectually think about the same or similar problem in the real world? If students could not solve similar problems in the real world, how that would change the reputation of the instructor and the university?

On the other side, are universities even doing a reasonable job of meeting the market needs and needs of students? If that student who uses AI succeeds in solving similar problems they currently solve using AI in the future, is the university doing a reasonable job of training students in the right direction?

Also, are these academic dishonesties detectable while grading assignments?

I was so confused and speechless seeing this.

Answer 1

A variation on the Flipped Classroom might help as a compensation since there are no currently viable, reliable detection methods. Like the other answers, so far, this doesn't answer the "detection" question, but is a suggestion to provoke changes in pedagogy to try to keep up with technological challenges.

In a flipped classroom all student (graded) work is done in face to face sessions with the professor and perhaps a few TAs present. Students "study" on their own time with materials provided: books, exercises, videos, whatever. During the meetings they do what was previously called "homework".

To prevent use of AI, you need to use either paper/pencil methods or computers isolated from the net. Both are hard to do. It has been noted that students at the present time aren't really taught how to write by hand, since keyboards have become ubiquitous. Remember Blue Books?

Flipping can be made to scale if TAs are well trained and given the materials to drive sessions. The professor isn't the only one involved.

Several other answers here suggest things like oral exams and discussion meetings. I agree. They can also be important components, provided you can scale them appropriately.

But I think, also, that even with a flipped classroom, more thought needs to be given to this issue.

Finally, I'll again emphasize, we give exercises to students to provoke thinking and learning, not to get answers. We can provide our own answers if we need them. Students don't often understand that. We grad things because students need feedback on their thoughts, not just to categorize them. If AI helps them learn, then I'm all for it. But if it only helps them provide "answers" in a painless way then it is worse than useless. Never mind plagiarism. It is the learning that is important.

---

A final thought. Imagine a "classroom" in which the "teaching" is done by an AI and the "students" are all just AI bots. Lots of "information" might fly about, but how much "learning" would be done?

Answer 2

I've heard of educators who are requiring writing be done in Google Docs and shared with them. They then check the revision history to ensure that each student actually typed the words and made revisions instead of pasting it all at once. It's probably still possible to cheat this system, but not without both planning and effort. Some will try. After all, there are plenty of memes out there about students who've copied a ChatGPT essay by hand onto paper (but forgot to remove the part where they state they're a large language model).

See Using Google Docs Version History for Good (and Evil).

Answer 3

This is something many, if not all, departments are thinking about. As of now, we cannot detect the (smart) use of AI. What I have seen is that many are rethinking the way they do their assignments.

My department is thinking again about what we want to teach our students and what not. This includes how to effectively use AI, and learn when they can and cannot rely on it. It takes time before that really permeates in all courses, and not all first attempts will be successful (and probably many second attempts will go wrong too). We have to learn how to handle this new situation.

If we want to test if students can do something or know something without the aid of AI, then we can no longer let them do it unsupervised. No more take-home exams, no more essay that can be written at home, etc. That is a pity, but that is the state of the world.

If the goal of the assignment is to give them experience and practice without AI, then we just have to communicate that with the students. They are adults and responsible for their own education. In those cases, the teacher gives feedback but no grades. If the student still uses AI in that case, they only hurt themselves. (Not quite, it also wastes the time I use to give feedback on AI generated stuff, but it is not a perfect world.)

If the goal of the assignment is to give them experience how to use AI, then there is obviously no problem with students using AI.

Answer 4

Make the essay compulsory but do not mark them (a simply adequate/inadequate mark), then decide your marking on an oral exam.

It works for very specific exams like n-dimensions manifold as well as for the first Algebra course. It is more work for the teacher, but there are solid reasons to have a teacher doing the teacher job and what you mention is a reputation problem for the university, the same university has to prevent it.

Too many students for a professor to have an oral exam with each one of them? This is again an university problem, it is time to hire more people and less manager to take care of their "clients".

Answer 5

The major concern is students may not know what they were answering about or what their answer actually means in the real world.

One method to find out whether that's the case is to have a discussion meeting for the assignment, in which the student needs to answer critical questions about their answers. At a minimum, the student must be able to exactly explain what they wrote, without looking at their answer. One could also require them to answer at least a few questions that are a bit to the right and left from the path of their argument.

Of course, the drawback with this is that it requires resources and might not scale nicely to large courses. One mitigation could involve TAs and having some "pre-filtering" through discussions between TAs and students, where only in critical cases students need to discuss with the main teacher.