Comparing tools for plagiarism detection in writing

Question

As a professor I’m planning to include more training about plagiarism in my courses to address some of the misconceptions and misunderstandings. What is your experience with such tools?

My university doesn’t have any tools, so I’m often left to my own devices:

“Hmm… that was a really good sentence. Wait, that was a really good sentence."
[Proceeds to copy and paste into Google and finds a match for the sentence.]

I'd like to use such a tool for scanning student assignments (probably only as a secondary scan when I think there are issues). If it has good reporting, I would use it as evidence for mandatory reporting of academic misconduct issues. The other case I might use it is for theses and dissertation drafts where it would be helpful to rely on something beyond my own attention and the conversations would be more formative and instructive before a final submission.

Because my university doesn't have anything set up, I'm think there are a few criteria: an easy workflow (would be standalone, not in the LMS), hopefully low cost, capable of scanning short or long documents. I might be missing other factors to consider.

Answer 1

At University of Michigan, we used the MOSS (measure of software similarity) system at Stanford to detect possible copying in the projects submitted by students in our introductory computer science courses. While not directly comparable to systems designed to work on essays like the ones offered by Turnitin, our experience may be helpful in setting expectations.

1. If you have lots of submissions to cross-check (or if, your case, you're hoping to cross-check with the entire web or a provider's database), you can't do it manually but automated systems do work. We used MOSS in classes with 1000 to 1100 students in 5 sections submitting projects that remained pretty much unchanged from semester to semester, inviting cheating. Over the years, we'd collected many thousands of submissions to be cross-compared. We typically reported about 6% of our students for academic violation, of which about 90% were found responsible. (Most common reason a student might be found not responsible was that their partner did it and they didn't know.)
2. These systems only detect possible copying. You still have to manually eyeball every single case and decide for yourself whether you find the evidence compelling.
3. It was lot of work feeding submissions to MOSS, getting it to run (it's cranky and doesn't always), sifting the results to find the ones you intend to report, and then writing up the reports for our Honor Council in a two-column format where they could see the copying side-by-side. I assigned one of my student staff to running MOSS, reviewing the results with me and then writing the reports we agreed on, but it was a big job and it usually took them a semester as an understudy to learn how to do it.

Answer 2

I’m planning to include more training about plagiarism in my courses to address some of the misconceptions and misunderstandings.

One of the most common misconceptions about plagiarism seen on this site is that software tools can tell you if there is plagiarism or not. Those tools measure text similarity. Plagiarism is not about text similarity. There can be plagiarism with zero text similarity.

Plagiarism is about taking other people's ideas without attribution. No software is able to compare ideas.

If you want to teach students about plagiarism, have them search real life examples for plagiarized ideas.

Answer 3

I hate to be a broken record and quote myself, but there is no magic bullet. I have been testing such software for many years. From our recent publication: "The sobering results show that although some systems can indeed help identify some plagiarized content, they clearly do not find all plagiarism and at times also identify non-plagiarized material as problematic."

https://link.springer.com/article/10.1186/s41239-020-00192-4