A teacher shares signs that a job is written by AI.
Mark Massaro has been teaching English Composition for years at the but in 2023 his task became much more difficult. Shortly after AI apps like ChatGPT appeared and became available, higher education in the US was hit by a tsunami of automated deception. Students have been using AI to write their papers, and the trend seems to be worsening over time. Massaro said that out of 25 students in his classes, it is not unusual for at least five to submit papers that seem to be written with AI.
When ChatGPT was launched and it became apparent that students were using it to cheat, Massaro used to use different AI scanners daily for many of the papers. If the majority indicated that AI had been used, he believed it. But now it is not allowed to scan students’ papers in that way under the argument that their privacy is being violated. In addition, the accuracy of AI detectors.
Massaro says he now has to rely on his own ingenuity to assess whether the work is legitimate or not. And to do so, he created a list of signals that reveal if the work is AI-generated. When he is given a paper, he consults that list. He shared with Gizmodo some of the signals, painting a clearer picture of the hellish conditions that AI has imposed on educators in the US.
Too many hyphens
An indication that a paper might be generated by AI is if it has many hyphens. Apps like ChatGPT include many, although the reason is unclear. Massaro has a way to test if he suspects a student generated their paper with AI. He calls them over to his desk and asks them to use the hyphen on the computer. Often, the student doesn’t even know how to do it, he says. “Your paper is full of hyphens. Just show me how you did it.” If the student struggles, it is obvious what happened.
No indentation
Massaro says that, contrary to the stipulations of traditional pre-college work, it is common for the copied and pasted text from a chatbot to have no indentation. What ChatGPT and other similar apps produce does not have indentation. If the student’s work is nothing but long blocks of text, it is most likely written by a robot.
Perfect grammar, shallow content
If a student submits a paper with perfect writing, but not much to say, that is another sign that it was generated by an algorithm, says Massaro. While many students may be able to write well but don’t have many original ideas, AI-generated writing has telltale signs. Massaro says that AI style has “uniform sentences and paragraphs, all the same size, feeling rhythmic, mechanical”. Furthermore, the writing often has a “hyper-formal tone,” he said, which is often characterized by a “overly polished academic voice that is not typical of a student’s work.”
No drafts
Another sign is that there is no participation in the drafting process that takes place in class, Massaro said. As part of his composition classes, Massaro conducts revisions and expert conferences. The student may claim to have gone to the school’s Writing Center to write their paper, but in reality, they have not. When they submit these papers, there is usually “unusual vocabulary for the student, with sudden leaps of sophistication not seen in their previous work or when speaking in class.”
Impersonal writing
Massaro occasionally asks for personal reflection papers. The student who writes the paper themselves based, for example, on their experience working at a local 7-Eleven, will include all kinds of personal details that an AI could not invent, says Massaro. On the other hand, if the paper was automatically generated, it is likely to refer very abstractly to the importance of “friendship” for humanity, he said in a joking tone.
Excess data
It is surprising that there are students so careless in their deception that they leave behind excess data, like their interaction with the chatbot, in the paper they submit to the professor.
Nonsense like citations
It is known that AI has the habit of “hallucinating” information. In a paper submitted by students, those “hallucinated” references by the AI will manifest in different ways, including directly incorrect or invented information, and false citations. This can create additional hours of work for Massaro, who is forced to go on a hunt to see if any unclear data is real or not. Sometimes he has to search in academic publications and books cited in the work, but that do not present the citations included by the student in the bibliography (because they do not exist).
It can be uncomfortable and difficult to confront a student because they have deceived, says Massaro, although it is also unusually easy to detect what has happened. Often, the guilty student will admit to using AI, or simply not respond to Massaro’s email (many study remotely). Innocent students usually protest, and Massaro says he usually believes them.
The most disheartening thing about the increase in AI-generated writing is that it robs students of their ability to cultivate their own intellectual and creative identity. “Now is supposed to be when they can find their voice,” he commented. Instead, it is AI that speaks (and thinks) for them.
This article has been translated from Gizmodo US by Lucas Handley. you can find the original version.
