Can Teachers Tell the Difference Between Student and AI Writing?
April 26, 2023
The deadline for your essay is in a day, fast approaching. Your Google Doc is entirely empty. Procrastination has gotten the better of you. As you make your coffee, preparing yourself for the long night ahead of you, an idea springs in your head. You could have your whole essay written for you, have a great night of sleep, while handing in your piece of work, right on time. The temptation is growing. But, will your teacher be able to tell that Artificial Intelligence (AI) is not truly you?
” ChatGPT, the new AI chat Module” By Unsplash Creative Commons (Taken by Choong Deng Xiang)
ChatGPT, an Artificial Intelligence chat module, has notoriously been spotlighted in the media recently for its eerily uncanny resemblance to human speech and writing mannerisms. Impacting the field of education significantly, educators across the globe are grappling to overcome the threat to academic integrity which the ChatBot poses. The software works by utilizing algorithms to analyze prompts which are inputted in the chat, and then generates responses based on data it has been trained on. Information is gathered from search engines, and then delivered back in a “Chat-like” manner. Considering the threat to which this new technology poses for academic integrity at our school, we led a small investigation to discover the extent to which teachers at ISB could tell the difference between a piece of student and AI written work.
Two paragraphs were shown to participants, both written under the prompt: “Write an Introduction Paragraph About Buddhist Doctrine.” One of them generated by AI, one of them written by a 9th grade student. Following the investigation, 1 out of 4 teachers were able to correctly identify the student piece from the AI piece, while 3 out of 4 incorrectly differentiated the two. Frank Sambula, the participant who correctly identified the student-written passage, quoted how the AI paragraph was “rather generic,” and “skimmed the surface,” whereas the other was more “in-depth in terms of content.” On the other hand, a participant who got the answer wrong mentions how “Our students here at ISB are not going to write something like this [referring to the detail in the student paragraph].” Another participant who incorrectly identified the student writing mentions how the passage “has some fancy language [but] it just summarizes what a google search would say, which lacks evaluation.” Furthermore, “This passage shows greater evaluation [referring to the AI]” The contradiction between teachers’ perception on what is AI and what is student writing portrays the lack of clarity present on the indicators of AI. This exemplifies the challenge ahead of the educational field to combat the usage of AI for academically dishonest reasons.
Derek Slater, an expert in content and copyright regulation mentions how repetitive and unnatural language can ring the alarm for AI written work. Furthermore, a lack of “context specific knowledge,” or simply “skimming the surface,” can portray how a robot might be the pen behind the page.
“The AI and Student paragraph shown to teachers side by side.” By Jeremy Stuit
Prior to uncovering which paragraph was which, participants were asked about the grade that each paragraph would receive. The overall consensus was that a maximum grade of five out of seven could be achieved through using AI, trickling down to a probable four. Participants mentioned how the lack of “writer’s voice,” in the writing of AI made paragraphs rather bland, lacking flavor. However, inherent flaws existed in the experiment, including the inability for teachers to judge writing based on the identity of the student. A prominent way for teachers to identify artificial intelligence is through learning the individual writing styles of their students. This experiment did not account for that factor.
Although we can all agree that artificial intelligence has complicated the field of education, we can also agree that it is here to stay. The International Baccalaureate has made clear that AI software will not be banned, but rather embraced in the curriculum. The negative stigma which exists behind AI is not entirely justified, considering the array of benefits it brings to students. The ability for the software to help students find sources and create citations brings forth a completely viable use for the technology, which is incredibly helpful. Just like the introduction of graphing calculators in 1921 and the internet in 1983, AI will soon develop to be a quotidian part of our lives.
So, no. Do not write your entire paper using AI, but using it to help generate ideas for you can be an alternative.
And by the way, here is this entire article if it was written by AI. Could you tell the difference?
COMPLETELY WRITTEN BY AI