AI writing detection tools perform poorly and are easy to fool, according to researchers. 

A study by a group of researchers led by Debora Weber-Wulff, professor of media and computing at the University of Applied Sciences, HTW Berlin, assessed 14 AI detection tools, including Turnitin, GPT Zero and Compilatio, to identify text generated by ChatGPT. 

The researchers assessed the tools by writing short undergraduate-level essays on a variety of subjects, including civil engineering, computer science, economics, history, linguistics, and literature. They wrote the essays themselves to be certain the text was not already online, which would have meant it might already have been used to train ChatGPT. 

Then each researcher wrote an additional text in Bosnian, Czech, German, Latvian, Slovak, Spanish, or Swedish. Those texts were passed through either the AI translation tool DeepL or Google Translate to translate them into English. 

The team then used ChatGPT to generate two additional texts each, which they slightly tweaked in an effort to hide that it’d been AI-generated. One set was edited manually by the researchers, who reordered sentences and exchanged words, while another was rewritten using an AI paraphrasing tool called Quillbot. 

The researchers found that the tools were good (average 96 percent accurate) at detecting text written by a human, but much less good when it came to spotting AI-generated text, especially when it had been edited (average 42 percent). 

According to Weber-Wulff, the tools "don't work" and "don't do what they say they do."