Debate Over AI Detector Reliability Sparks Concerns in Academia

I ran my PhD thesis through an AI detector. It told me 70% of it was written by AI.

I wrote it in 2010.

ChatGPT did not exist. Generative AI as we know it did not exist. So either I am a time traveller, or these tools do not work.

The detector flagged my thesis, ‘The Suitability of the IASI Instrument for Observing CO from Space’, as 70% machine writing. Then it offered to ‘humanise’ it for me, for a fee, and told me it had been checked against four other detectors.

I used a tool called textguard.ai, but I am not singling it out. I could have used any of them. The model is the same across the sector: tell you your own writing looks artificial, sell you a service to disguise it, and take your work as free training data on the way through. I handed over a real thesis for nothing and got a sales pitch back.

For me this is funny. A physics thesis from 2010, about a satellite measuring carbon monoxide, flagged as AI.

For a student in front of a misconduct panel, it is their degree. And the burden of proof falls on the person with the least power to carry it, often including those writing in a second language.

The Financial Times reported this same unreliability this week, as universities start to drop these tools over accuracy. My own research auditing UK universities is part of that story.

If you want to test it yourself, my thesis is free and open here. Run it through whichever detector you like and see what it says.

Academic integrity comes from good assessment design: deciding what you are actually trying to assess, then building the task around the thinking you want to see. A probability score was never going to do that.

#AIDetection #HigherEducation #AcademicIntegrity #CriticalAILiteracy #AIinEducation