‍

Humans have been wondering how to identify AIs for a long time. Famously, Alan Turing developed what he called “the imitation game” - now better known as the Turing Test - in 1950, and in 1968 Philip K Dick wrote Do Androids Dream Of Electric Sheep?, spawning the Blade Runner franchise and the infamous Voight-Kampff test for identifying androids. Sadly, the Voight-Kampff test is of little use to us in the 21st century, as our Androids are phones, and our AIs do not have physical bodies that betray their lack of emotions. What methods are left to us to identify content generated by the computers in our midst? Here are some tips that can help you sniff out the spoofs. 

‍

‍

Be aware, and rely on instinct

Let’s get the bad news out of the way first: at the time of writing, there are no guaranteed methods to spot AI-generated text. Instead, fleshy detectives must rely on a combination of imperfect methods, and one of the most reliable is human instinct. Because AI-generated text is cobbled together using probability, rather than human creativity, AI output tends to feel off to a human reader.

‍

‍

This vibes-based approach may seem like an unscientific solution, but science has shown that it’s becoming more effective as people grow more familiar with AI output. As recently as 2019, studies found that humans struggled to detect AI-generated text because it was grammatically perfect, and conformed to what they thought of as “proper” writing. Now, we expect AI output to be flawless, and as we become more familiar with its measured, robotic rhythms, the chances of detecting them in the wild increase! And once your suspicions are aroused, you can run some other checks… 
‍

Ask an AI!

Many software tools exist that claim to help sniff out these grammatically perfect AI authors, but even the best of them struggle to maintain a high success rate, and as AI language models grow more complex the task will only grow harder. Detection tools are far from futile, though; at the time of writing the most effective tool seems to be GPTZero, which analyzes text using two metrics, “perplexity” and “burstiness”. 

‍

‍

“Perplexity” relates to how unusual a sequence of words is, with a low perplexity score indicating that a text is more likely to be AI-generated. The most basic indicator of low perplexity is the frequency of the word “the”, which is the most commonly used word in the English language. AIs construct their sentences by searching for the most commonly used word sequences, so they use “the” more often than humans do.  Meanwhile, “burstiness” analyzes the rhythms of text. AIs tend to write with a measured and even pace, favoring sentences and paragraphs of similar lengths. Humans are more chaotic, with more variation in punctuation, pacing, and sentence construction. Combining these two metrics makes for a useful - if far from foolproof - test, but are there more reliable clues to be found?
‍

Human errors vs AI errors

An AI’s probability-based writing style can give it away, but so can its desire to avoid sloppy mistakes. AIs do not make typographical errors, so if you spot a spelling mistake in a piece of text, you can be certain that it came from a human hand. Another excellent tell is the double space, a frequent occurrence in human-generated text written with a standard keyboard. Double spaces are hard to spot with the naked eye, but are easy to find using a computer’s text search function; simply search suspicious text for two consecutive spaces; if you find them you’re extremely  likely to be dealing with a human’s work. 

Of course, AIs make errors all the time, and even though we can’t explain exactly how those errors are generated, we can be confident that chatbot AIs will continue to suffer from “hallucinations” for some time. With that in mind, one of the most effective AI detection tools at our disposal is simple fact checking. AI errors tend to be more extreme than human goofs!

‍

The long game

Finally, it’s worth noting that identifying AI content should get much easier in the near future. Concerted efforts are underway to devise a system for watermarking AI generated text; one likely means of achieving this would be the use of “red lists” and “green lists”, which either forbid AIs from using certain commonly occurring sequences of words, or require them to always use one particular common sequence when many frequently used options are available. Researchers are also looking at “poisoning” datasets; fake data (poison) is inserted into the dataset that the AI is trained on, and when that data appears in the output it gives the AI author away. 

These methods will surely be more effective than those currently at our disposal, but it’s unlikely that we will ever have a failsafe method for filtering out the fakes. As AIs grow more sophisticated, they will become more adept at emulating human behaviors, and relying on the creators of popular AI software to watermark their work will only reduce the scale of the problem, not eliminate it. In the long term, we may well find that our best bet for sniffing out the spoofs may be back where we started, relying on good old human instinct.