AI is already everywhere, and readers may not be as good at detecting it as we think we are.

Using AI-detection tool Pangram, researchers from Stony Brook University found that one in five of the 14,000 randomly selected Kindle ebooks in their data set contained more than 25% AI-generated text.

• Particularly notable is the finding, per Pangram, that #BookTok’s favorite dark dystopian romance, Daggermouth by H.M. Wolfe, is 60% AI-written.

Because AI-detection is almost as controversial as AI itself, the researchers also checked books suspected of containing AI text for “rare expressions.”

• These “rare expressions” are phrases containing five words or more in a particular order that rarely appear in texts written by humans but show up often in those suspected of being AI-generated.
• Daggermouth contains many such phrases, including multiple sentences that appear word-for-word in self-published ebooks by other authors.

Daggermouth author H.M. Wolfe denied the allegations in a statement her lawyer sent to The Atlantic, noting, “I have been outspoken about my opposition to generative AI and what it’s doing to writers, artists, and the creative community. I don’t believe it belongs in the writing process.”

• Wolfe did not address the question of “rare expressions.”

No matter how damning the evidence may appear, it’s important to note that there is not yet a definitive way to prove AI was used to generate a written text.

• The slowness of Wolfe’s publication pace relative to self-published authors with hundreds of books to their name may support her claim that, “the suggestion that I used generative AI to write  Daggermouth is wholly untrue.”
• OTOH, an AI expert not affiliated with the study found the researchers’ methods to be sound, stating that it is “almost statistically impossible” for a book that was actually written by humans to trip AI-detection software as 60% AI-generated.

The implications of these findings go well beyond concerns about one uber-popular book, as the study also indicated that the 20% of ebooks in Amazon’s catalog found to contain “substantial” AI-generated or -assisted text account for 12% of sales and 10% of bestsellers.

• This is evidence that AI-generated ebooks, now easily produceable for self-publishing at scale, are cannibalizing the market for human-written texts.

See the data for yourself. Here’s the whole study and Will Oremus’s coverage for The Atlantic.