Getty Images
Thus far, even AI companies have had trouble coming up with tools that can reliably detect when a piece of writing was generated using a large language model. Now, a group of researchers has established a novel method for estimating LLM usage across a large set of scientific writing by measuring which “excess words” started showing up much more frequently during the LLM era (i.e., 2023 and 2024). The results “suggest that at least 10% of 2024 abstracts were processed with LLMs,” according to the researchers.
In a pre-print paper posted earlier this month, four researchers from Germany’s University of Tubingen and Northwestern University said they were inspired by studies that measured the impact of the COVID-19 pandemic by looking at excess deaths compared to the recent past. By taking a similar look at “excess word usage” after LLM writing tools became widely available in late 2022, the researchers found that “the appearance of LLMs led to an abrupt increase in the frequency of certain style words” that was “unprecedented in both quality and quantity.”
Delving in
To measure these vocabulary changes, the researchers analyzed 14 million paper abstracts published on PubMed between 2010 and 2024, tracking the relative frequency of each word as it appeared across each year. They then compared the expected frequency of those words (based on the pre-2023 trendline) to the actual frequency of those words in abstracts from 2023 and 2024, when LLMs were in widespread use.
The results found a number of words that were extremely uncommon in these scientific abstracts before 2023 that suddenly surged in popularity after LLMs were introduced. The word “delves,” for instance, shows up in 25 times as many 2024 papers as the pre-LLM trend would expect; words like “showcasing” and “underscores” increased in usage by nine times as well. Other previously common words became notably more common in post-LLM abstracts: the frequency of “potential” increased 4.1 percentage points; “findings” by 2.7 percentage points; and “crucial” by 2.6 percentage points, for instance.
These kinds of changes in word use could happen independently of LLM usage, of course—the natural evolution of language means words sometimes go in and out of style. However, the researchers found that, in the pre-LLM era, such massive and sudden year-over-year increases were only seen for words related to major world health events: “ebola” in 2015; “zika” in 2017; and words like “coronavirus,” “lockdown” and “pandemic” in the 2020 to 2022 period.
In the post-LLM period, though, the researchers found hundreds of words with sudden, pronounced increases in scientific usage that had no common link to world events. In fact, while the excess words during the COVID pandemic were overwhelmingly nouns, the researchers found that the words with a post-LLM frequency bump were overwhelmingly “style words” like verbs, adjectives, and adverbs (a small sampling: “across, additionally, comprehensive, crucial, enhancing, exhibited, insights, notably, particularly, within”).
This isn’t a completely new finding—the increased prevalence of “delve” in scientific papers has been widely noted in the recent past, for instance. But previous studies generally relied on comparisons with “ground truth” human writing samples or lists of pre-defined LLM markers obtained from outside the study. Here, the pre-2023 set of abstracts acts as its own effective control group to show how vocabulary choice has changed overall in the post-LLM era.
An intricate interplay
By highlighting hundreds of so-called “marker words” that became significantly more common in the post-LLM era, the telltale signs of LLM use can sometimes be easy to pick out. Take this example abstract line called out by the researchers, with the marker words highlighted: “A comprehensive grasp of the intricate interplay between […] and […] is pivotal for effective therapeutic strategies.”
After doing some statistical measures of marker word appearance across individual papers, the researchers estimate that at least 10 percent of the post-2022 papers in the PubMed corpus were written with at least some LLM assistance. The number could be even higher, the researchers say, because their set could be missing LLM-assisted abstracts that don’t include any of the marker words they identified.
Those measured percentages can vary a lot across different subsets of papers, too. The researchers found that papers authored in countries like China, South Korea, and Taiwan showed LLM marker words 15 percent of the time, suggesting “LLMs might… help non-natives with editing English texts, which could justify their extensive use.” On the other hand, the researchers offer that native English speakers “may [just] be better at noticing and actively removing unnatural style words from LLM outputs,” thus hiding their LLM usage from this kind of analysis.
Detecting LLM use is important, the researchers note, because “LLMs are infamous for making up references, providing inaccurate summaries, and making false claims that sound authoritative and convincing.” But as knowledge of LLMs’ telltale marker words starts to spread, human editors may get better at taking those words out of generated text before it’s shared with the world.
Who knows, maybe future large language models will do this kind of frequency analysis themselves, lowering the weight of marker words to better mask their outputs as human-like. Before long, we may need to call in some Blade Runners to pick out the generative AI text hiding in our midst.