AI-generated slop is quietly conquering the internet. Is it a threat to journalism or a problem that will fix itself?

It’s not disinformation. It’s vague text, filled with buzzwords, with no real point. It’s a hastily made meme illustration of a celebrity or a politician. It’s a news article where something about the tone or the facts doesn’t seem right. These are examples of what experts now call AI slop. Fears of AI-generated misinformation have been overstated. Hallucinations, frequently seen in the first iterations of AI chatbots, even in public demonstrations, have dwindled in later versions. But slop is alive and growing, with lots of low-quality, lazy text copied and pasted from chatbot responses threatening to take over the internet. 

What is slop?

The word slop “conjures images of heaps of unappetizing food being shovelled into troughs for livestock” and was first applied to AI-generated content on online messaging boards, writes New York Times journalist Benjamin Hoffman. The concept mirrors similar terms for large quantities of low-quality information on the internet. A good example is ‘pink slime’, a phrase often used to describe politically motivated networks of low-quality local news sites. 

Slop can take the form of text, images, video, and sometimes even entire websites. It can also seep into real life, as thousands discovered when they recently flocked to Dublin’s city centre for a Halloween parade that didn’t exist, promoted on an AI-generated website. Similarly, the now-infamous Willy Wonka experience held in Glasgow earlier this year could be billed as real-life ‘slop’: the event really did take place, but the AI-generated visuals and text used to advertise it belied a much lower-quality reality.

Experts have accused digital platforms of leaning into slop heavily. YouTube has announced plans to add a feature for users to create AI-generated ‘Shorts’ videos. A similar shift is also in the works for Facebook and Instagram, both Meta-owned platforms. Will these updates help creators make unique and engaging posts? Or will they fill feeds with the digital equivalent of fast food, rather than updates from friends, family and people we admire?

AI-generated slop often acts as fodder for websites whose only purpose appears to be optimising for SEO as cheaply as possible. This could impact the way we engage with information online, and the traffic news outlets receive.

On the other hand, some have compared slop to email spam, which platforms became very efficient at filtering out. Will slop fizzle out or will it proliferate to an extent where it degrades the entire information ecosystem, as some critics fear? 

I spoke to three experts to find out: Professor Sandra Wachter, a Professor of Technology and Regulation at the Oxford Internet Institute at the University of Oxford; McKenzie Sadeghi, editor of AI and Foreign Influence at NewsGuard; and David Caswell, the founder of a company designing and applying generative AI to news and media products.

The issue of careless speech

One specific set of risks that could fall under our definition of slop is what academics Sandra Wachter, Brent Mittelstadt and Chris Russell call ‘careless speech’. 

Careless speech is AI-generated output that contains “subtle inaccuracies, oversimplifications or biassed responses that are passed off as truth in a confident tone,” the authors explain in a paper published in August. 

This is different from disinformation, Wachter said. The aim is not to mislead but to convince or to sound confident. This is similar to philosopher Harry G. Frankfurt’s concept of ‘bullshit’, whereby a speaker is unconcerned with whether or not they’re saying the truth – it’s irrelevant as their goal is persuasion. 

“As Frankfurt says, the thing that is most dangerous to a democratic society is not a liar, it's a bullshitter,” Wachter said. 

Identifying careless speech is difficult. Unlike deepfakes, which often are ‘outrageously wrong’, Wachter describes careless speech as ‘subtly wrong’. Sometimes it’s missing nuances or even partially correct. The difficulty in classifying chatbot responses as true or false is something we have also noticed at the Reuters Institute in our analyses of AI responses to questions about elections. 

As we noted in those cases, “users may not always notice these errors given the authoritative tone of these systems.”

Mistakes in careless AI output can also be harder to spot. The inaccuracies are not predictable to someone looking out for false information, as there is no agenda behind them. “You're just not able to see this is wrong as it wouldn't be something a human would lie about,” Wachter said. 

Careless speech can easily pass under the radar. “These models are trained to be engaging, human-sounding, helpful, profitable,” she said. “Whether they speak the truth or not is not of concern. Technically, they can't. But they are presented as truth-tellers, and that's what makes them dangerous.”

According to Wachter and her colleagues, most large language models (LLMs) are not designed to tell the truth but just to replicate human speech realistically. “LLMs are incidental truth tellers,” they write, but they sound like they’re telling the truth. 

This clashes with our interests as a society, the authors argue. If the way LLMs operate doesn’t change and we keep relying on them, we could suffer “longer-term, collective or social, and cumulative” harms, including the homogenisation of information, or the inclusion of incorrect information into shared records like Wikipedia.

A gradual erosion

Another concern of Wachter’s is what she calls recursion: the idea that AI-generated text, easier, faster and cheaper to produce, will proliferate on the internet, eventually being input back into LLMs as training data, in a feedback loop leading to a gradual erosion of quality and value. 

Wachter likens this issue to environmental pollution. “Everybody's just throwing their empty cans into the forest. So it's going to be much harder to have a nice walk out there because it's just being polluted, and because those systems can pollute so much quicker than humans could,” she said. “It's like a serpent biting its own tail.”

Wachter, Mittelstadt and Russell advocate for a new legal framework to hold LLM developers accountable for the harms of careless speech. This framework, they say, should require developers to build models able to reliably tell the truth. 

These authors think LLMs should employ experts and create a feedback process focused on truthfulness and involving the public. “It's not about deciding what's right or wrong,” said Wachter, who explained it is about setting up systems and due diligence mechanisms that can raise the likelihood that an AI output is truthful.

This would not be foolproof, and there may be situations in which these guardrails are not enough to guarantee a truthful output, such as in situations like an ongoing news event. The duty they want to be recognised is to minimise careless speech through a design geared towards telling the truth, not to eliminate all incorrect output.

“This is not about material truthfulness… You don't have the obligation to find out what is truthful and not. This is not about liability if you print something that is wrong, or if [the model] creates something that is wrong,” Wachter said.

Wachter also advises LLM users to be mindful about what they rely on the models to do. A separate 2023 paper she co-authored says that in science or any other sector interested in producing truthful information, “you should never use [generative AI] as an oracle or a crystal ball.”

One LLM developer who’s touted the goal of telling the truth is Elon Musk. In 2023, when he announced his intention to create his own chatbot, he said it would be a ‘TruthGPT’ and “maximum truth-seeking”. 

So far, Musk’s company xAI has released the chatbot Grok, which is trained on posts on the Musk-owned social platform X. Grok was recently found to be repeating election-related misinformation. After this was brought to light by a group of US officials, Grok now redirects any election-related queries to official websites, like many other chatbots. 

AI-generated news websites

From the point of view of the news industry, the AI slop that is most likely to have the pollution effect Wachter describes takes the form of AI-generated websites masquerading as news. 

Using AI is an efficient way to generate text to pad out websites. People may do this to advance a political agenda, as in local news networks recently identified by the Tow Center for Digital Journalism in the US. But they may also do it just to produce sites optimised for SEO to maximise advertising revenue. Think of clickbait headlines, repeated search terms, Buzzfeed on steroids. 

The most blatant examples of the latter motivation are sites belonging to newly defunct news outlets bought and filled with AI-generated articles. The goal is to mop up the remaining traffic from audiences who may not be aware of the change in ownership. 

In February, Wired’s Kate Knibbs reported on a Serbian entrepreneur who has bought abandoned news sites, filled them with AI-generated content and pocketed ad revenue.

NewsGuard, a US-based company that rates news sites based on a range of criteria including transparent finances and accuracy, keeps a running tally of ‘unreliable AI-generated news websites. At the time of this writing, this tally stands at over a thousand websites ‘operating with little to no human oversight’. 

Most of these are “low-quality clickbait farms publishing a lot of content about celebrities, entertainment, and politics,” said NewsGuard’s McKenzie Sadeghi. Some also cover issues like cryptocurrencies and obituaries. 

Most of the sites identified by NewsGuard as AI-generated appear to be financially motivated. They optimise content to specific keywords and searches to rank highly on search results, gain higher traffic and then more advertising revenue. But there are exceptions. Sadeghi mentioned 170 websites belonging to a Russian disinformation network. They run no ads and their sole goal seems to be to spread propaganda.

How the project started

NewsGuard’s project began shortly after tech companies began rolling out generative AI chatbots. Sadeghi and her colleagues noticed that more and more social media posts contained messages like “I am unable to answer this question” or “I’m unable to access up-to-date information, my knowledge is cut off”. These phrases, typical of AI chatbot error messages, are signs not only of their usage but also of minimal human oversight, she said.

The NewsGuard team performed a search for those phrases. “Initially, we found 49 and then we realised that this was just the tip of the iceberg,” she said. Since the project began in the spring of 2023, Sadeghi and her colleagues have conducted weekly searches in English, French and German. 

Once they have identified a website to look into, the team looks for three to five error messages, as well as other indicators like human content creators or writers identified in bylines or author pages. Even listed authors are checked, as occasionally ‘journalists’ are faked with stock photos. 

“After we feel there's a strong enough amount of evidence that it's solely AI-generated with minimal human oversight, we'll categorise it as such,” Sadeghi said.

Human involvement varies, she said, with most sites registered anonymously. When a person is listed, NewsGuard attempts to contact them and offers the opportunity to comment. 

When the team succeeded in speaking to one person behind an AI-generated website, they said that they used to employ human writers and replaced them with AI to avoid having to pay their wages. The more human involvement there is, Sadeghi said, the more sophisticated the websites are.

Is slop’s impact overstated?

For David Caswell, who works in applying generative AI to news processes and products, AI is just a tool and it’s up to the people who use it to do so responsibly. He views low-quality AI-generated information as the product of get-rich-quick-minded early adopters. 

“A lot of those early adopters are the same that were in crypto,” he said. “It's the whole bro culture, making fast money. It’s a pump and dump strategy, if not in stocks, then in intention.”

Low-quality information has long predated AI, as have generalisations, false statements and careless mistakes. Sometimes, human-made speech and photos share a lot of qualities with the typical AI-generated output. 

In Caswell’s opinion, low-quality AI-generated content will be increasingly identified and curtailed by platforms. “It's like spam,” he said. “In the early days of email, it was completely out of control. But then we learned how to take care of it, and how to minimise it.” 

Just like most of the spam emails we receive are now relegated to a hidden section of our inbox, Caswell predicts the internet will have a “vast underbelly of AI crap that very few people ever see.”

Medium CEO Tony Stubblebine recently made a similar argument regarding the influx of AI slop on the blogging platform. 

“The vast majority of detectable AI-generated stories in the raw feeds for these topics already have zero views. Zero views is the goal and we already have a system that accomplishes [that],” he told Wired’s Kate Knibbs. Stubblebine said an increase in low-quality AI-generated content on his platform “doesn’t matter” as long as safeguards, including human curation and an AI policy, mean it doesn’t reach readers.

While Sadeghi’s team doesn’t track engagement on the AI-generated websites they identify, she does point to some cases in which specific sites are cited elsewhere. 

Two months before the US election, a false claim regarding the US presidential debate between Donald Trump and Kamala Harris was published by CountryLocalNews.com, a fake news website mentioned in NewsGuard’s first report about AI-generated news. This story was cited by social media users to claim the debate was rigged, Sadeghi said. 

In 2023 a claim published by Global Village Space, a website NewsGuard found to be AI-generated, also gained traction. It falsely claimed that Israeli Prime Minister Benjamin Netanyahu’s psychiatrist had died by suicide. The man named as the purported psychiatrist doesn't seem to have existed. This falsehood was reposted across social media and covered by Iranian state television. The original article on Global Village Space is now labelled as satire, with a note reading: “For sake of clarity this information was taken from a satire site about a psychiatrist who killed himself in 2010.”

The short-term impact

It’s too early to figure out the long-term impact of AI slop in the news ecosystem. But there is some evidence that mistakes published by AI-generated news websites can cause real-world harm in the short term. 

Earlier this year, Hong Kong-based BNN Breaking accompanied a story about an unnamed Irish broadcaster’s trial for sexual misconduct with a photograph of a prominent Irish TV and radio host who had nothing to do with the case. The mistake was a result of the use of an AI chatbot to produce the piece. 

This was not an isolated incident for the outlet, whose journalists were told to use a generative AI tool to paraphrase stories published by other newspapers, Kashmir Hill and Tiffany Hsu reported for the New York Times. Dave Fanning, the broadcaster wrongly identified in the BNN story, is suing the outlet for defamation. 

News deserts, places where there is little to no local news coverage, may be particularly vulnerable to AI-generated news websites, due to the lack of alternative sources of information. They have already long been targeted by so-called ‘pink slime networks’, defined by NewsGuard as websites backed by partisan funders that are presented as local news websites. 

These sites have long used automation to pad out their content. Recent generative capabilities have “allowed these operations to produce [content] at a faster and cheaper rate,” Sadeghi said. She worries about the impact of AI-powered fake websites on trust in news, particularly local news. 

For Caswell, the issue is not AI but creating the right incentives. News deserts are easy targets for cheap, low-effort and low-quality content that some people will read anyway because there’s nothing else available. 

“If there is a news desert, and you have this horrible piece of fermenting pink slime, and that's all you get as far as news in your desert, that's value, right?” he asked. The solution to this problem, he said, is a high-effort local news initiative, one that could still possibly use AI to keep costs manageable.

News providers should distinguish themselves from the mounds of low-quality information online, both AI-generated and not, Caswell argued. 

“Differentiation isn't like a switch that you flip: it has to be based around something real,” he said. “What’s going to make a difference is being relevant to audiences.”

Caswell thinks publishers should use AI in a way that doesn’t compromise editorial judgement or a newsroom’s values. AI can help with scale, flexibility and efficiency, but it does not change the key questions any journalist should be asking themselves: "Are you delivering value? Are you being relevant? Are you adhering to your values? Are you serving a societal need? All of these things remain the same."