AI Generated Slop Is Taking Over Facebook

Theodore, a 19-year-old who lives in Paris, has become the top curator of what he calls “Insane Facebook AI slop,” and shares his favorite examples on a dedicated Twitter account with more than 100,000 followers. On that site, it’s customary to mock the content, as well as the gullible Boomers assumed to be eating it up. There’s even an AI slop bingo card meme that followers can use to keep track of some common tropes, including the generic captions most often appended to the images, like: “Why don’t pictures like this ever trend” and “You will never regret liking this photo.” (Scarlett Johansson‘s name also shows up a lot, for some reason.)  

“I don’t use Facebook a lot myself but saw plenty of screenshots of those types of posts on Twitter, and the insane amount of likes these had made the phenomenon funnier,” Theodore tells Rolling Stone. “So I thought I might as well document those.” At this point, he almost exclusively shares content he receives in DM submissions.

Theodore believes the success of AI slop is mostly due to the elderly and naive. “Facebook is full of old people and tech-illiterate people in general,” he says, which is “also why the kind of posts that get the most likes are the ones that will target old people, using soldiers, American flags, Jesus imagery, etc.” Since some themes are so dominant, he tries to maintain variety, lately branching out into illustrations of “third-world children doing all sorts of impossible crafts.”

Meta has taken a rather gentle approach to AI-derived images and videos, opting to add an “AI info” label “when we detect industry standard AI image indicators or when people disclose that they’re uploading AI-generated content,” it said in a statement this summer. This label did not appear on any of the AI images shared by more than a dozen engagement-farming pages reviewed by Rolling Stone. As for more stringent moderation, Meta’s Oversight Board has argued that they “unnecessarily risk restricting freedom of expression when we remove manipulated media that does not otherwise violate our Community Standards.”

That is to say, because the typical AI slop isn’t strictly misinformation (nor is it hate speech, graphic nudity, and so on), there’s no glaring reason to delete it — unless Meta were to deem it spam. The company’s policy states that it does not allow spam, which it defines as “content that is designed to be shared in deceptive and annoying ways or attempts to mislead users to drive engagement.” The AI slop pages are certainly deceptive, repetitive, and gaming the system for engagement; there’s also no telling how many are monetized and profiting from an abundance of reactions and comments.

Meta tells Rolling Stone that while “eradicating spam is a nearly impossible task,” the company takes significant measures against it, since this type of content “detracts from people’s ability to engage authentically in online communities.” In the first quarter of 2024, it reported, Meta “removed 436 million pieces of spam content from Facebook,” with 98.2 percent of this “actioned before it could be reported.”

The question remains whether Meta would deem the AI-generated images to be spam in violation of their policy, or just take the engagement as a sign to feed those users more of the same. The pseudonymous data researcher and software developer Conspirador Norteño in June published a Substack investigation that suggested the latter: that Facebook’s recommendation algorithm is favoring the AI slop over authentic content.

After logging in with a dummy account he had only ever used to shop for music gear, Conspirador simply scrolled through more than a thousand posts on his feed over half an hour and collected information on both the content and the pages that shared them. He found that less than 5 percent of the material corresponded to his past browsing of music equipment, 12.7 percent was sponsored ads, and the rest was “mostly a mix of AI-generated images and plagiarized photographs posted by content aggregator accounts with large numbers of followers.” AI-generated content accounted for 22 percent, or almost a quarter, of all the posts he viewed.

“I think a lot of people still operate with the expectation that images that look like photos are real photos, and don’t notice that they’re really looking at an AI image unless there are really obvious problems,” Conspirador says. He noted the prevalence of pages such as “Log Cabin Living,” “Mountain Cabins,” and “Barndominium Gallery,” which tend to feature images of lavishly appointed country mansions set in lush nature scenes — pictures that often don’t immediately betray signs of being fake. These reliably pick up likes, comments, and subscribers. “Unusual-looking houses and pretty outdoor scenes tend to do well, even when an account posts dozens or hundreds that are basically the same,” Conspirador says. A typical “Log Cabin Living” post features a preposterously large house with an uncanny pool that seems to blur the distinction between artificial and natural bodies of water:

People gazing with admiration at fantasy houses that don’t exist is one thing, and a social media algorithm amplifying this stuff is another. But Conspirador was particularly surprised by “Facebook’s failure to intervene in cases where the account [was] obviously hijacked” from an ordinary user before it began spewing AI spam. This was the situation with “Barndominium Gallery,” a page which he found had previously belonged to a hair salon in Oklahoma. In January, the owner of the salon posted that her page had been hacked and asked friends to report it to Facebook, yet eight months later, no action has been taken, while the page has accrued hundreds of thousands of followers with its AI rubbish. Worse, whoever stole the page has begun promoting links to a fake home-construction business that charges the customer $79.99 for floor plans and cost-of-build estimates on its AI-generated architecture.

Carissa Véliz, an associate professor of philosophy at the University of Oxford’s Institute for Ethics in AI, views the slop takeover as the logical result of how Facebook works. “This phenomenon seems to be a secondary effect of social media’s focus on engagement,” she says. “Instead of focusing on fostering wholesome relationships, or in being trustworthy sources of information, social media has focused on gaining people’s attention. And it turns out that what attracts most of our attention is akin to junk food.”

“Generative AI was designed to engage people,” Véliz says. “It’s good at producing images that people find captivating.” Facebook, she points out, needs that content that keeps people glued to the app, and may tolerate clickbait because they’re concerned that “users were no longer sharing as many personal stories as before.” What’s more, Véliz proposes, companies like Meta are incentivized to create Frankenstein’s monsters beyond their control, as the “inadequate current regulation doesn’t demand of social media companies anything that can be too burdensome.” Therefore, if you have a system running amok, but fixing it would be too difficult, “it magically absolves you from its effects.”