August 7th, 2024

Where Facebook's AI Slop Comes From

Facebook is struggling with AI-generated spam, driven by creators in developing countries exploiting the platform for profit. The Creator Bonus Program incentivizes sensational content, overwhelming moderation efforts.

Read original articleLink Icon
Where Facebook's AI Slop Comes From

Facebook is facing a significant issue with AI-generated spam content, often referred to as "AI slop." This phenomenon is largely driven by creators in countries like India, Vietnam, and the Philippines, who are incentivized to produce bizarre and emotional images that garner high engagement on the platform. These creators learn their techniques from YouTube influencers and utilize tools like Microsoft's AI Image Creator to generate content. The Facebook Creator Bonus Program pays users for viral posts, which has led to a surge in the production of sensational images, including those depicting poverty and disaster, as they tend to attract more likes and shares. Influencers teach others how to exploit Facebook's algorithms and content moderation systems, often sharing strategies through videos and guides. Despite Facebook's efforts to manage this spam, the company's content moderation teams are reportedly overwhelmed, allowing these tactics to proliferate. The result is a cycle where creators continuously adapt to new trends and vulnerabilities in Facebook's system, leading to a flood of low-quality, AI-generated content that dominates user feeds.

- Facebook's AI spam problem is fueled by creators in developing countries seeking to monetize viral content.

- The Creator Bonus Program incentivizes the production of sensational and emotional AI-generated images.

- Influencers share strategies for exploiting Facebook's algorithms, leading to widespread spam.

- Content moderation efforts at Facebook are reportedly insufficient to keep up with the volume of spam.

- The phenomenon reflects a broader trend of using AI tools to generate engaging but low-quality content.

Link Icon 10 comments
By @snowwrestler - 9 months
Long but absolutely worth the read. People in other countries with much lower costs of living are essentially working full time to post meaningless crap to Facebook, because the “creator” payouts from Facebook, while tiny by U.S. standards, are meaningful sums in these other countries. And this has in turn spawned “how to get rich quick” influencer content which itself is also monetized the same way, as well as pretty sophisticated software to automate it all.

This story has surprisingly altered the way I feel about this AI spam on Facebook. While it’s disconcerting and annoying content to look at… I have to give props to these folks for finding clever ways to suck money out of lazy rich tech companies. Respect for the international hustle.

The part of the ecosystem missing from this story is the revenue stream to Facebook. Are advertisers actually ok with their ads showing up next to this crap, and getting charged by Facebook for the privilege? Maybe? Not all ads are run by big conservative image-conscious consumer companies. If ads get real clicks, maybe that’s enough for some advertisers, regardless of how silly the AI content next door.

By @yimmothathird - 9 months
Pretty much what I expected. It was very obvious to me it was just engagement hacking for money when I saw damn near every one has a list of irrelevant celebrity hashtags attached at the end.

Still, there are ones even stranger than the trash AI images targeting the attention of idiots. I saw a post seemingly advertising a real book, but for some reason, in the content of the post. The claimed the book was written by [Person A] while the photo of the book they posted clearly noted the author as [Person B]. Didn't seem like a simple accident either. The actual author had a name that looked to be of middle eastern origin, while the fale author was some west-African insert.

By @jazzyjackson - 9 months
good reporting but what I really want to see is a profile on the kind of people who actually like and share this content, let alone follow the page. Are they kids that think it's funny? senile pensioners who think that's actually a photo of a wounded veteran on his birthday?

are the people of facebook OK?

By @iamleppert - 9 months
So Facebook is basically paying people in third world countries to spam people with low quality AI content? Isn't this technically advertiser fraud? They are diluting the value of the content on the platform, thereby decreasing the signal to noise ratio. I can't imagine the effect on advertisers can be good, unless you're Temu peddling more junk.
By @CM30 - 9 months
Ah, it's the same issue that every user created content site and service with a payout system has. Because the cost of living is much lower in other countries, there's an incentive for people there to exploit these systems with useless crap in an attempt to get rich quick.

I saw it on forums with revenue sharing all the way back in the 90s and 00s. You had people from all kinds of less well off countries signing up and posting junk en masse because the tiny amount that Adsense paid out was a king's ransom in those parts of the world.

And it's been a constant ever since. 'Web 3.0' games that paid people to play had these people turn them into full-time jobs, Quora has an endless deluge of junk being posted because of their contributor program, and the same is true of everything from Medium to YouTube to TikTok and Instagram. Any time there's a way to 'make money from home', then it turns into a get rich quick scheme/full-time business for those in low cost regions or with few other options.

But hey, it's inevitable. If posting junk online pays better than local businesses and jobs do, then of course you're going to see an increasing number of people trying to get in on the hustle.

By @manx - 9 months
So many stakeholders with different incentives: advertisers, facebook, regular users, paid content creators, ...

How could we design incentives which produce a positive outcome for everyone?

By @ilrwbwrkhv - 9 months
This is fantastic! The hacker in me really loves the thinking, coordination and arbitrage these guys are doing. More power to them.
By @manx - 9 months
The question now becomes: how to classify useful content. I doubt that's possible without downvotes and some more advanced algorithms than majority voting. Right now, those systems are full of probably unintended feedback loops.