Meta Failed to Catch Hundreds of AI Child Abuse Ads. Some Included Images of Real Kids


The ads pointed people towards almost 50 apps combined across Google’s Play Store and Apple’s App Store. Of the 350 ads, the researchers say that around 300 of them directed people to AI image or video generator ads, with most of these apps being linked to China-based developers. Some apps had already been removed by the time WIRED started reporting this story.

Among those that would generate explicit imagery of adults were eight identical apps listed on Apple’s App Store, multiple of which were linked to Chinese developer names. When the apps opened, they would show explicit AI-generated content, including a menu of sexual scenarios, including one called “threatening knife” where a knife is held to a woman’s throat. WIRED identified another seven identical apps that shared the same design, explicit content, and listed the same developer contact details.

“The App Store was designed to be a safe and trusted place for users, and we have zero tolerance for developers who deliberately attempt to evade our review process,” says Apple spokesperson Adam Dema. “The developers mentioned in this report initially submitted apps that are compliant with the App Review Guidelines, only to later surface nudify capabilities, which are against our rules.” The company says that 17 of the 20 apps included in the research had already been removed from its store and it has taken action against the remaining three, as well as removing the seven apps WIRED found.

Google spokesperson Dan Jackson says the company does not allow apps that contain sexual content or the ability to nudify. “We continually take proactive steps to detect and remove apps with harmful content,” they say, adding that it removed all the apps included in the report. “When violations are reported to us, we investigate and take swift action, which in the case of these apps has included suspending hundreds of violating apps and restricting related search terms like ‘nudify’ on our store.”

“The share of Chinese developers creating nudify apps has grown year over year,” says David Pham, a former Meta employee who investigated nonconsensual imagery and abuse on its platforms. Pham says that while at Meta he observed similar ads to those discovered by the researchers and those uploading the content would constantly try to get around safety systems.

“Every time the platform or investigators would build a new detection model or a new way to have some type of enforcement, they quickly—within a day, maybe five days—built a new way to avoid detection,” Pham says. Those uploading the ads would change domains they used, obscure who posted them, and include measures such as blurring imagery or subtly changing images to avoid detection. However, Pham, who left Meta last year, says it has the funding and resources to be able to tackle the uploads. “I would like to live in a world where we’re not getting CSAM or NCII material sent directly to our phones,” he says.

Meta’s current protections against ads containing CSAM and NCII appear to need work. More than 11,000 undress ads have appeared on Meta’s platforms since February this year, according to a recent Indicator Media report, with many of these being reportedly posted by “repeat offenders.” Analysis by WIRED also found hundreds of ads including explicit content that contain similar language to those ads found by the TTP researchers. (Meta says malicious advertisers use adversarial tactics to continually try to avoid its detection systems and it has “incorporated” the language in the ads highlighted by WIRED into its moderation systems.)

One ad containing one of the teenagers that the researchers found the real-world identity of was reported to Meta when it had reached four accounts in Europe, according to the TTPs research. However, when it was removed days later, it had been seen by more than 330 users. “This is not something that it should take a small nonprofit doing research to stumble across and then have to investigate and report,” Paul from TTP says. “We’re talking about one of the most well-resourced— technologically and financially—companies on the planet.”



Source link

  • Related Posts

    People’s Picks: 4 Phone Brands Dominate, but 1 Reigns Supreme and It’s Not Apple

    Every few years, you may think about upgrading your smartphone. A lot of thought and research goes into your next choice, whether it’s the price, features or brand reputation. Maybe…

    Continue reading
    Apple says ‘Fold’ and competitors ask ‘how wide?’

    Apple is almost certain to launch its first foldable iPhone tomorrow, seven years after Samsung’s Galaxy Fold initially went on sale. And while Apple might be a few years late…

    Continue reading

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    You Missed

    People’s Picks: 4 Phone Brands Dominate, but 1 Reigns Supreme and It’s Not Apple

    People’s Picks: 4 Phone Brands Dominate, but 1 Reigns Supreme and It’s Not Apple

    Who has played the most career Test matches?

    Who has played the most career Test matches?

    Trump regime’s Tetris ripoff yanked from White House website after Tetris Company declare they “take copyright infringement very seriously”

    Trump regime’s Tetris ripoff yanked from White House website after Tetris Company declare they “take copyright infringement very seriously”

    Joint Statement of the Foreign Ministers of Canada, Denmark, Finland, France, Iceland, Ireland, Norway, Poland, Portugal, Spain, Sweden and the UK on the Two-state Solution

    Joint Statement of the Foreign Ministers of Canada, Denmark, Finland, France, Iceland, Ireland, Norway, Poland, Portugal, Spain, Sweden and the UK on the Two-state Solution

    Carney warns of tough times ahead in national address as retaliatory tariffs kick in

    Carney warns of tough times ahead in national address as retaliatory tariffs kick in

    WATCH: Race car driver saves fellow driver from fiery crash

    WATCH:  Race car driver saves fellow driver from fiery crash