Meta is rolling out new AI-driven content enforcement systems across its apps while scaling back reliance on third-party moderation vendors, a shift the company says will improve accuracy, speed, and user safety at platform scale. The move concentrates more moderation work in-house and on models trained to spot fast-evolving abuses, from illicit drug sales and scams to terrorist propaganda and child exploitation.

According to Meta, early tests show the new models detect twice as much violating adult sexual solicitation content as human review teams while cutting error rates by more than 60%. The systems also identify roughly 5,000 scam attempts per day aimed at stealing login credentials, and they are catching more impersonation accounts targeting celebrities and public figures.

The company frames automation as essential to outmaneuver adversaries who constantly switch tactics, language, and imagery. Tasks like repetitive reviews of graphic material are better suited to machines that can triage at scale, freeing human experts to adjudicate edge cases and high-impact decisions. Meta says these systems will be deployed broadly only after they consistently outperform current methods on real-world tests.

While Meta did not publish model architectures, the company describes a multilayered approach combining text, image, and video analysis with account and network signals. For example, suspected account takeovers are flagged by patterns such as logins from unfamiliar locations, sudden password resets, or rapid profile edits, paired with content cues that can indicate coercion or fraud. Similar multimodal signals help the system spot covert drug marketing that relies on emojis, misspellings, and ephemeral Stories.

Meta says experts design, train, and continuously evaluate these models, with humans remaining in the loop for appeals, legal escalations, and the most sensitive policy calls. In practice, that means AI handles the bulk of repetitive enforcement while specialized teams focus on ambiguous or high-risk content, a split recommended by groups like the Partnership on AI and reflected in guidance from the NIST AI Risk Management Framework.

Vendor Cutbacks And The Continuing Human Role In Safety

Reducing third-party vendors marks a strategic rebalancing after years of heavy outsourcing across regions like the Philippines, India, and Ireland. Outsourced moderators have been instrumental for scale but have also faced well-documented well-being challenges. By shifting repetitive and graphic review to automation and concentrating complex judgments with in-house experts, Meta aims to improve quality control and speed while limiting exposure to the most harmful content.

Crucially, Meta emphasizes that people will still make the “highest risk and most critical” decisions, including account disablement appeals and referrals to law enforcement. That stance aligns with research from the Stanford Internet Observatory and the Atlantic Council’s DFRLab, which warns that fully automated moderation can amplify bias, miss nuanced context like satire or newsworthy exceptions, and create opaque pathways for users to contest mistakes.

Policy Pressures And Safety Tradeoffs Facing Meta Now

The technology shift comes as Meta revisits its broader policy posture. Over the past year, the company has eased some moderation rules, ended its third-party fact-checking program in favor of a community notes-style model, and pushed a more personalized approach to political content. At the same time, Meta and other platforms face lawsuits alleging harms to young users, and regulators in major markets are tightening oversight, including under the EU’s Digital Services Act.

Automation could help reconcile these pressures by catching more genuine violations while reducing over-enforcement that chills legitimate speech. Yet the risks are real: model drift as tactics change, false positives on edge cases, and the need for transparent appeals. Independent audits, detailed transparency reporting, and red-team evaluations—best practices supported by the Carnegie Mellon Software Engineering Institute and civil society groups—will be critical to prove the systems work as intended.