Meta's Former Trust and Safety Head Is Building the Content Moderation Infrastructure for the AI Era

The person who helped Meta build one of the world's most complex content moderation systems is starting over — this time with AI as both the subject of moderation and the primary tool for doing it. The move reflects a broader recognition that the content moderation problem that challenged social platforms over the past decade has a sequel, and the sequel arrives with AI-generated content at volumes that make previous challenges look manageable.

The Scale Problem Gets Worse Before It Gets Better

The fundamental challenge of content moderation at platform scale has always been asymmetry: creating harmful content is orders of magnitude cheaper and faster than reviewing and removing it. Human moderators, working at maximum throughput, cannot keep pace with the volume of content generated by hundreds of millions of users. Meta, YouTube, and Twitter built increasingly sophisticated AI-assisted moderation systems over the past decade precisely because the alternative — pure human review — was economically and operationally untenable.

The arrival of high-quality AI content generation doesn't change that asymmetry. It dramatically worsens it. Generating realistic synthetic media — images, video, audio, text — at scale now requires minimal expertise and near-zero marginal cost. The content moderation systems that struggled to handle human-generated abuse at scale are now facing volumes of AI-generated content that would be impossible to review through any primarily human workflow.

What the New Approach Looks Like

The company being built by Meta's former trust and safety lead is focused on building moderation infrastructure specifically designed for AI-generated content — with provenance tracking, synthetic media detection, and policy enforcement at a layer below the content itself. Rather than reviewing content after it's published, the approach intercepts it at the generation or distribution layer, using AI models specifically trained to identify synthetic media signatures, policy violations, and coordinated inauthentic behavior patterns.

The enterprise pitch is to the platforms that cannot afford to build this infrastructure themselves — the mid-tier social platforms, messaging applications, and community tools that have the same moderation obligations as larger platforms but a fraction of their resources. The new company is positioning itself as the content moderation infrastructure layer for applications that need serious policy enforcement without the ability to hire the thousands of people that Meta and Google employ for the purpose.

The Timing Is Deliberate

The founding of a content moderation startup in early 2026 is not coincidental. The EU's AI Act, which imposes specific obligations on AI-generated content detection and disclosure, takes full effect for most categories of providers in 2026. US platforms are watching for federal regulation that may follow similar patterns. And the volume of AI-generated content that reached platforms in 2025 — during the first full year of widespread access to capable text-to-image and text-to-video tools — has already created visible moderation failures at every major platform. The market for serious moderation infrastructure is no longer hypothetical.