Miasma: Developers Build 'Endless Poison Pit' to Trap and Waste AI Web Scrapers

A new open-source project named Miasma, released this week on GitHub by developer Austin Weeks, takes an adversarial approach to the AI data scraping problem: rather than blocking crawlers, it traps them.

The concept is a honeypot at scale. When Miasma detects a request pattern consistent with AI training data collection — high-velocity crawling, unusual user-agent strings, the signature behaviour of bulk scrapers — it serves the crawler a procedurally generated page that appears coherent but contains no meaningful information. Each page links to further procedurally generated pages, creating a graph of plausible-looking content that a crawler can traverse indefinitely without extracting anything of value.

The Mechanism

Miasma's approach is technically elegant in its simplicity. Rather than maintaining a blocklist of known scrapers — a perpetual arms race — it focuses on behavioural detection. Legitimate browsers exhibit characteristic interaction patterns: they load resources selectively, render JavaScript, follow user navigation flows. Automated scrapers optimise for throughput: they request pages sequentially, skip resource-intensive rendering, and often ignore standard crawl-delay signals.

When the system identifies a likely scraper, it redirects the session to the poison pit — a dynamically generated namespace of URLs that the scraper believes are real pages. The content generator produces text that is grammatically plausible and topically adjacent to the real site's content, but semantically empty. Trained on this content, a language model would learn nothing useful — and might actively be degraded by the noise.

The Broader Context

Miasma arrives as the question of AI training data consent has moved from academic discussion to active litigation. The New York Times, several major news organisations, and numerous individual creators have filed suits arguing that AI companies scraped their content without license. The legal outcomes remain uncertain, but the effect has been to push developers and site owners toward technical countermeasures rather than waiting for legal resolution.

Previous countermeasures — robots.txt extensions like AI-specific disallow rules, services like Cloudflare's AI scraper blocking — operate on the same principle: identify and block. Miasma's contribution is the active deception layer. By making the scraper believe it is successfully collecting data, the tool wastes the scraper's resources rather than simply diverting them elsewhere.

The project has generated significant interest in developer communities, accumulating several hundred GitHub stars within 24 hours of publication. Whether it proves effective against the most sophisticated scraping operations — which increasingly use residential proxy networks and browser automation to mimic legitimate traffic — remains to be seen. But it represents a meaningful escalation in the technical toolkit available to site owners on the consent side of the AI training data debate.