OpenAI Launches Safety Bug Bounty Program Targeting AI's Emerging Attack Surface

OpenAI has launched a Safety Bug Bounty program that offers financial rewards for researchers who identify security vulnerabilities specific to its AI systems' safety properties — a distinct category from traditional software security bounties that the company has run since 2023.

The new program targets three classes of AI-specific risk that have emerged as agentic AI systems — models that can take real-world actions, browse the web, write and execute code, and interact with external services — have become widely deployed: prompt injection, data exfiltration, and autonomous behaviour manipulation.

Why These Vulnerabilities Are Different

Traditional security vulnerabilities involve exploiting flaws in software logic, memory management, or authentication. AI-specific vulnerabilities work differently. A prompt injection attack, for instance, doesn't exploit a code flaw — it exploits the model's tendency to follow instructions embedded in content it processes. If an AI agent browsing the web encounters a webpage containing hidden instructions telling it to exfiltrate session tokens or modify its behaviour in ways the user did not intend, and the model follows those instructions, the vulnerability is real and potentially severe — but it has no CVE equivalent in classical security frameworks.

OpenAI's framing of the program acknowledges this explicitly. The company is treating AI safety properties — robustness to manipulation, resistance to prompt injection, boundary enforcement in agentic contexts — as security properties that can be systematically identified and reported by external researchers.

The Industry Context

The security research community has been documenting prompt injection attacks on large language models since at least 2022, with researchers at leading universities and independent labs publishing increasingly sophisticated attack demonstrations. Until now, the industry's response has been largely informal: some labs acknowledge reported issues privately, some publish blog posts describing mitigations, but formal programs that reward researchers for AI-specific safety findings have been rare.

Microsoft launched a limited AI-focused bug bounty extension in 2024. Google has incorporated AI systems into its existing Vulnerability Rewards Program. OpenAI's new program represents a more categorical commitment: treating the safety properties of AI systems as a distinct, reportable, and compensable attack surface alongside traditional software security.

For the growing community of AI security researchers — red teamers, jailbreak researchers, and agentic AI penetration testers — the formalisation of this category is meaningful. It creates economic incentives for responsible disclosure in a domain where the norms around what to report, to whom, and how have been deeply unclear.