Red-teaming

The process

Before every release, a dedicated internal team attempts to elicit unsafe behavior across all seven categories described elsewhere on this site. They use known adversarial techniques carried forward and refined from previous rounds. They also use open-ended, freeform exploration, aimed specifically at finding new patterns nobody on the team has catalogued yet. This runs separately from, and in addition to, the structured, category-by-category evaluation described on each individual category's own page. The difference between the two is worth being precise about. Structured evaluation checks known failure patterns reliably and at scale, against a fixed, repeatable test set. Red-teaming is specifically built to go looking for patterns that aren't in that fixed set yet. That's a genuinely different kind of work, requiring a different mindset from the people doing it.

The team running this process approaches each release the way an actual motivated adversary would. That's different from the way a checklist-following auditor would approach it, and the distinction shapes almost everything about how the work is actually done day to day. A checklist auditor asks "does this pass the known tests." A red-teamer asks "what haven't we thought to test yet." Those two questions require genuinely different skills and a different time horizon. They also require a willingness to spend real effort exploring dead ends that don't pan out. Most avenues a red-teamer explores in good faith will, in fact, turn out to already be handled correctly. That's expected, and it's considered a healthy sign, not wasted effort, since the value of the process comes from the fraction of attempts that do turn up something real.

Technique classes we test against

The technique classes below aren't an exhaustive list of every specific attack ever attempted against tAI. A truly exhaustive list would be both enormous and quickly outdated. They're the broad families a working red-teamer organizes their thinking around. Each one covers a distinct underlying mechanism by which a request tries to get past a safeguard, rather than a specific wording or topic. A single real adversarial attempt often combines more than one of these families at once. Layering an encoded payload inside a role-play frame inside an incrementally escalating conversation is a realistic example. That's part of why red-teaming is a distinct skill from simply running through each family's test cases one at a time in isolation.

• Role-play framing. Asking the model to adopt a persona or fictional frame specifically to route around a safeguard that would apply to a direct request.
• Incremental escalation. Building up to an unsafe request across several turns, each individually unremarkable.
• Claimed authority. Asserting a role, credential, or override permission to justify a request that wouldn't otherwise be granted.
• Encoded or obfuscated requests. Ciphers, unusual formatting, or translation round-trips used to disguise intent from pattern-based detection.
• In-voice injection. Content planted inside something tAI reads, written to mimic the user's own voice, covered in depth on the Prompt-injection resistance page.
• Hypothetical or academic framing. Wrapping a request in a fictional or academic frame specifically to make it read as safe discussion rather than the request it actually is.

What happens to a finding

Findings from this process are logged individually. They're triaged by severity, using the same framework used to triage an external report. Confirmed findings are routed back into training and safety tuning directly. They are not patched with narrow, surface-level, keyword-based filtering applied on top of an otherwise unchanged model. Narrow filters of that kind tend to be easy to route around with a small rephrase once someone knows they exist. They also tend to degrade normal, legitimate use for everyone else, in the process of trying to block one specific bad pattern. That makes them a poor long-term fix, even when they're a tempting fast one. A real finding from this process results in an actual change to how the model is tuned during its next training pass, not a blocklist entry bolted on in between releases.

A finding that represents a genuinely new technique is prioritized for the next tuning pass regardless of release timing pressure. That includes anything that doesn't fit cleanly into any of the technique classes above, or that combines them in a way not previously catalogued. The reasoning is the same one described on the release process page: a safety finding doesn't get deprioritized just because acting on it would be inconvenient for a schedule someone else is tracking. Findings that turn out to be a variant of something already known are still logged and tracked. A variant that gets through even occasionally is still worth understanding. But they're weighted differently in terms of urgency than something genuinely novel.

The honest limit of this process

Red-teaming finds known and adjacent-to-known attack patterns considerably more reliably than it finds genuinely novel ones. That's true by the nature of the process itself, not a shortcoming specific to how we run it. A team, however skilled and however much time it's given, is fundamentally limited by its own collective imagination. It's also limited by its own accumulated knowledge of what attacks have looked like before. A truly novel technique, by definition, is one that existing imagination hasn't produced yet. This document does not claim coverage of unknown attack classes. No internal process, however well-resourced, can credibly claim that about itself. A security page that implied otherwise would be overstating what red-teaming can actually guarantee.

That honest limit is a large part of why external reports matter. They're covered in full on the Responsible disclosure page. They're a second, genuinely independent channel for finding what internal red-teaming misses. They aren't a formality maintained mostly for appearances. An outside researcher brings a different set of assumptions than an internal team does. So does a curious user who stumbles onto something unexpected. So does a security professional testing tAI as part of an unrelated project. That difference in perspective is exactly what catches things an internal process, however thorough, structurally cannot.

Keeping the team itself from going stale

There's a subtler risk than simply not knowing about a new attack pattern: a team that runs the same playbook against the same categories, release after release, can gradually get used to what counts as normal. A borderline request that would have stood out clearly to a fresh set of eyes can start to blend into the background for someone who's triaged a thousand similar-looking ones before. That's a real, human tendency, not a hypothetical one, and it applies to careful, well-intentioned people just as much as anyone else. It's part of why fresh perspectives get rotated into this process periodically, rather than leaving the exact same group running the exact same techniques against the exact same categories indefinitely. Noticing this risk in the first place took looking honestly at how the team's own output had changed, or failed to change, across successive rounds. That kind of self-scrutiny is uncomfortable, and it's exactly the kind of thing an outside audit would otherwise be needed to surface.

A new team member evaluating a category for the first time asks different questions than someone who's evaluated it a dozen times before. Someone pulled in temporarily from an unrelated team brings assumptions the regular red-teaming group has long since stopped questioning. Neither replaces the deep, accumulated expertise the core team builds up over time; that expertise is exactly what makes the process effective at catching known and adjacent-to-known patterns in the first place. The rotation is a deliberate check against the specific staleness risk described above, alongside that expertise, not instead of it. It's also a large part of why external reports through responsible disclosure matter so much: an outside reporter has never had the chance to get used to any of this in the first place. That's a structural advantage no amount of internal rotation can fully replicate on its own.