Production monitoring

Why pre-release testing isn't enough on its own

Every category described elsewhere on this page is evaluated before a version ships. That evaluation uses a test set built with real care and real adversarial thinking behind it. But pre-release evaluation, however thorough that process is, is still evaluation against a necessarily fixed and finite test set. Real usage at genuine scale is different. It spans a much larger and more varied population of users, requests, and edge cases than any pre-release test set could realistically anticipate. It finds patterns a fixed test set structurally cannot. The space of real human requests is larger and stranger than any test-set author's imagination.

That's the specific reason production monitoring exists as a genuinely separate, ongoing layer. It isn't a formality that runs briefly after the real work of pre-release testing is already considered finished. It's treated internally with the same seriousness as pre-release evaluation. It isn't a lesser, catch-all backstop for whatever the real testing missed. It's staffed and reviewed on an ongoing basis, not only spun up reactively after something has already gone wrong. The team responsible for it has a standing mandate, not a rotating, ad hoc assignment picked up between other projects.

What gets watched

Usage patterns consistent with any of the seven evaluation categories described elsewhere on this site are monitored on an ongoing, continuous basis in production. That monitoring is entirely independent of the pre-release evaluation described on each category's own page. A pattern that a fixed test set missed still has a real chance of being caught, once it shows up in genuine usage at scale. Confirmed abuse identified through this monitoring results in account-level enforcement under our usage policies. Enforcement ranges from a warning through to suspension, depending on severity, using the same enforcement framework described on the Incident response page. None of this monitoring reads the content of your conversations to do its job; it watches for patterns in tool actions and account behavior, not for what you personally said.

A pattern that reveals a genuinely new evasion technique feeds directly back into red-teaming. That includes anything that doesn't match any previously catalogued attempt. It feeds into the red-teaming process described on its own page. This closes a loop. Production monitoring informs red-teaming. Red-teaming informs the next round of safety tuning. That loop is the mechanism by which the evaluation categories on this site stay current against real-world attempts, rather than gradually going stale against a fixed, aging picture of what an attack looks like.

Rate limits as a safeguard, not just a cost control

Default rate limits applied on the API and on tool actions generally are set with abuse resistance specifically in mind. That's not purely a cost-management measure, the way rate limits are often assumed to exist for. A limit on how many commands can be run, or how many file edits or draft messages can be produced, in a short window is a safeguard against a compromised account. It's also a safeguard against a runaway automation causing damage at speed and at scale. That's true independent of whether any single individual action taken within that window would, on its own, have passed every category check described elsewhere on this site. These limits sit alongside the connector-specific and sandbox-specific scoping described on other pages in this section, not in place of them.

That distinction matters. A single bad action and a thousand bad actions taken in rapid succession are genuinely different risks. That's true even when each individual action would, evaluated in isolation, look acceptable. A category-by-category safety evaluation alone doesn't naturally catch that difference in scale. These limits aren't fixed forever for every user. They can be raised for verified, higher-trust use cases on request. The goal is resisting abuse at scale specifically, not artificially constraining ordinary, legitimate high-volume usage that has a clear, verifiable, good-faith explanation behind it.

Enforcement

Enforcement actions arising from this monitoring range from a warning through to full account suspension. The severity of what was found determines where on that range a given case lands. So does whether it reflects an isolated mistake or an established, repeated pattern of misuse. The most severe categories described elsewhere on this site are hazardous-material misuse and child safety. Both are enforced without a warning step at all. That's consistent with the zero-tolerance framing those two categories are given on their own dedicated pages. A warning implies a second chance, and those two categories are deliberately not structured to offer one.

Avoiding false positives

A monitoring system tuned aggressively enough to catch every real abuse pattern will, if it isn't checked carefully, also catch a fair number of legitimate heavy users along the way. An actual penetration tester running a genuinely high volume of security-research requests, a developer whose tCode session issues an unusually large number of file edits in a short window because the task genuinely called for it, and a team running bulk, entirely legitimate administrative work through the API are all realistic examples of usage that can superficially resemble an abuse pattern without being one. Treating them as though they were is a real cost, not a safe default, in the same way over-refusal is a real cost on the individual evaluation categories described elsewhere in this section. We track a false-positive rate on this monitoring the same way we track a catch rate on genuine abuse, rather than only measuring the side of the ledger that makes the system look effective. A monitoring system that only reports its wins would be telling an incomplete, and misleadingly flattering, story about how well it actually works. Reporting both sides honestly is harder, and it's the standard we'd want applied to this monitoring if we were the ones reading the report instead of writing it.

If an enforcement action was applied incorrectly, there's a path to get it reviewed and reversed. That path is the same contact channel described on the Responsible disclosure page, since it routes to the same team that can actually look into an account-level action and correct it. We'd rather a legitimate user have a fast, clear way to get a mistaken restriction lifted than treat every enforcement action as final the moment it's applied. Getting this wrong in the direction of over-restricting genuine, good-faith usage is taken exactly as seriously as under-catching real abuse, a theme that runs throughout this entire section rather than being specific to this page alone. A reversed enforcement action feeds back into the monitoring system the same way a confirmed abuse case does, since both are useful signal about where the current tuning is off. The goal is a system that gets more accurate over time in both directions, not just more aggressive.