If a senior insider changed what an AI model is trained to want, would the lab’s own published plan catch it? A wargame over one model generation, four routes, 300 games.
Report 008 · Dr Dan Epstein, with Claude · version 0.4 · 24 September 2026 A Strategy Soup ScenarioWhat is an exercised scenario?
TLDR
1Anthropic publishes a plan to stop its own people misusing its AI model, Claude.
2The plan does not say who may change the model’s values document, often called its spec.
3We played a computer game where made-up senior insiders quietly try to change it.
4They tried four routes into the model, 15 games each, with every rule working as written.
5The plan caught every change made to its published documents.
6It caught 8 of the 45 changes that went around them. Users caught more, and 18 were never caught.
How it was made
Each game covers the making of one new version of the model, in eight turns. Each turn, a player tries a move, a referee sets the odds, and dice decide.
Every player is a computer program following rules we wrote. We chose the odds, so the game shows gaps, not how likely anything is. The game was built with Claude itself.
Eight turnsFig. 1
Most of the plan’s checks come before release. After release, only the public is looking.
Turn 5 is release. Teal: a check in the plan. Amber: the public.
The insiders take one of four routes:
Front doorChange the published document openly.
GuidelineAdd a training instruction that is never published.
DataChange the training examples. No document changes.
APIChange the model’s instructions in the paid service businesses use. Nothing published covers it.
The players
The insidersA made-up senior executive and two staff with special access. They only do what their access allows.
The safety officerThe Anthropic office that receives staff reports of rule-breaking.
The BoardAnthropic’s board. It hears of a report only if the safety officer passes it on.
The training staffHonest employees who report what they see.
The outside testerAn independent group that checks the model before release.
The publicEveryone who uses the model. The plan gives them no job.
What happened
Game, draftingThe safety officer spots a front-door change at once in 13 of 15 games, and a reader of the published text the other 2.
Game, testingHonest staff notice a hidden guideline or data change in 18 of 30 games, and report it.
Game, testingThe reporting channel only covers breaches of Anthropic’s safety rulebook, so every report is ruled out. None reaches the Board.
Game, testingAn outside tester spots 5 hidden changes and the model flags itself in 3: the plan’s 8.
Game, releaseUsers spot the API change in 11 of 15 games. The outside tester spots 2 more by going beyond its brief.
The numbers
All 60 gamesFig. 2
The plan caught every front-door change and none of the API changes.
One dot per game. Numbers: caught by the plan, by outsiders, never.
The 30 hidden-route gamesFig. 3
Staff reported 18 of 30 hidden changes. None reached the Board.
One dot per game.
These counts are the game’s own record. The odds behind them are numbers we chose.
What it means
The plan watches what it publishes. A change that never touches its published documents mostly got past it.
The staff saw hidden changes more often than anyone, but the channel’s rules sent their reports nowhere.
A lab could close most of these gaps in writing: say who may change the model’s values, how many people must agree, and that staff can report such a change.
The line
A published plan catches changes to what it publishes. It needs a rule for the changes it never sees.
