Outside safety tests
Independent pre-deployment evaluation. An approved outside evaluator tests the model and reports before public release.
What it does
An approved outside group, such as a government AI safety institute, tests the model before the public can use it and reports what it found.
Applies to: Frontier models before external deployment.
- Give an independent evaluator (a government institute or an approved organization) access to a representative checkpoint before release.
- Receive the evaluator's report before public deployment; the report can be redacted for publication.
- No fixed access period is set; the evaluator gets what it gets.
Where things stand
Every major U.S. release now goes through some external testing. UK AISI and U.S. CAISI tested GPT-5.5. METR had ten business days on Claude Opus 5.5 and three weeks on o3 and GPT-5. But access is uneven: Apollo had three days on GPT-6 Astra and SecureBio six calendar days. The practice exists; the mandate makes it universal and makes the report a precondition.
Why it costs almost no lead
The lab keeps using and improving the model while outsiders test it, so the U.S. frontier barely moves. The public launch waits, which slows China only if it depends on U.S. models.
Biggest unknown: Whether labs would share checkpoints earlier or simply wait longer at the end.
Why it lowers p(doom) by ~0.2%
Gets independent experts to check for dangerous abilities the lab might miss or play down.
Outside testers catch things labs miss or play down, and a launch deadline can't overrule them.
The strongest case that it costs more
Evaluators are capacity-constrained. If every frontier release from four labs must queue for the same institute, the queue is the delay, and it lands in the lab's release schedule whether or not the model is done. METR itself says deployment-tied evaluations often leave little time for thorough analysis. A mandate that fixes that problem by adding time is, by construction, adding time. And the labs' own behavior suggests they treat evaluator windows as release-blocking rather than parallel.
The debate
For
- Sens. Hawley and Blumenthal would bar release until a Department of Energy evaluation is done, 2025.
- U.S. AI Safety Institute, now CAISI signed agreements for access to Anthropic and OpenAI models before release, 2024.
- UK and U.S. AI Safety Institutes jointly tested Claude 3.5 Sonnet before its launch, 2024.
- Americans for Responsible Innovation backed the Hawley–Blumenthal testing bill, 2025.
Against
- Dean Ball, Foundation for American Innovation called mandatory federal testing a veto point on the future of AI, 2025.
Sources
- GPT-5.5 System Card: external evaluations: SecureBio April 2–9; UK AISI and CAISI pre-deployment testing
- GPT-6 Astra deployment safety page: Apollo: three days
- METR evaluation of Claude Opus 5.5: 10 business days of API access
- METR evaluation of o3 and o4-mini: three weeks prior to release
- METR Frontier Risk Report, Feb–Mar 2026: pre-deployment evaluations often leave little time
- AISI, Early lessons from evaluating frontier AI systems
- Expanding External Access to Frontier AI Models (arXiv 2601.11916): Apollo and UK AISI had less than a week on Claude Sonnet 4.5
Rough starting points, not precise forecasts. Lead costs assume China doesn't depend on U.S. models, the case least favorable to safety laws, and count 3 years. On the menu you can change every assumption and put in your own numbers. Last priced 2026-09-26.