U.S.–China safety deal
U.S.–China AI safety agreement. Washington and Beijing agree on AI red lines, share safety testing and set up a crisis hotline.
What it does
The two governments would agree on a short list of things no AI should be allowed to do, such as helping build bioweapons or copying itself without permission, run some joint safety tests, and set up a hotline to warn each other about dangerous AI incidents. It would not stop either country from building more powerful AI.
Applies to: Government-to-government commitments.
- Washington and Beijing agree on AI red lines, share safety testing and set up a crisis hotline.
Where things stand
There is no hotline, no joint testing and no agreed red lines between governments. Unofficial scientist dialogues (IDAIS: Beijing 2024, Venice 2024, Shanghai 2025, London April 2026) have produced consensus statements only.
Why it costs almost no lead
Talks and a hotline cost labs almost nothing. Under a day.
Biggest unknown: Whether any deal would contain verifiable commitments rather than just a communication channel.
Why it lowers p(doom) by ~0.1%
Sets shared red lines, joint safety testing and a crisis hotline between the two AI superpowers.
Shared red lines and a hotline lower the chance of an accident spiralling. Without verification, the effect is modest.
The strongest case that it costs more
China may use talks to slow U.S. policy without changing its own behaviour. Critics worry about legitimizing Chinese AI governance.
The debate
For
- Presidents Joe Biden and Xi Jinping Nov 2024: affirmed the need to maintain human control over the decision to use nuclear weapons and to develop military AI prudently.
- IDAIS scientists (Yoshua Bengio, Andrew Yao, Geoffrey Hinton and others) Beijing consensus (Mar 2024) set red lines against AI that self-replicates, seeks power, deceives creators or aids WMD and cyberattacks; Shanghai consensus (July 2025) warned of AI deception.
- Treasury Secretary Scott Bessent Sept 2026: proposed a U.S.-China AI dialogue and notification system for national-security-level AI incidents, saying more transparency between the top two AI powers is 'very important'.
- Sen. Chris Van Hollen and 16 Democratic senators Sept 2026 letter urging a formal U.S.-China agreement on frontier AI guardrails with technical verification.
Against
- China Foreign Ministry (Guo Jiakun) and state media Sept 2026: called U.S. calls to slow AI 'fear-mongering, confrontation and vicious competition'; Global Times framed them as a Cold War playbook to contain China.
- Analysts cited by Semafor (incl. Goldman Sachs investor survey) Sept 2026: expected the AI safety talks to be 'largely symbolic' given disputes over chips, trade and Taiwan.
Sources
- White House readout, Biden-Xi meeting in Lima (Nov 16, 2024): Nuclear human-control affirmation.
- NSC statement on U.S.-PRC talks on AI risk and safety (May 13, 2024): First official U.S.-China AI talks in Geneva.
- TIME: The Narrow Path to a Trump-Xi AI Deal (Sept 22, 2026): Likely scope: dialogue channel, incident notification, red-phone style link; over two years since last official AI talks.
- International Dialogues on AI Safety: Track 2 red lines and consensus statements 2023-2026.
Rough starting points, not precise forecasts. Lead costs assume China doesn't depend on U.S. models, the case least favorable to safety laws, and count 3 years. On the menu you can change every assumption and put in your own numbers. Last priced 2026-09-26.