SB 1047

What it does

California's 2024 bill that Governor Newsom vetoed. It required a safety plan before training, a kill switch, yearly audits, 72-hour incident reports and ID checks on cloud customers, and banned releasing a model with an unreasonable risk of catastrophe.

Applies to: Models above 10^26 FLOP costing more than $100M to train.

• Before training: cybersecurity protections, full shutdown capability, and a written safety and security protocol, published in redacted form and sent to the Attorney General (self-certified, no approval).
• Before release: assess whether the model could cause or enable critical harm, retain test records, and do not release if there is an unreasonable risk of critical harm.
• From 2026: annual independent third-party audit, published in redacted form.
• Report safety incidents to the Attorney General within 72 hours.
• Compute providers: know-your-customer and shutdown capability for frontier-scale customers.
• Whistleblower protections and an anonymous internal channel.
• Penalties up to 10% of training compute cost for a first harmful violation, 30% thereafter.

What's in it: Safety plan, Kill switch, Incident reports, Yearly audit, Cloud ID checks, Whistleblower protection

Where things stand

Every obligation except the audits, the KYC rules and the release prohibition was already voluntary practice at the covered labs by 2024. SB 53, enacted a year later, kept the framework, transparency and incident provisions and dropped the shutdown mandate, pre-training protocol, audits and compute-based penalties.

Why it gains ~2 days

Its release rules and audits cost about a day of U.S. progress. Its ID checks on cloud customers slow China slightly more, so on net it slightly widens the lead.

Biggest unknown: How a lab would have interpreted the duty not to release a model posing an unreasonable risk of critical harm, and whether a self-certified pre-training protocol ever delays a run.

Why it lowers p(doom) by ~0.33%

The vetoed California bill: safety plans, a kill switch, audits and incident reports.

The strongest case that it costs more

The compute-based penalty, up to 30% of training cost, creates liability that scales with ambition. A rational lab facing that exposure would evaluate more, mitigate more and release later, and the marginal caution is not captured by pricing the procedures. The undefined "unreasonable risk" standard is where the cost hides, and a court could read it in ways the lab cannot plan around. Newsom's veto message did not say the bill was expensive; it said it targeted the wrong models. The lead cost was never the strongest argument against it.

The debate

For

• Sen. Scott Wiener wrote the bill, which passed the legislature in 2024.

Against

• Gov. Gavin Newsom vetoed it, saying it applied stringent standards to even basic functions, 2024.
• Chamber of Progress, NetChoice, R Street and others signed a joint letter urging a veto, 2024.
• a16z, Meta and Yann LeCun argued against it, as reported by TechCrunch, 2024.

Sources

• SB 1047 enrolled text
• SB 1047 bill history: vetoed September 29, 2024
• Governor Newsom's veto message
• FPF, California's SB 53 explained: what SB 53 dropped from SB 1047