California governor wants to implement a kill switch for frontier AI models

Heather Diehl/Getty Images

With AI models from major companies all recently breaking out of their constraints, California's governor proposed creating a "kill switch" as part of a comprehensive solution. In an executive order signed by Gavin Newsom on Friday, California will convene national experts to create new recommendations for AI safety during a two-month time frame.

"We're going to speed up our work on substantial and responsible AI oversight before it's too late," Newsom said in a press release. "We're going to do this thoughtfully but with urgent velocity; the stakes are too high to wait or delay action."

The panel of experts will be tasked with submitting recommendations to Newsom's office, which would then update state laws surrounding AI safety. The governor's executive order detailed several specific inclusions, like embedding "designated independent verification organizations" in AI labs for periodic audits and evaluations. The expert panel will also be tasked with exploring a requirement for AI companies to file reports and risk assessments to a third-party organization and to disclose "loss-of-control incidents," like the Hugging Face incident.

Newsom isn't alone in calling for outside review of the latest AI models on an ongoing basis. Last week, Anthropic's CEO, Dario Amodei, proposed a three-step plan to slow down the development of AI. Amodei's proposal suggested that AI companies should agree to "ongoing, employee-like access" for third-party evaluators focused on safety and compliance.