California Governor Rejects Key AI Safety Legislation

California Governor Gavin Newsom has vetoed the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act (SB 1047). In his veto message, he expressed concerns about the bill's potential burden on AI companies, California's leadership in AI, and its broad scope.

"While well-intentioned, SB 1047 does not consider whether an AI system is used in high-risk settings or involves critical decision-making," Newsom stated. He noted that the bill imposes stringent standards on even basic functions of AI systems and could create a false sense of security regarding the management of rapidly evolving technology.

Newsom also warned that smaller, specialized models could pose equal or greater risks than those the bill targets, hindering innovation that benefits the public good. He stressed the necessity of safety protocols and enforceable repercussions for misconduct but emphasized that solutions must be informed by a thorough analysis of AI systems and their capabilities.

Senator Scott Wiener, the bill's main author, described the veto as "a setback for those advocating for oversight of corporations making critical decisions about public safety." He criticized the absence of binding restrictions on companies developing powerful technologies amid ongoing legislative inaction on tech regulation.

Introduced in late August, SB 1047 aimed to establish a rigorous legal framework for AI, requiring companies operating in California with AI models costing over $100 million to train or $10 million to fine-tune to implement safety measures like a “kill switch” and testing protocols. Additionally, it sought to protect whistleblowers and empower the Attorney General to pursue damages resulting from safety incidents.

Despite amendments that removed the proposal for a new regulatory agency and allowed preemptive legal action by the Attorney General, the bill faced backlash from many in the tech industry. OpenAI's chief strategy officer, Jason Kwon, stated that SB 1047 could hinder progress, advocating for federal oversight. Anthropic CEO Dario Amodei acknowledged improvements in the amended bill, suggesting its benefits might outweigh its drawbacks.

The Chamber of Progress, representing tech giants like Amazon, Meta, and Google, echoed concerns that the law could stifle innovation. A spokesperson for Google welcomed the veto, expressing gratitude for maintaining California's leadership in responsible AI development.

Opponents of the bill have included former House Speaker Nancy Pelosi and San Francisco Mayor London Breed, while supporters have ranged from industry leaders like Elon Musk to prominent figures in Hollywood, along with unions such as SAG-AFTRA and SEIU. Federal regulators are also exploring avenues for AI oversight, with a proposed $32 billion roadmap addressing various implications of AI technology.

Overall, the veto reflects ongoing tensions between the need for regulation and the desire to foster innovation in the AI sector.