As Chief Information Security Officer at Anthropic, Jason Clinton has a multifaceted role, directly reporting to CEO Dario Amodei. With a dedicated team, he manages various aspects of security, including data and physical protection, at this Google and Amazon-backed startup renowned for its advanced language models, Claude and Claude 2. Despite raising over $7 billion in investment and employing around 300 people, Clinton's primary focus is safeguarding Claude’s model weights—housed in a massive terabyte-sized file—against unauthorized access.
In machine learning, particularly within deep neural networks, model weights represent the critical numerical connections that enable the neural network to learn and make predictions. The final values of these weights significantly influence the model's overall performance. A recent Rand Corporation research report highlights the importance of protecting these weights, which encapsulate the extensive resources and complex processes involved in training advanced models. If obtained by malicious actors, these weights could allow full access to the model at a fraction of the training cost.
“I probably spend almost half my time as a CISO thinking about protecting that one file,” Clinton remarked in a recent interview, noting that it receives significant attention and resources within the organization.
Concerns About Model Weights
Clinton, who transitioned to Anthropic after an 11-year tenure at Google, pointed out that while some consider the weights highly valuable intellectual property, the company's primary concern is preventing the technology from falling into the wrong hands. He explained that misuse by opportunistic criminals, terrorist groups, or nation-states could have dire consequences. “If an attacker accessed the entire file, that's the entire neural network,” he cautioned.
This concern is echoed by recent U.S. government initiatives. The White House's Executive Order on the “Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence” mandates that foundation model companies document the ownership and security measures surrounding their model weights.
OpenAI, a notable player in the field, stated in an October 2023 blog post that it is diligently investing in cybersecurity measures to safeguard proprietary model weights, limiting distribution outside its organization and technology partner Microsoft.
Attack Vectors Identified in New Research
Co-authors Sella Nevo and Dan Lahav from Rand Corporation's recent report, “Securing Artificial Intelligence Model Weights,” identified around 40 potential attack vectors that bad actors might exploit to steal model weights. From unauthorized physical access to supply chain attacks, the report highlighted real-world examples of these vectors in action.
Nevo emphasized that concerns are less about current capabilities and more focused on future risks, predicting significant national security implications as models advance.
Risks of Open Foundation Models
Not all experts agree on the severity of risks associated with AI model weight leaks, particularly regarding open-source models. A Stanford HAI policy brief indicated that widely available open foundation models can foster innovation and transparency, suggesting that the risks associated with them should be evaluated against closed models.
Kevin Bankston from the Center for Democracy & Technology commended the brief for its balanced, evidence-based analysis. The brief highlighted mixed outcomes, citing Meta's Llama 2 model, which was released with publicly available weights despite a previous leak.
While advocates argue for open-source security, Heather Frase from Georgetown University pointed out that as generative models evolve, the potential for harm also increases, particularly for individuals targeted by malicious technologies.
Emphasizing Openness in Security
Nicolas Patry, an ML engineer at Hugging Face, stated that the risks associated with model weights require regular security protocols. Nonetheless, he believes that transparency enhances security. William Falcon, CEO of Lightning AI, echoed this sentiment, arguing that attempts to control model weight leaks are futile as the open-source community rapidly evolves.
Clinton agrees that open-source models don't pose the most significant risks Anthropic must prioritize. He urges that governments should focus on regulating ‘frontier’ models while emphasizing the importance of ongoing research and security.
Ongoing Security Challenges
Despite optimism from researchers, Nevo cautioned against complacency, warning that current security measures may not adequately protect against future threats. Clinton highlighted the challenge of a talent shortage in AI security, stating, “There are no AI security experts… We need top security engineers who can adapt quickly to this evolving landscape.”
He expressed concern about the increasing ease with which attackers might exploit vulnerabilities. Looking toward the future, he anticipates a shift in cybersecurity practices from periodic to daily updates, which would require a significant change in mindset across the industry.
Clinton's commitment to balancing rapid research advancements with robust security measures underscores the urgency of proactive strategies to safeguard AI model weights. “It's crucial for our research team to feel supported while securely managing model weights,” he concluded.
Ranking
