Anthropic Launches Initiative to Develop Advanced AI Performance Benchmarks
Anthropic has announced a new program aimed at funding the creation of innovative benchmarks to evaluate the performance and impact of AI models, specifically generative models like its own, Claude.
Introduced Monday, this initiative will provide financial support to third-party organizations that, as stated in Anthropic's blog, can “effectively measure advanced capabilities in AI models.” Interested entities can apply for funding on a rolling basis.
“Our commitment to enhancing evaluation standards is designed to uplift the field of AI safety, offering valuable tools for the entire ecosystem,” Anthropic shared on its official blog. “Crafting high-quality evaluations that prioritize safety remains a significant challenge, and the demand far exceeds the current supply.”
As previously discussed, the AI community grapples with a benchmarking dilemma. The most widely recognized benchmarks fall short in accurately reflecting how everyday users interact with the systems being evaluated. Additionally, many benchmarks, particularly those developed before the emergence of modern generative AI, may not accurately measure what they claim to assess due to their age.
Anthropic's proposed solution, though ambitious, aims to develop benchmarks that delve into AI security and societal effects through innovative tools and methods. The company is specifically looking for assessments that evaluate a model's capacity to execute tasks such as conducting cyberattacks, enhancing weapons of mass destruction (e.g., nuclear arms), and manipulating or misinforming people (e.g., through deepfakes or misinformation). Regarding national security concerns, Anthropic is dedicated to creating an early warning system for identifying and assessing AI-related risks, though the specifics of this system remain undisclosed.
Furthermore, Anthropic aims to support research into benchmarks and "end-to-end" tasks that investigate AI's potential in scientific research, multilingual communication, mitigating biases, and self-censoring harmful content. To facilitate this, Anthropic envisions new platforms that empower experts to design their own evaluations and conduct extensive trials with “thousands” of users. The company has appointed a full-time program coordinator and may invest in projects that demonstrate scalability.
“We offer a variety of funding options tailored to meet the unique needs of each project,” Anthropic stated in the blog post, although a spokesperson did not provide further insights into these options. “Teams will have the opportunity to engage directly with Anthropic’s domain experts in areas such as red teaming, fine-tuning, and safety.”
While Anthropic's initiative to create new AI benchmarks is commendable, its success hinges on sufficient financial resources and manpower. Given the company’s commercial objectives within the AI space, some skepticism about its trustworthiness is warranted.
In the blog post, Anthropic expresses its desire for certain evaluations it funds to align with the AI safety classifications it has developed, incorporating input from organizations like the nonprofit AI research group METR. While this aligns with the company's goals, it may compel applicants to accept definitions of “safe” or “risky” AI that they do not necessarily endorse.
Moreover, a segment of the AI community may challenge Anthropic’s references to “catastrophic” and “deceptive” AI risks, such as those associated with nuclear weapons. Experts argue that there is scant evidence that AI, in its current form, will attain world-ending capabilities anytime soon, if ever. They contend that claims of imminent “superintelligence” distract from immediate regulatory concerns, such as AI's propensity for hallucination.
In closing, Anthropic hopes its program will act as “a catalyst for progress towards a future where comprehensive AI evaluation is an industry standard.” This mission resonates with numerous open, corporate-independent efforts to develop better AI benchmarks. However, it remains uncertain whether these independent initiatives will collaborate with an AI vendor whose primary allegiance lies with shareholders.
Why Many AI Benchmarks Fall Short
Ranking
