The National Institute of Standards and Technology (NIST), an agency of the U.S. Commerce Department dedicated to developing and testing technology for the government, businesses, and the public, has reintroduced a test bed designed to assess the impact of malicious attacks—especially those that "poison" AI training data—on AI system performance.
Named Dioptra, after a classic astronomical and surveying instrument, this modular, open-source web tool was first launched in 2022. It aims to empower organizations that train AI models, as well as users of these models, to evaluate, analyze, and monitor AI risks effectively. According to NIST, Dioptra serves as a benchmark for research and provides a unified platform for subjecting models to simulated threats in a “red-teaming” environment.
“Testing the effects of adversarial attacks on machine learning models is a primary goal of Dioptra,” stated NIST in a recent press release. “This open-source software, available for free download, can assist various stakeholders—including government organizations and small to medium-sized enterprises—in evaluating the claims of AI developers regarding their systems’ performance.”
Dioptra was launched alongside documents from NIST and its newly established AI Safety Institute, which detail strategies for mitigating AI-related risks, including the potential for misuse in generating nonconsensual pornography. It comes on the heels of the U.K. AI Safety Institute's Inspect, a similar tool aimed at evaluating model capabilities and overall safety standards. This collaboration between the U.S. and U.K. focuses on advancing AI model testing, a partnership highlighted at the U.K.’s AI Safety Summit held at Bletchley Park in November of the previous year.
Dioptra also stems from President Joe Biden’s executive order (EO) on AI, which includes directives for NIST to assist with testing AI systems. This EO also sets forth safety and security standards for AI, requiring companies—like Apple—to inform the federal government and share results from all safety tests prior to public deployment.
As discussed previously, establishing AI benchmarks poses significant challenges. The most advanced AI models often operate as black boxes, with crucial details like infrastructure and training data concealed by the developing companies. A recent report from the Ada Lovelace Institute, a U.K.-based nonprofit that studies AI, highlighted that evaluations alone may not be sufficient to gauge the real-world safety of AI models, as current policies permit vendors to choose which evaluations to implement selectively.
While NIST doesn’t claim that Dioptra can eliminate all risks associated with models, the agency suggests that it can illuminate the types of attacks that might hinder an AI system’s performance and quantify the consequent impact.
A notable limitation of Dioptra is that it currently supports only out-of-the-box models that can be downloaded and utilized locally, such as Meta’s expanding Llama family. Models restricted behind APIs, like OpenAI's GPT-4, are not currently compatible with the tool.
Ranking
