AI Pioneer Bengio Joins Project for Developing AI Safety Standards: Enhancing AI Ethics and Security

Yoshua Bengio, a Turing Award winner and pioneer in deep learning, has joined Safeguarded AI as the scientific director. This initiative, supported by the U.K. government, aims to create an AI system that can monitor and address risks posed by other AI agents, essentially acting as a "gatekeeper" for AI safety. The gatekeeper system is designed to oversee autonomous AI interactions, ensuring they operate within predefined safety parameters.

The project proposes that the gatekeeper system will not only mitigate risks associated with advanced AI but also enable its safe use in critical applications, unlocking the potential benefits of cutting-edge AI in business and commercial settings where reliability is crucial. Backed by a substantial $75 million investment from the U.K. government’s Advanced Research and Invention Agency, Safeguarded AI is well-positioned for the next four years and has garnered the expertise of a prominent figure in the AI field.

Yoshua Bengio brings his scientific and strategic insights to the project, collaborating with program director David “davidad” Dalrymple and the rest of the Safeguarded team. Bengio, who won the prestigious Turing Award in 2018 for his work on deep learning, advocates for a cautious approach to AI safety. In his latest efforts, he has co-authored a paper outlining a new framework called "Guaranteed Safe AI," which aims to provide stringent safety guarantees for AI systems, particularly those with high autonomy or used in safety-critical applications.

The Guaranteed Safe AI approach includes a world model, safety specifications, and a verifier to ensure AI systems meet safety requirements. Bengio and the Safeguarded team believe this framework can offer viable safety assurances while remaining competitive with other AI systems. They see various approaches within this framework as complementary efforts in the pursuit of AI safety.

In addition to Bengio's involvement, Safeguarded AI has launched a funding call for organizations interested in utilizing the gatekeeper AI system to enhance the safety of their products. The project envisions applying this tool to specific domains such as energy networks, clinical trials, and telecommunications networks. Businesses have until October 2 to submit their proposals for consideration.