OpenAI Forms New Team to Tackle Core Technical Challenges of Controlling Superintelligent AI

OpenAI has established a new team, led by co-founder and Chief Scientist Ilya Sutskever and Alignment Director Jan Leike, to explore methods for guiding and controlling the next generation of superintelligent AI systems. This initiative is a top priority, utilizing 20% of the company's computational resources. The team aims to address the core technological challenges of controlling superintelligent AI within the next four years.

Sutskever and Leike believe that next-generation superintelligent AI could emerge within the next decade, representing a breakthrough technology with the potential to solve complex problems that previously demanded considerable human effort. However, they also warn of potential risks, including threats to human autonomy and even extinction due to excessive power.

Currently, the team lacks a comprehensive solution to ensure that superintelligent AI systems align with human intentions. The existing approach relies on adjusting AI generation processes through feedback, which does not guarantee reliable oversight from the outset. To overcome this challenge, OpenAI plans to develop an AI "automated alignment researcher" that approaches human-level reasoning, utilizing extensive deep learning to enhance its capabilities and iteratively refine its superintelligence.

Researchers are focused on scalable training methods, validating generated models, and performing controllability tests. Their goal is to leverage this AI system to assess other AI systems, enabling scalable supervision and insights into extending oversight beyond human capabilities.

As research progresses, Sutskever and Leike expect the team's focus may shift, potentially leading to new areas of exploration. The team is actively recruiting machine learning researchers and engineers, highlighting the importance of expanding expertise to confront the challenges posed by next-generation superintelligent AI.

This initiative complements OpenAI's ongoing efforts to enhance the safety of current models, such as ChatGPT, while also addressing broader AI-related risks. These include misuse, economic disruption, misinformation, bias and discrimination, addiction, and overreliance. The team is working in collaboration with interdisciplinary experts to tackle socio-technical issues, ensuring that their solutions consider the wider human and societal impacts of AI technologies.