OpenAI’s VP of Global Affairs Declares o1 ‘Virtually Perfect’ at Bias Correction, Yet Data Shows Otherwise

OpenAI's New Reasoning Models: A Step Toward Reducing Bias in AI

While the recent departures within OpenAI may be capturing headlines, Anna Makanju, the company’s VP of Global Affairs, sparked important discussions on AI bias. During a panel at the UN’s Summit of the Future, Makanju highlighted the potential of emerging "reasoning" models, like OpenAI's o1, to significantly reduce bias in artificial intelligence.

Makanju explained that these reasoning models can identify biases in their responses and adhere more closely to guidelines that discourage harmful answers. "Models like o1 actually take longer to process information and can evaluate their own answers," she noted. "This allows them to assess their approach and recognize when a flaw in reasoning may exist."

She added, "This self-analysis is done exceptionally well, enabling the model to detect its own biases and produce improved responses. We are continually evolving in this area."

OpenAI's internal research supports Makanju's claims, revealing that o1 generally generates less toxic, biased, or discriminatory content compared to traditional non-reasoning models, including its predecessor GPT-4o. However, the phrase "virtually perfectly" may overstate the current capabilities.

In OpenAI's bias assessment, which involved sensitive questions related to race, gender, and age, o1 showed mixed results. While it was less likely than GPT-4o to imply bias based on race, age, and gender, it performed worse in terms of explicit discrimination on age and race. The findings highlighted the model's need for further refinement.

Moreover, a smaller, more cost-effective version, known as o1-mini, displayed even greater challenges. The bias test indicated that o1-mini was more prone to explicit discrimination based on gender, race, and age compared to GPT-4o and more likely to implicitly discriminate on age.

These findings underscore that current reasoning models still have room for improvement. OpenAI acknowledges that o1 provides minimal advantages in certain tasks, is slower—with some inquiries taking over ten seconds for a response—and is significantly more expensive, costing three to four times more than GPT-4o.

If reasoning models are indeed the most promising path toward achieving unbiased artificial intelligence, as Makanju suggests, they must enhance their overall performance beyond just bias reduction. Without these improvements, only organizations with substantial resources willing to navigate latency and performance challenges will be able to fully leverage these models.