AI Monitoring AI: OpenAI Unveils CriticGPT, a New Model Based on GPT-4 for Detecting Errors in ChatGPT Code Outputs

The issue of AI hallucinations—instances where AI systems generate inaccurate or nonsensical outputs—has gained significant attention across the industry. Whether through China’s Wenxin Yiyan, Kimi, and Hunyuan, or international models like ChatGPT and Gemini, these inaccuracies, contradictions, and fabrications are prevalent.

To address these challenges, OpenAI has launched CriticGPT, a new tool built on the GPT-4 architecture. CriticGPT is specifically designed to identify coding errors in ChatGPT’s outputs and enhances the efficiency of human trainers by 60% compared to those who do not use the tool.

Although OpenAI acknowledges that CriticGPT's suggestions may not always be flawless, they emphasize its effectiveness in boosting the performance of human trainers. A significant part of ChatGPT's success over earlier AI versions is due to Reinforcement Learning from Human Feedback (RLHF), which fine-tunes language models based on human input to align outputs with user preferences.

CriticGPT embodies the concept of using AI to improve AI, facilitating self-correction within language models through an iterative feedback loop. It improves upon OpenAI’s earlier AI Text Classifier, which struggled with a mere 26% accuracy rate in distinguishing AI-generated text and misidentified 9% of human-written content as being produced by AI.

Utilizing a vast dataset built on GPT-4, CriticGPT applies RLHF innovatively. It involves introducing deliberate errors into training data, creating a controlled testing scenario where human annotators insert mistakes into ChatGPT's responses before CriticGPT analyzes the identified issues.

OpenAI also introduced Forced Sampling Beam Search (FSBS) to ensure that CriticGPT generates a variety of feedback options. This array of variations is evaluated using a reward model, helping to determine an optimal balance between comprehensiveness and accuracy in feedback.

CriticGPT has shown remarkable proficiency in bug detection. In tests with human-inserted errors, reviewers caught only about 25%, while CriticGPT achieved over 75% accuracy. Notably, when evaluating naturally occurring bugs, 63% of human trainers preferred the evaluations from CriticGPT over those from human programmers. Even beyond coding tasks, CriticGPT identified numerous cases where outputs deemed “perfect” by human annotators were, in fact, incorrect.

The effectiveness of CriticGPT highlights its significance as a reliable tool for detecting AI inaccuracies, proving essential in training larger models. While RLHF serves as the foundation for sophisticated language models like ChatGPT, it has inherent limitations linked to human intelligence. Without the advancements of CriticGPT, the capabilities of large models would be constrained by human understanding, complicating the reliable assessment of AI systems.

The launch of CriticGPT exemplifies OpenAI's dedication to "Scalable Oversight," ensuring that as models advance beyond human capabilities, they remain aligned with human expectations and continuously evolve. This initiative suggests that only large models may effectively oversee other large models, paving the way for a future where artificial intelligence surpasses human intelligence.