GPT-4 Achieves Perfect Score on MIT Undergraduate Math Exam: A Self-Grading Success Story

GPT-4 Achieves Perfect Score on MIT Mathematics Exam!

In an exciting development, a research team from MIT, Boston University, and Cornell University announced that GPT-4 successfully passed the MIT mathematics exam with a perfect score. This remarkable achievement highlights the model's advanced capabilities in tackling undergraduate-level questions within the Mathematics and Electrical Engineering and Computer Science (EECS) departments.

In contrast, its predecessor, GPT-3.5, could only answer a third of the questions correctly. The stark difference in performance—GPT-4's flawless execution versus GPT-3.5's results—has captured significant online interest, with many questioning whether models beyond GPT-4 will be necessary for future academic challenges.

The research team utilized a comprehensive dataset of 4,550 questions derived from the MIT undergraduate curriculum, covering courses such as Electrical Science and Engineering, Artificial Intelligence, and various mathematics topics. For the assessment, 228 unique, non-image-based problems were randomly generated, involving a mix of practice questions, midterms, finals, and specialized topics.

Participating in the exam alongside GPT-4 were advanced models like StableVicuna-13B and two versions of LLaMA. The results were telling: finely tuned GPT-4 scored 100%, while LLaMA-30B managed just 30%. Even the unrefined GPT-4 performed impressively with a score of 90%. Various optimization techniques, including Few-Shot learning and Chain of Thought (CoT) strategies, contributed to these results.

The self-scoring aspect of GPT-4's exam has raised eyebrows among observers, leading to concerns about the results' integrity. Critics argue that automated scoring by the model itself may not reflect unbiased outcomes and resembles a form of self-promotion.

Moreover, the assertion that achieving high scores depends on "good prompts" has sparked further debate. The vague nature of what constitutes a "good prompt" has led some to speculate that with proper guidance, MIT students could achieve similar results.

Interestingly, StableVicuna-13B, despite its smaller size, scored 48%, outperforming both LLaMA-65B and MIT's own fine-tuned LLaMA-30B, challenging the notion that larger models inherently perform better.

In summary, while GPT-4's accomplishments are impressive, they raise important questions about the training data and evaluation methods used. The ongoing discussion about artificial intelligence's role in academia continues to evolve, carrying significant implications for the future of education and technology.