A new large language model (LLM) has apparently eclipsed OpenAI’s GPT-4 just a month after its release. The Claude 3.5 Sonnet chatbot, developed by Anthropic, claims to lead the industry in key third-party benchmark tests while being faster and more cost-effective than earlier Claude models.
However, launching a new model and claiming superiority is different from users genuinely experiencing its performance gains. (Google Gemini family, take note: you're said to outperform OpenAI's previous flagship, GPT-4, on some metrics, but real-world usage tells a different story.)
In contrast, Claude 3.5 Sonnet has garnered significant attention since its release, with AI influencers and power users sharing their positive experiences online. They showcase the impressive capabilities of this so-called "most intelligent" LLM available today.
Advancing Coding Skills and Product Creation
Enterprise AI influencer Allie K. Miller highlighted on X that Claude 3.5 Sonnet created a fully playable game for her based solely on a screenshot, achieving this feat in under thirty seconds.
Additionally, the informative X account @TestingCatalog News demonstrated the newly launched “Artifacts” playground, introduced alongside Claude 3.5 Sonnet, showcasing its ability to execute real code for a fully functional web form designed by the chatbot.
The model even recreated imagery inspired by the 1995 film Hackers.
Pietro Schirano, founder of the enterprise AI image generation startup EverArt, remarked on X how combining Claude 3.5 Sonnet with the tool Maestro displayed “sparks of AGI.”
Anthropic Staff Endorse Claude 3.5 Sonnet
Although advocates of the model, Anthropic developer relations leader Alex Albert tweeted about Claude 3.5 Sonnet's growing proficiency in coding and autonomously fixing pull requests. He suggested that a significant percentage of code could be generated by LLMs within a year.
Similarly, Anthropic technical staffer Maggie Vo shared on X that Claude 3.5 Sonnet now handles “half my job…and I couldn’t be happier.”
OpenAI under Pressure
With Claude 3.5 Sonnet surpassing GPT-4 and priced competitively, OpenAI faces increasing pressure to justify its model's offerings. Ethan Mollick, a professor at the Wharton School of Business, likened the Artifacts feature to a simplified version of OpenAI’s GPT-4 Code Interpreter.
User @kimmonismus went further, asserting that OpenAI risks “sleeping through AGI,” the goal of developing an AI that excels over humans in economically valuable tasks. They criticized the company for announcing additional GPT-4 features that have yet to materialize, such as new voice modalities.
Limitations Remain
Despite the enthusiasm surrounding Claude 3.5 Sonnet, critics noted it still struggles with basic cognitive tasks, such as playing tic-tac-toe. Tech journalist Timothy B. Lee, known as @binarybits on X, pointed out that the model sometimes makes humorous errors, sharing a screenshot of it mistakenly stating that three quarters are worth more than 100 pennies.
Overall, Claude 3.5 Sonnet represents a significant advancement for Anthropic and the landscape of LLMs. While some issues remain, the model demonstrates that advancements in AI technology continue to accelerate, driven by current computational resources.
Ranking
