OpenAI Launches GPT-4 Turbo with Vision for General API Access

As enterprise developers and savvy business leaders recognize, the application programming interface (API) is central to modern software development, enabling third-party applications to seamlessly connect with tech platforms. OpenAI has recently made significant enhancements to its API for the powerful GPT-4 Turbo large language model (LLM).

The company announced on its X accounts that the GPT-4 Turbo with Vision model is now “generally available” through its API. The vision capabilities were introduced alongside audio uploads in September 2023, while GPT-4 Turbo was unveiled at OpenAI's developer conference in November. This version promises faster processing, larger input context windows (up to 128,000 tokens—roughly the equivalent of a 300-page book), and cost-effective usage.

Developers can now utilize the model’s vision recognition and analysis features via text format JSON and function calling, enabling automation of various actions within connected apps—such as sending emails, posting online, or making purchases. OpenAI emphasizes the importance of implementing user confirmation flows before executing actions that affect users' environments.

An OpenAI spokesperson stated that these enhancements streamline developers’ workflows, as they previously had to engage separate models for text and images. Now, a single API call allows seamless image analysis and reasoning.

OpenAI showcases several customers leveraging GPT-4 Turbo with Vision, including Cognition, a startup using the model to autonomously generate code, and Healthify, a health and fitness app that offers nutritional analysis and meal recommendations from user-submitted photos. Additionally, UK-based startup TLDraw employs GPT-4 Turbo with Vision to enhance its virtual whiteboard, converting users' drawings into functional websites.

While GPT-4 Turbo has faced competition from newer models like Anthropic's Claude 3 Opus, Cohere's Command R+, and Google's Gemini Advanced in benchmark tests, the rollout of GPT-4 Turbo with Vision aims to attract more enterprise customers and developers. This move positions OpenAI's models as an appealing choice as the industry anticipates the release of its next LLM.