Exclusive: Kong Unveils AI Gateway for Enterprises to Govern and Scale Generative AI Solutions

Kong Inc., a leader in cloud API technologies, has launched the Kong AI Gateway, an AI-native API gateway designed to govern and secure generative AI workloads across any cloud environment. Following its beta release in February, the product has gained considerable traction as organizations strive to operationalize generative AI.

The Kong AI Gateway includes a comprehensive suite of infrastructure capabilities tailored for AI, such as support for multiple large language models (LLMs), semantic caching, semantic routing, semantic firewalling, and model lifecycle management. It seamlessly integrates with Kong’s existing API platform, allowing enterprises to manage AI alongside traditional APIs.

“Organizations and developers are creating innovative generative AI use cases to enhance user and customer experiences. However, scaling this cutting-edge technology in production requires robust infrastructure,” explained Marco Palladino, Kong’s CTO and co-founder.

Palladino asserts that the Kong AI Gateway represents one of the most advanced AI infrastructure technologies available, leveraging Kong’s established gateway features while providing extensive AI-specific functionalities. He likened this evolution to the early days of APIs, which gained traction once suitable infrastructure was in place to support developers.

Serving as a central hub, the Kong AI Gateway provides a unified API interface for managing and securing diverse AI technologies across applications. Its extensive capabilities encompass governance, observability, and security, enabling enterprises to deploy and scale generative AI initiatives effectively.

A significant advantage of the Kong AI Gateway is its ability to introspect AI traffic and create a unified API for accessing multiple AI providers. “Unlike other API management platforms that treat LLM APIs as standard APIs, Kong elevates the approach,” Palladino noted.

The gateway offers robust features such as prompt security, compliance, governance, templating, and lifecycle management around AI prompts. Additionally, it provides "L7 AI observability metrics," offering insights into provider performance, token usage, and costs, a level of detail that Palladino claims competitors lack.

Kong’s unified control plane, Kong Konnect, facilitates the monetization of fine-tuned AI models alongside traditional APIs. “The next frontier in API monetization is the models themselves,” Palladino explained. “Fine-tuning models with proprietary corporate intelligence holds significant value.”

The launch arrives amid a surge of interest in generative AI, particularly following the success of OpenAI’s ChatGPT. Yet, many enterprises face challenges in effectively deploying and governing the technology. Kong aims to simplify this process with its specialized AI gateway.

“Every enterprise was eager to replicate what we have created,” Palladino noted, citing overwhelming demand as the catalyst for accelerating the product to general availability. “We already have enterprise customers using this capability in production, even with the beta version."

Originally founded as Mashape in 2009 and rebranded as Kong in 2017, the company has raised over $170 million in venture funding and supports trillions of API transactions for more than 500 global organizations. With its strategic focus on AI-native infrastructure, Kong is well-positioned to drive the next wave of generative AI adoption in enterprise settings.

“To unlock generative AI at scale, deploying the right AI infrastructure is critical,” Palladino concluded. “Kong AI Gateway is precisely that solution.”