In the era of generative AI, chatbots have become more prevalent than ever—yet are they truly more helpful and accurate?
Today, Vectara, a platform builder for generative AI, unveiled a new module designed to enable enterprises to create and implement highly accurate chatbots. Utilizing a Retrieval Augmented Generation (RAG) method paired with Boomerang vector embeddings, Vectara ensures the delivery of real-time information while minimizing the risk of hallucinations. Since its emergence from stealth in October 2022, Vectara has progressively enhanced its platform to align with user needs, and the introduction of this chat module marks a significant extension of its capabilities.
“When you use our chat feature, it leverages RAG to generate your responses,” said Tallat Shafaat, co-founder and Chief Architect at Vectara. “The answers come from your own documents, enhancing accuracy.”
Modern Chatbots: Beyond Q&A to Conversational AI
What sets Vectara's new Chat module apart from its previous offerings? According to Amr Awadallah, co-founder and CEO of Vectara, the key lies in scalability and conversation persistence.
Historically, Vectara's API focused primarily on Q&A interactions, where users would ask a question and receive an answer. If users had follow-up questions, they needed to restate their original inquiry due to Vectara’s stateless session approach, which did not retain conversational history. This limitation often required customers to build their own persistence layers for stateful conversations. However, the new Vectara Chat module changes this by integrating persistent memory, allowing the platform to track conversation history seamlessly. “This new extension enables our API to maintain conversation history, so there's no need to rephrase previous queries,” Awadallah stated.
For deployment, Vectara offers both an API and simple widgets for organizations to easily integrate the chat module into their websites or applications using just a few lines of JavaScript and HTML.
Looking ahead, Shafaat noted plans to enhance Vectara Chat with additional enterprise management features. Account owners will be able to analyze customer chat histories semantically, gaining insights into user sentiments and query trends. The RAG-based query functionality will also facilitate inquiries into user chats.
Addressing Hallucinations and Bias in Generative AI
A significant challenge in using generative AI in enterprises is the risk of hallucination. Vectara’s RAG approach is among several strategies designed to mitigate the likelihood of inaccurate responses in its chat module.
Awadallah emphasized that Vectara's system offers explainable responses with citations, enhancing accuracy. The platform also incorporates bias mitigation through a pioneering approach known as maximal marginal relevance.
“Maximal marginal relevance increases the diversity of results we return,” he explained.
Awadallah added that for debatable topics with varying opinions, a robust algorithm is crucial for presenting multiple perspectives. “We ensure that we capture the primary views as well as secondary opinions, even if they are less relevant,” he stated.
By focusing on accuracy, diversity, and user engagement, Vectara aims to elevate the role of chatbots in enterprise communication.
Ranking
