Overcoming Challenges: Why Enterprises Struggle to Operationalize AI at AI Summit NY 2023

Deploying generative AI models poses significant challenges for companies seeking to transition from simple proofs of concept to effective operationalization at scale. According to insights shared by industry experts at the AI Summit New York 2023, overcoming these hurdles begins with identifying roadblocks.

One of the primary challenges is acquiring and curating the right data. Sesh Iyer, managing director and senior partner at BCG, emphasized the critical need for a sustainable data pipeline that includes well-organized metadata. Generative AI models thrive on extensive data input, so companies must refine their knowledge bases to maximize the potential of large language models. Gaurav Dhama, director of product development for AI at Mastercard, also pointed out the importance of optimized data management.

Establishing an effective governance framework to manage the inherent risks associated with generative AI is another significant hurdle. A pervasive “confidence problem” exists among senior leaders regarding the security risks, copyright issues, and potential for AI-generated inaccuracies. Lucinda Linde, senior data scientist at Ironside, highlighted these concerns, indicating that leaders must navigate these risks carefully to embrace the technology fully.

The shortage of skilled professionals capable of leveraging generative AI effectively is another obstacle. Many organizations still grapple with identifying the business value and return on investment (ROI) from their AI initiatives, while the fluctuating costs associated with generative AI contribute to ongoing uncertainty.

Given the nascent nature of generative AI technology, Vik Scoggins, who leads AI/ML product strategy and development at Coinbase, noted, “It is not a paved road yet.” This landscape requires that companies approach generative AI cautiously. Dhama predicts that generative AI will remain in a “copilot phase” for an extended period, particularly in highly regulated sectors like financial services, where human oversight will remain crucial.

Furthermore, security vulnerabilities can arise from using generative AI, especially in coding tasks. As Dhama noted, the expertise of those implementing these tools is paramount. Linde advised companies to start deploying generative AI internally to enhance employee productivity and efficiency, suggesting that initial use in back-office functions can pave the way for broader implementation when confidence within the organization grows.

Despite the challenges associated with adopting new technologies, the potential productivity gains from generative AI are compelling. Iyer estimates that organizations may experience efficiency improvements ranging anywhere from 10% to 90%.

Another critical aspect of effectively utilizing generative AI is diversification in technology use. Linde highlighted the importance of employing multiple generative AI models, despite the prevalence of OpenAI's technology in current applications. The recent upheaval involving OpenAI's CEO, Sam Altman, underscores the risks of dependency on a single provider.

Exploring various models is essential, as different systems excel in different areas. Linde noted that emerging models, such as Mistral, have demonstrated exceptional performance and should be considered as part of a broader strategy. Dhama echoed this sentiment, advocating for a diverse array of systems to enhance resilience.

When designing a generative AI framework, key considerations include accuracy, latency, and cost. To stand out in a marketplace where many organizations are utilizing similar foundational models, panelists underscored that the differentiating factor lies in the quality of the underlying data. As Dhama succinctly put it, “It is in the data, not the model.”

Maximizing the value derived from generative AI requires a strategic intersection of business insights and operational execution, with a strong emphasis on curating the right data. Iyer concluded with a powerful reminder: “If you have the data, you win.”