There isn't a single large language model (LLM) that fits all needs, a conclusion drawn from a recent survey of enterprise IT leaders conducted by IBM.
This insight is part of the IBM Institute for Business Value's report titled “The CEO’s Guide to Generative AI: AI Model Optimization.” The report, developed in collaboration with Oxford Economics, aims to equip CEOs with actionable insights for effective AI implementation and optimization within their organizations. It also reveals intriguing perspectives on how enterprise AI adoption is unfolding in practice.
Key Findings from the Report:
- Model Specialization: The data challenges the notion of a universal AI model, underscoring the importance of selecting task-specific models.
- Model Diversity: Organizations currently utilize an average of 11 AI models and anticipate a 50% increase in usage over the next three years.
- Cost Barriers: 63% of executives identify model costs as the primary hurdle to adopting generative AI.
- Model Complexity: 58% express concern over the complexity of AI models.
- Optimization Techniques: Techniques such as fine-tuning and prompt engineering can enhance model accuracy by 25%, yet only 42% of executives apply these methods consistently.
- Open Model Growth: Enterprises expect a 63% rise in the adoption of open models over the next three years, outpacing proprietary alternatives.
“Enterprise technology leaders are well-informed about today's AI models and recognize their strengths and limitations for specific use cases,” remarked Shobhit Varshney, VP and senior partner at IBM Consulting, in an exclusive interview. “However, many C-suite leaders are still learning the capabilities and limitations of LLMs, often envisioning a single large AI model that can manage diverse tasks.”
Optimizing AI Cost Efficiency
Cost efficiency is a critical concern for any enterprise IT initiative, particularly concerning generative AI models. Varshney pointed out that various factors influence the cost of enterprise AI models. Organizations can either host models internally, incurring costs for compute and storage, or rely on cloud providers, who typically charge based on input and output tokens consumed.
The report advocates for a balanced approach, suggesting that enterprises deploy large models for complex, high-stakes tasks requiring broad knowledge and high accuracy while considering niche models for specialized, efficiency-driven applications. “Larger models can deliver excellent performance out of the box, but investing in fine-tuning smaller models can yield comparable results,” Varshney stated. “Before starting an AI project, enterprises must assess the potential business impact of the use case and compare the incremental costs of using LLMs against traditional AI options.”
The Importance of Open Models in Enterprise AI
The study highlights a strong preference among IT leaders for open models over closed alternatives in generative AI. This preference is reinforced by recent advancements in open models, such as Meta’s Llama 3.1 and Mistral’s Large 2, which have begun to outperform proprietary counterparts in benchmarks.
Varshney emphasized the advantages of the open model community and security for enterprise AI. “Open models benefit from community input, which strengthens AI systems. Enterprises can also customize these models for their specific domains and use cases.”
While the shift towards open models is evident, Varshney stresses that companies should develop an AI strategy before selecting models. IBM Consulting assists clients in identifying processes and use cases where AI could yield significant benefits, such as customer service, IT operations, and back-office functions like HR and supply chain. Once a use case is prioritized, IBM can break down the workflow and integrate the appropriate technology—be it automation, traditional AI, or generative AI.
“If generative AI is identified as the right solution, factors such as task complexity, cost constraints, required accuracy, response latency, compliance auditability, and context must be considered to select the optimal model,” he advised. “By aligning models with specific tasks and business process constraints, organizations can create a well-rounded AI strategy.”
Ranking
