This week, Gartner released a report indicating that nearly one-third of generative AI projects within enterprises may be shelved after the proof-of-concept phase by the end of 2025. The challenges prompting this trend range from poor data quality and insufficient risk management to rising infrastructure costs.
One of the most significant obstacles hindering generative AI adoption is the ambiguous business value it offers. According to Gartner, organizations eyeing widespread implementation of generative AI face substantial investments, with costs ranging from $5 million to an astonishing $20 million. For instance, a straightforward coding assistant might require an initial outlay between $100,000 and $200,000, along with recurring expenses exceeding $550 per user annually. In contrast, an AI-powered document search tool could demand an upfront investment of $1 million, with yearly costs per user ranging from $1.3 million to $11 million.
These hefty price tags hinder corporate buy-in when the anticipated benefits are challenging to measure and may take years, if ever, to materialize. According to a recent survey conducted by Upwork, rather than boosting productivity, AI has become an added burden for many workers. The survey, covering 2,500 executives, full-time employees, and freelancers, reveals that nearly half (47%) of AI users are unsure how to achieve the productivity improvements expected by their employers. Furthermore, over 75% (77%) feel that AI tools have diminished productivity and increased their workload in at least one respect.
The initial enthusiasm for AI appears to be waning, even amidst robust venture capital activity. This shift is hardly surprising, as numerous anecdotes illustrate that generative AI often presents more challenges than solutions due to unresolved fundamental technical shortcomings.
Just this Tuesday, Bloomberg reported on a Google-powered tool currently being tested in Florida's HCA hospitals that uses AI to analyze patient medical records. However, users expressed frustration, as the tool reportedly struggles to provide reliable health information; one user noted that it failed to indicate whether a patient had any drug allergies.
As expectations for AI rise, it’s vital for vendors to manage these expectations responsibly, especially without significant research breakthroughs to address the technology's critical limitations.
News Highlights
- SearchGPT: OpenAI recently introduced SearchGPT, a new search feature aimed at delivering “timely answers” to user queries by leveraging web resources.
- Bing's AI Boost: Not wanting to be left behind, Microsoft previewed its AI-infused search experience known as Bing generative search, currently accessible to a limited number of users. Similar to SearchGPT, Bing generative search compiles information from across the internet and generates comprehensive summaries in response to searches.
- X's Data Opt-In: X (formerly Twitter) has made a controversial change by defaulting user data into the training pool for its chatbot Grok, a shift that raised eyebrows among users and attracted swift criticism from EU regulators. (For those wanting to opt out, here's a helpful guide.)
- EU AI Consultation: The European Union has initiated a consultation process to establish guidelines for providers of general-purpose AI models under the AI Act, which will regulate AI applications based on risk.
- Perplexity's Publisher Licensing: The AI search engine Perplexity plans to share advertising revenue with news publishers when its chatbot displays their content in response to queries, a measure to counteract criticism regarding copyright infringement and unethical scraping.
- Meta's AI Studio Launch: Meta announced the rollout of its AI Studio tool to all U.S. creators, allowing them to craft personalized AI-driven chatbots. The company first introduced AI Studio last year and began testing it with select content creators in June.
- Commerce Department Report: On Monday, the U.S. Commerce Department released a report supporting “open-weight” generative AI models like Meta’s Llama 3.1, while advising the government to develop “new capabilities” to monitor potential risks associated with such models.
- Avi Schiffmann's $99 Companion: Harvard dropout Avi Schiffmann is developing a $99 AI-powered device called Friend—a neck-worn gadget designed to serve as a companion. However, its effectiveness remains uncertain.
Research Paper of the Week
Reinforcement learning from human feedback (RLHF) is the standard method for ensuring generative AI models comply with instructions and safety protocols. However, this approach demands extensive resources to gather feedback, making it time-intensive and costly.
To address these challenges, OpenAI is exploring alternative methods. In a recent paper, OpenAI researchers introduced rule-based rewards (RBRs), utilizing a structured set of rules to evaluate and direct a model’s responses. RBRs break down desired behaviors into specific guidelines, fostering the creation of a “reward model” that educates the AI on appropriate responses in various scenarios.
OpenAI asserts that models trained with RBRs exhibit superior safety performance compared to those relying solely on human feedback, significantly reducing the need for vast amounts of human input. According to the company, RBRs have been a part of its safety stack since the launch of GPT-4 and will be included in forthcoming models.
Model of the Week
Google's DeepMind is making strides in developing AI systems capable of solving complex mathematical challenges. Recently, DeepMind revealed that it successfully trained two AI systems—AlphaProof and AlphaGeometry 2 (the successor to AlphaGeometry)—to solve four out of six problems presented at this year’s International Mathematical Olympiad (IMO), a prestigious high school math competition.
AlphaProof and AlphaGeometry 2 worked collaboratively to tackle two algebra problems and one number theory problem, marking the first time AI has achieved silver medal-level performance in IMO challenges. However, there are caveats, as some problems took days for the models to resolve. While their reasoning abilities are remarkable, these systems struggle with open-ended problems that involve multiple potential solutions.
We look forward to what future advancements may bring!
Grab Bag
AI startup Stability AI has launched a generative AI model, Stable Video 4D, designed to convert video footage of an object into various clips that simulate different angles.
This innovative model could have significant applications in game development, video editing, and virtual reality, according to Stability. Users can upload footage and specify desired camera angles; within about 40 seconds, the model produces eight five-frame videos, although optimization may extend processing time by an additional 25 minutes.
Stability AI is actively refining the model, aiming to enhance its ability to process a wider array of real-world videos beyond its original synthetic dataset training. The potential for creating realistic, multi-angle videos is vast, and the company is excited to continue its research and development in this area.
Ranking
