Stanford Report: AI Outpaces Humans in Multiple Areas, Yet Costs Continue to Skyrocket

AI Progress Report 2024: Key Findings and Trends

Artificial intelligence (AI) witnessed significant advancements in 2023, particularly in technical benchmarks, research output, and commercial investments, according to the Stanford University Institute for Human-Centered AI's AI Index 2024 report. However, there are notable limitations and increasing concerns regarding its risks and societal implications.

The AI Index 2024 report provides an in-depth analysis of global AI progress, revealing that AI systems have surpassed human performance in benchmarks such as image classification, visual reasoning, and English comprehension. Yet, they still fall short in complex areas like advanced mathematics, commonsense reasoning, and strategic planning.

Surge in AI Research and Escalating Costs

The report highlights a remarkable increase in AI research and development activity in 2023, predominantly driven by the private sector. Companies released 51 significant machine learning (ML) models, compared to just 15 from academic institutions. Partnerships between industry and academia contributed an additional 21 high-profile models.

The costs associated with training sophisticated AI systems have soared. For instance, OpenAI’s GPT-4 language model consumed approximately $78 million in computing resources, while Google's Gemini Ultra model incurred an astonishing estimated cost of $191 million.

The authors note, "Training costs for state-of-the-art AI models have reached unprecedented levels."

Geographic Dominance in AI Production

The United States remains a leader in AI model development, producing 61 notable systems in 2023. China and the European Union followed with 15 and 21 models, respectively.

Investment trends reveal a mixed landscape. Although overall private AI investment declined for the second consecutive year, funding for generative AI—capable of generating text, images, and other media—surged nearly eightfold to $25.2 billion. Noteworthy companies such as OpenAI, Anthropic, and Stability AI attracted substantial funding.

The report states, "Despite a decline in overall AI investment, the generative AI sector witnessed remarkable growth."

Need for Standardized AI Testing

The rapid advancement of AI raises concerns regarding the absence of standardized testing for system responsibility, safety, and security. Major developers like OpenAI and Google frequently evaluate their models based on varied benchmarks, complicating cross-comparisons.

The AI Index analysis warns, "Robust and standardized evaluations for large language model (LLM) responsibility are critically lacking, complicating efforts to systematically assess the risks and limitations of leading AI models."

Emerging Risks and Public Concern

Emerging risks are highlighted, including the proliferation of political deepfakes, which are "easy to generate and difficult to detect." Additionally, the report uncovers vulnerabilities within language models that may lead to harmful outcomes.

Public sentiment reflects increasing anxiety regarding AI. The proportion of individuals anticipating a "dramatic" impact from AI in the next 3-5 years rose from 60% to 66% globally. Over half of respondents now express discomfort regarding AI products and services.

In the U.S., concerns about the growing role of AI have markedly increased; the share of Americans who feel more apprehensive than excited about AI has jumped from 37% in 2021 to 52% in 2023, while those feeling more excited have decreased to 36%.

The report concludes, "Public awareness of AI's potential impact is growing, accompanied by heightened anxiety." As AI technology evolves, the AI Index aims to deliver objective insights to help policymakers, business leaders, and the public navigate the complexities and opportunities ahead.