AI models constantly astonish us, not just with their capabilities, but also with their limitations—and the reasons behind them. One intriguing behavior that highlights this is how these systems select random numbers, a process that mimics the often flawed methods of humans.
But what does this really mean? Can’t people choose numbers randomly? How can we determine whether someone is picking numbers successfully or not? This is an age-old limitation human beings face: we have a tendency to misinterpret and overthink randomness.
Consider the challenge of predicting the outcomes of 100 coin flips. When we compare these predictions with 100 actual flips, it’s easy to identify the difference. Surprisingly, real coin flips tend to seem less random. You’ll often find sequences where there are six or seven heads or tails in a row—something most human predictors don’t include.
The same pattern holds true when asked to choose a number between 0 and 100. People rarely select 1 or 100, and choose few multiples of 5, or numbers like 66 and 99 that have repeating digits. Instead, most gravitate toward numbers around the middle, often opting for those that end in 7.
There’s an abundance of psychological research illustrating this predictability, yet the fact that AI models exhibit similar behavior is still surprising.
Recently, engineers at Gramener conducted an intriguing experiment. They tasked several leading language models (LLMs) with selecting a random number between 0 and 100. The results? Far from random.
Each of the three tested models consistently favored specific numbers even when parameters were adjusted for greater variability. OpenAI’s GPT-3.5 Turbo showed a strong preference for 47, previously favoring 42—a number popularized by Douglas Adams as “the answer to life, the universe, and everything.”
Anthropic’s Claude 3 Haiku also opted for 42, while Gemini displayed a preference for 72.
What’s particularly fascinating is that all three models exhibited human-like biases in their selection of other numbers, even under high variability conditions. They generally avoided low and high numbers, with Claude never exceeding 87 or dropping below 27, and even those were anomalies. Double-digit numbers like 33, 55, or 66 were largely ignored, with only 77 making an appearance (and ending in 7). Round numbers were similarly scarce, although at maximum variability, Gemini unexpectedly chose 0.
Why does this happen? AIs aren’t human! What prompts them to mimic what “seems” random? Have they achieved consciousness and this is their way of revealing it?
The reality, however, is simpler. We’re anthropomorphizing these systems too much. These AI models don’t possess any concept of randomness; they don't "care" about it. Instead, they generate responses by analyzing their training data, repeating what appeared most frequently in response to prompts resembling “pick a random number.” Thus, the more frequently a number appears, the more likely the model is to produce it.
In their training data, numbers like 100 are rarely chosen, making it an unlikely response. With no genuine reasoning ability or comprehension of numbers, these models function as stochastic parrots. Interestingly, they struggle with basic arithmetic as well—how frequently will you find a precise multiplication phrase like “11289432=3,204,096” in their training data? While newer models can recognize math problems and direct them to appropriate subroutines, many still lag in performance.
This scenario serves as a reminder about the behaviors of large language models and the semblance of humanity they depict. It’s crucial to remember that these systems are trained to replicate human interaction, often unintentionally leading to anthropomorphic interpretations.
The headline may suggest that these models “think they’re people,” but that’s misleading. They don’t think in any capacity. Their responses, however, mimic human behavior, without the need for actual cognition. Whether you're seeking a chickpea salad recipe, financial advice, or a random number, the mechanism is the same. The outputs feel human because they are drawn from human-created content and reprocessed—offering convenience while also benefiting the broader AI industry.
Ranking
