H2O AI Unveils Danube: A Super-Tiny LLM Optimized for Mobile Applications

Today, H2O AI—dedicated to democratizing AI through open-source and proprietary tools—announced the launch of Danube, an ultra-light large language model (LLM) designed specifically for mobile devices.

Named after Europe's second-largest river, this open-source model boasts 1.8 billion parameters and is reported to match or even exceed the performance of similarly sized models across various natural language tasks. This positions it alongside notable competitors like Microsoft, Stability AI, and Eleuther AI.

The timing of this announcement aligns seamlessly with the growing interest among enterprises developing consumer devices to leverage the potential of offline generative AI. By enabling models to operate locally on devices, users can access quick assistance without relying on cloud-based services.

“We're thrilled to introduce H2O-Danube-1.8B as a portable LLM for mobile devices. The rise of smaller, cost-effective hardware and more efficient training techniques has made modestly-sized models more accessible. We believe H2O-Danube-1.8B will revolutionize mobile offline applications,” said Sri Ambati, CEO and co-founder of H2O.

Key Features of Danube-1.8B LLM

Though recently unveiled, H2O asserts that Danube can be fine-tuned for a variety of natural language applications on compact devices, including common sense reasoning, reading comprehension, summarization, and translation.

To train this model, H2O collected a trillion tokens from diverse web sources and employed advanced techniques drawn from Llama 2 and Mistral models to enhance its capabilities.

“We adapted the Llama 2 architecture to contain around 1.8B parameters. We utilized Llama 2's tokenizer with a vocabulary of 32,000 and trained the model to achieve a context length of 16,384. Additionally, we integrated Mistral's sliding window attention mechanism with a size of 4,096,” the company detailed on Hugging Face.

Benchmarks show that Danube performs comparably or even better than most models within the 1-2B parameter range. For example, in the Hellaswag test for evaluating common sense natural language inference, it achieved an accuracy of 69.58%, trailing only Stability AI’s Stable LM 2, which has 1.6 billion parameters pre-trained on 2 trillion tokens. In the Arc benchmark for advanced question answering, Danube placed third, achieving 39.42% accuracy, following Microsoft Phi 1.5 (1.3 billion parameters) and Stable LM 2.

Tools for Seamless Adoption

Released under an Apache 2.0 license, Danube-1.8B is available for commercial use. Teams interested in implementing this model for mobile applications can download it from Hugging Face and customize it for specific use cases.

To facilitate this process, H2O plans to introduce additional tools shortly. Moreover, a chat-tuned version of the model—H2O-Danube-1.8B-Chat—is also available for conversational applications.

In the long term, the introduction of Danube and similar compact models is expected to boost offline generative AI applications on smartphones and laptops, enhancing tasks such as email summarization, typing assistance, and image editing. Samsung has already taken steps in this direction with the release of its S24 smartphone line.