Stable Diffusion Creator Expands Focus to Language Models

Stability AI, the company behind the innovative Stable Diffusion technology, is making a strategic pivot towards language models as it seeks to revitalize its prospects. The firm has recently launched the first model in its new ‘Stable LM 2’ series, named ‘Stable LM 2 1.6B.’ Although this model comprises 1.6 billion parameters, it boasts significant capabilities for a language model.

Stable LM 2 1.6B was trained on a staggering two trillion tokens over two epochs, utilizing a diverse multilingual dataset that spans seven languages, including English, Spanish, and French. This model is crafted to reduce hardware barriers, promoting a more inclusive environment that encourages “more developers to participate in the generative AI ecosystem.” In various tasks, it surpasses other models with fewer than 2 billion parameters, outperforming popular smaller systems like Microsoft’s Phi-1.5, TinyLlama 1.1B, and Falcon 1B.

Stability AI aims to empower developers and model creators to innovate and refine their projects rapidly by offering one of the most powerful small language models to date and providing complete transparency about its training process. The company has introduced both a base model and an instruction-tuned variant. Additionally, it has released detailed data about the pre-training process, including optimizer states, which facilitates seamless pre-training and fine-tuning for developers.

In recent months, Stability AI has increasingly focused on developing language models. The release of the StableLM Zephyr 3B model in December and the initial StableLM model nine months ago marks a new chapter for the company, which is now navigating significant financial challenges. Reports suggest that Stability is under pressure from investors, with rumors of a potential sale to companies like Cohere or Jasper gaining traction. Despite facing high operational costs—running into millions for computing resources and salaries—while generating minimal revenue, Stability AI remains committed to research and development.

By shifting focus to language-based systems, Stability AI not only enhances the capabilities of its existing image and video generation technologies but also positions itself as a competitor in the growing text-based model market.

### Access to Stable LM 2 1.6B

The Stable LM 2 1.6B model is available for both commercial and non-commercial use. However, a Stability AI Membership is required for commercial applications. Memberships for non-commercial use are free, designed for personal and research purposes. For professional creators and developers with less than $1 million in annual revenue or institutional funding—and one million monthly active users—the monthly fee is $20. Those exceeding these criteria are categorized under enterprise level, necessitating negotiations with Stability AI.

Users can explore the model for free via Hugging Face. Nonetheless, it’s important to note that, like many advanced AI systems, Stable LM 2 1.6B might exhibit common challenges such as hallucinations and the potential for generating inappropriate language. Stability AI encourages the community to adopt responsible practices in developing applications using this model, ensuring a safer environment for all users.