Stability AI Unveils 12B Parameters in Exciting Update to Stable LM 2 Model

Stability AI is keeping up its swift pace of innovation with the introduction of the Stable LM 2, a 12-billion-parameter model launched today.

While Stability AI is widely recognized for its text-to-image technology, Stable Diffusion, the company encompasses a broader array of generative AI models beyond just image generation.

Stable LM debuted in April 2023 as a large language model (LLM) for text content, receiving its first update with the 1.6 billion parameter Stable LM 2 in January. The new Stable LM 2 12B substantially enhances Stable LM's capabilities, boasting more parameters and improved performance that, according to the company, outmatches larger models like Llama 2 70B on specific benchmarks.

The Stable LM 2 12B features both a base version and an instruction-tuned variant aimed at refining conversational skills in seven languages: English, Spanish, German, Italian, French, Portuguese, and Dutch. These models are available for commercial use through a Stability AI membership, the company's revenue model.

“The instruction model is specifically designed to engage users in a conversational manner,” stated Carlos Riquelme, head of Stability AI’s language team. “Additionally, we have invested significant effort in enhancing safety.”

This update follows the recent resignation of co-founder and CEO Emad Mostaque amid management controversies, but it demonstrates the company's unwavering momentum under interim co-CEOs Shan Shan Wong and Christian Laforte.

Stability AI aims to blend performance with accessibility through Stable LM 2, claiming that the 12B model finds an ideal balance between power, accuracy, and usability. With its 12 billion parameters, it can tackle tasks typically reserved for much larger models. Benchmark results indicate that Stable LM 2 12B holds up remarkably well against significantly larger counterparts.

Importantly, the methodology employed to develop the smaller 1.6B model was also applied in creating the new 12B version.

“We envision a future where models function not in isolation but as part of complex systems, enabling interactions among multiple language models along with external software tools,” Riquelme explained. “The 12B model has been trained to fulfill a ‘master’ role, connecting with and utilizing various functions and APIs relevant to diverse user needs.”

Stability AI is also enhancing the previously released 1.6B model. Riquelme pointed out that while the smaller model can effectively handle specific tasks, the larger 12B offers greater capability. Nonetheless, the updated Stable LM 2 1.6B has improved its conversational skills, increased safety measures, and enhanced compatibility with other software tools.

“It retains its previous size, ensuring it remains fast and lightweight,” Riquelme noted. “Conversely, the 12B model is more powerful and reliable but demands greater computational resources.”

He emphasized that depending on use cases and resource availability—such as response time, memory, and budget—each model presents unique trade-offs.

“We believe there's no single optimal model size for every scenario, which is why we provide two distinct models,” he said.