Stability AI Launches Stable Code 3B to Enhance AI-Powered Code Generation and Fill in Code Gaps

Generative AI Code Generation: Stability AI Unveils Stable Code 3B

Stability AI has enhanced its suite of generative AI tools with the launch of Stable Code 3B, a powerful new model optimized for code generation. This announcement marks the company's first significant release of 2024, building on its reputation for pioneering text-to-image generation with stable diffusion technology.

Compact Yet Powerful

Stable Code 3B, featuring 3 billion parameters, is designed for software development, offering impressive code completion capabilities while running efficiently on standard laptops—no dedicated GPUs required. Despite its smaller size, it competes effectively against larger models, such as Meta's CodeLLaMA 7B.

The trend towards smaller, high-performance models was initiated by Stability AI in late 2023 with releases like StableLM Zephyr 3B for text generation. Stability AI first introduced Stable Code in August 2023 and has continuously enhanced its features.

Advancements in Stable Code 3B

The latest improvements in Stable Code 3B allow the model to not only suggest new lines of code but also fill in larger, incomplete sections, an advanced capability referred to as Fill in the Middle (FIM).

Training optimizations include an extended context size using Rotary Position Embeddings (RoPE), enabling an impressive context length of up to 100,000 tokens. This technique is also utilized by other models, such as Meta's Llama 2 Long.

Built on Stability AI’s Stable LM 3B natural language model, Stable Code 3B was specifically trained on software engineering data. This extensive training, which incorporated code repositories, developer forums, and various technical resources, bolsters its proficiency in 18 programming languages, including Python, Java, JavaScript, Go, Ruby, and C++. Initial benchmarks indicate it matches or surpasses the completion quality of models over twice its size.

Competitive Landscape

The generative AI code generation market is rapidly evolving, with several notable players. Alongside Meta’s CodeLLaMA 7B, the StarCoder LLM, developed collaboratively with IBM, HuggingFace, and ServiceNow, emerges as a primary contender in the 3-billion parameter space. Stability AI asserts that Stable Code 3B outperforms StarCoder in languages like Python, C++, JavaScript, Java, PHP, and Rust.

Subscription Access

Stable Code 3B is available for commercial use through Stability AI's new membership subscription service, initially announced in December. Members gain access not only to Stable Code 3B but also to a suite of AI tools, including SDXL for image generation, StableLM Zephyr 3B for text content, Stable Audio for audio generation, and Stable Video for video generation.

Image Credit: Stability AI