"Open Source Dracarys Models Ignite the Future of Generative AI-Powered Coding"

For fans of HBO's Game of Thrones, the term “Dracarys” conjures images of dragons unleashing fire. In the realm of generative AI, however, Dracarys now refers to a groundbreaking family of open large language models (LLMs) designed specifically for coding, developed by Abacus.ai.

Abacus.ai is known for its innovative AI model development platform and has previously named a model after a fictional dragon, releasing Smaug-72B in February. While Smaug serves as a general-purpose LLM, Dracarys focuses on enhancing coding tasks.

The initial rollout of Dracarys employs what Abacus.ai calls the “Dracarys recipe” for its 70 billion parameter models. This method incorporates optimized fine-tuning and various training techniques to enhance coding proficiency.

Bindu Reddy, CEO and co-founder of Abacus.ai, stated, “It’s a combination of training datasets and fine-tuning techniques that improves the coding abilities of any open-source LLM. We’ve shown improvements with both Qwen-2 72B and Llama-3.1 70B models.”

The Growing Landscape of Generative AI for Coding

The market for generative AI in application development and coding is rapidly expanding. GitHub Copilot was an early pioneer, assisting developers with code completion and application tasks. Numerous startups, such as Tabnine and Replit, are also integrating LLM capabilities into their tools.

Dracarys offers an enhanced version of Meta’s Llama 3.1 general-purpose model. In 2024, Anthropic introduced Claude 3.5 Sonnet, another competitive LLM for coding. Reddy remarked, “Claude 3.5 is a strong coding model but it’s closed-source. Our approach empowers open-source models, making Dracarys-72B-Instruct the best in its category.”

Performance Metrics of Dracarys in Coding

The LiveBench benchmarks reveal significant improvements thanks to the Dracarys recipe. The meta-llama-3.1-70b-instruct turbo model scores 32.67, while the Dracarys-tuned version enhances this to 35.23. For the Qwen-2 model, the baseline coding score of 32.38 rises to 38.95 after applying the Dracarys enhancements.

Currently, the Dracarys recipe is applied solely to the Qwen-2 and Llama 3.1 models, but Abacus.ai plans to expand this offering. Reddy confirmed, “We will also release Dracarys versions for Deepseek-coder and Llama-3.1 400B in the future.”

Benefits of Dracarys for Enterprises

Developers and enterprises stand to gain significantly from Dracarys's advanced coding performance. Abacus.ai provides model weights for both the Llama and Qwen-2 models on Hugging Face. Reddy mentioned that these fine-tuned models are also accessible as part of Abacus.ai’s enterprise solutions.

“They are excellent alternatives for enterprises that prefer not to send their data to public APIs like OpenAI and Gemini,” Reddy explained. “We will consider making Dracarys available on our popular ChatLLM service for small teams and professionals if there is enough demand.”