Google Introduces Two New Open-Source Large Language Models (LLMs)

Just a week after unveiling its latest Gemini models, Google has introduced a new family of lightweight open-weight models called Gemma. Starting with Gemma 2B and Gemma 7B, these models are designed for both commercial and research applications and are “inspired by Gemini.”

While Google hasn't released a detailed performance comparison against competing models from Meta and Mistral, it describes the Gemma models as “state-of-the-art.” These models utilize a dense decoder-only architecture, similar to that of the Gemini and earlier PaLM models. Performance benchmarks will be available later today on Hugging Face’s leaderboard.

For developers eager to explore Gemma, Google provides ready-to-use Colab and Kaggle notebooks, along with integrations with platforms like Hugging Face, MaxText, and Nvidia’s NeMo. Once pre-trained and fine-tuned, these models are versatile enough to run in various environments.

It’s essential to clarify that while Google promotes these models as "open models," they are not open source. In a press briefing prior to the launch, Google’s Jeanine Banks emphasized the company's commitment to open models but noted a careful distinction in terminology. “The term '[open models]' has become quite common in the industry,” Banks stated. “It often refers to models with open weights, providing broad access for developers to customize and adjust the models. However, the terms of use—covering redistribution and ownership of customized variants—vary based on each model’s terms. Therefore, we consider it more accurate to refer to our Gemma models as open models.”

Developers can leverage these models for inference and fine-tuning as needed. Google's team argues that the smaller model sizes are well-suited for many use cases. “The quality of generated output has significantly improved over the last year,” said Tris Warkentin, product management director at Google DeepMind. “Tasks that once required large models are now feasible with smaller, state-of-the-art models. This paves the way for innovative AI application development, allowing inference and tuning on local developer desktops or laptops equipped with RTX GPUs, or on single-host setups in Google Cloud with Cloud TPUs.”

The same applies to open models from Google’s competitors, so it will be interesting to see how Gemma models perform in real-world applications. Alongside the new models, Google is also launching a responsible generative AI toolkit designed to provide essential guidance and tools for creating safer AI applications with Gemma, along with a debugging tool.