Matrix Multiplication-Free Language Models: A Breakthrough in Efficiency
Matrix multiplications (MatMul) are the most computationally intensive operations in large language models (LLMs) that utilize Transformer architecture. As these models expand to larger sizes, the costs associated with MatMul operations significantly escalate, resulting in increased memory usage and latency during both training and inference.
Researchers from the University of California, Santa Cruz, Soochow University, and the University of California, Davis, have developed an innovative architecture that eliminates matrix multiplications from language models while delivering robust performance at scale.
Introducing MatMul-Free Language Models
In their groundbreaking paper, the researchers present MatMul-free language models that match the performance of state-of-the-art Transformers but require considerably less memory during inference.
Understanding Matrix Multiplication in Deep Learning
Matrix multiplication is essential in deep learning for combining data with weights in neural networks, facilitating the transformation of input data to generate predictions. GPUs excel in executing numerous MatMul operations simultaneously due to their parallel architecture, which is crucial for efficiently training and deploying complex models.
Despite this advantage, as LLMs grow to include hundreds of billions of parameters, MatMul operations become bottlenecks, necessitating vast GPU clusters for training and inference. Transitioning away from MatMul could lead to substantial savings in memory and computation. However, past attempts to substitute MatMul operations have yielded inconsistent results, often slowing down processes due to suboptimal performance on GPUs.
Revolutionizing Operations with Ternary Weights
The researchers propose an exciting alternative: replacing traditional 16-bit floating-point weights in Transformers with 3-bit ternary weights that can represent three states: -1, 0, and +1. They introduce additive operations to replace MatMul, leading to significant reductions in computational costs. Their models employ “BitLinear layers” utilizing these ternary weights.
“By constraining the weights to the set {−1, 0, +1} and applying additional quantization techniques, we have replaced MatMul with addition and negation operations,” the researchers explain.
Innovative Architectural Changes
The architecture differs fundamentally from traditional Transformers, which comprise token and channel mixers. The token mixer, responsible for integrating information across sequence tokens using self-attention mechanisms, transitions to a MatMul-free Linear Gated Recurrent Unit (MLGRU). The MLGRU processes tokens by updating hidden states through simple ternary operations, bypassing expensive matrix multiplications.
Additionally, the channel mixer, which integrates information across different feature channels of a token's representation, employs a modified Gated Linear Unit (GLU) that accommodates ternary weights. This adjustment minimizes computational complexity and memory usage while maintaining effective feature integration.
“By combining the MLGRU token mixer with the GLU channel mixer using ternary weights, our architecture relies solely on addition and element-wise products,” the researchers note.
Performance Evaluation of MatMul-Free Language Models
The researchers contrast their MatMul-free LMs against the advanced Transformer++ architecture, as utilized in Llama-2, across various model sizes. Their findings indicate that the MatMul-free LM effectively utilizes additional computational resources to enhance performance compared to Transformer++.
In evaluating language tasks, the 2.7B MatMul-free LM exceeded the performance of its Transformer++ counterpart on benchmarks like ARC-Challenge and OpenbookQA, while achieving comparable results in other tasks.
“These results demonstrate that MatMul-free architectures can deliver strong zero-shot performance across diverse language tasks, including question answering and commonsense reasoning,” the researchers assert.
Lower memory consumption and latency for MatMul-free LMs become more apparent with increasing model sizes. For instance, the 13B model requires only 4.19 GB of GPU memory with a latency of 695.48 ms, whereas the Transformer++ demands 48.50 GB at a latency of 3183.10 ms.
Optimized Implementations and Future Directions
The researchers developed an optimized GPU implementation and custom FPGA configuration for MatMul-free language models. With this optimization, they achieved a 25.6% acceleration in training and up to 61.0% reduction in memory use compared to an unoptimized baseline.
“This work transcends software-only implementations of lightweight models, demonstrating that scalable and efficient language models can effectively reduce computational demands and energy consumption,” the researchers conclude.
Although constraints limited testing on models exceeding 100 billion parameters, the researchers hope to encourage institutions to invest in lightweight models, paving the way for more accessible language models independent of high-end GPUs. The researchers have made their code and models available for the research community.
“By prioritizing the development of MatMul-free architectures, the future of LLMs will trend toward greater accessibility, efficiency, and sustainability,” the researchers advocate.
Ranking
