Researchers upend AI status quo by eliminating matrix multiplication in LLMs

Researchers have developed a new method to enhance the efficiency of AI language models by eliminating matrix multiplication, a process crucial for neural network operations accelerated by GPU chips. The study, led by researchers from various universities and tech companies, introduces a MatMul-free approach that could significantly reduce power consumption and operational costs of AI systems. By creating a custom 2.7 billion parameter model without using MatMul, the researchers achieved similar performance to traditional large language models (LLMs). They demonstrated running a 1.3 billion parameter model on a GPU accelerated by a custom-programmed FPGA chip, consuming only 13 watts of power. This breakthrough challenges the conventional belief that matrix multiplication is essential for high-performing language models, potentially making large models more accessible and sustainable, especially for deployment on resource-constrained devices like smartphones. The technique, although not yet peer-reviewed, aims to pave the way for more efficient and hardware-friendly AI architectures, offering a promising alternative to current GPU-intensive approaches.