Aug 21, 2024
Light bulbs have energy ratings — so why can’t AI chatbots?
The Mixture of Experts (MoE) framework has become a popular architecture for large language models due to its superior performance over dense models. However, training MoEs from scratch in a large-scale regime is prohibitively expensive.
Authors
Sasha Luccioni, Boris Gamazaychikov, Sara Hooker, Régis Pierrard, Emma Strubell, Yacine Jernite & Carole-Jean Wu
Abstract
The rising energy and environmental cost of the artificial-intelligence boom is fuelling concern. Green policy mechanisms that already exist offer a path towards a solution.
Related works
