We’ve reached a turning point in what has been a core assumption about AI – that progress in AI will only come from building bigger, more resource-hungry models.

The emergence of models that use smarter, more efficient design rather than “brute force” compute power to achieve high performance is poised to transform the way policymakers and business leaders view and adopt AI solutions. According to one recent AI report, the inference cost (the expense of running a trained model) for a system performing at the level of ChatGPT-3.5 dropped 280-fold in the two years following its release in November 2022.

A few leading enterprise AI innovators have long anticipated this shift away from compute-heavy models, and invested early in architectures that deliver efficiency without sacrificing performance. Going forward, business and technology leaders will need to recalibrate priorities – placing cost-effectiveness, reliability, and strong partnerships ahead of raw power alone. The future belongs not just to the biggest models, but rather the ones that can scale AI securely.

More efficient models will also ease growing concerns about AI’s substantial, and fast-growing, energy footprint and costs, opening the way for faster, more flexible adoption. Lower hardware needs will also help to broaden access to advanced AI, allowing smaller organizations and developing nations to share in the benefits.

In this article, we’ll examine how the AI efficiency paradigm is changing and what it means for government and business adoption.

Hitting the limits of “bigger is better”

The dominant narrative in recent years has been that scaling up compute power is the key to delivering AI progress. This has come at a rising cost in terms of hardware needs and energy consumption, which in turn has narrowed access to the technology and restricted more practical use cases.

The energy-intensive GPU chips needed to power ever-larger models has led to the growth of massive data centers.

Global data center electricity consumption has grown by 20-40% annually to now total between 1 and 1.3% of global energy demand and 1% of greenhouse gas emissions.

For policymakers globally, this has become a major concern. While many see AI as a potential game-changer for private-sector productivity and for governance, its resource-thirsty nature is clashing with other priorities, such as lowering carbon emissions, developing expensive critical infrastructure, and keeping energy costs under control.

For companies, it’s created a difficult trade-off between fully embracing the competitive advantages that generative AI can bring and accounting for the spiraling cost of hardware and energy needed for the most performant models.

At the same time, it’s become clear that simply throwing more compute power at AI models is delivering diminishing returns. The old consensus that AI scaling laws guarantee ever-more powerful LLMs for more compute power is breaking down.

The shift to greater efficiency has parallels in other technologies. Computers that used to fill warehouses now fit in our pockets. Digital cameras used to be bulky and expensive, but now come built into smartphones. It’s these breakthroughs in efficiency, rather than more power, that lead to wider use and greater benefits.

AI has often been likened to a new industrial revolution or a new space race. If so, we are now witnessing the equivalent of a shift from steam engines to electric turbines – a move to more efficient, controllable power. It’s an exciting shift, one that can make AI more accessible and beneficial to all. But it won’t happen automatically; it requires forward-thinking policy and a willingness to break from old assumptions.

The model efficiency wake-up call

The January release of DeepSeek-R1 – which matched the performance of leading models at a fraction of the compute power and cost – woke the world up to a future in which efficiency and innovation will be key to getting more results from AI.

By then, Cohere engineers were already deep into building Command A, which excels at advanced, business-critical tasks while running on just two GPUs, compared to other models that require up to 32 of the expensive, energy-thirsty chips.

These next-generation models point to a future in which leaps in AI performance will come not only from more compute power, but from innovation and smarter applications. New techniques in areas such as model design, pre-training, fine-tuning, and inference are creating significant efficiency gains.

Many AI applications can run perfectly well on smaller models – using the biggest, most powerful models for these use cases is like using a sledgehammer to crack a nut. Going forward, the big gains in value from AI are going to come from more right-sized, tailored solutions that meet organizations' needs for flexibility, security, and cost-effectiveness.

Lower barriers to AI adoption and innovation

This shift will have deep and broad implications for how governments and enterprises implement AI.

Lowering the energy barrier.

More efficient models will allow governments to move forward with national AI frameworks and adoption with less concern about the strain it could put on their countries’ energy infrastructure and climate targets. The rapid growth of energy-intensive AI data centers has led to debate about the need to increase energy production, including through nuclear plants and fossil-fuel sources. U.S. policy officials have warned the country lacks the energy infrastructure to power the proposed $500 billion Stargate project to expand data-center capacity for AI. The head of the U.K. National Grid said data-center power use was predicted to rise six-fold over the next decade, straining energy infrastructure. In addition to easing the strain on energy resources, more efficient AI models will make it easier for companies to comply with regulations around energy use, which could expand. The EU AI Act, for example, requires model providers to document their energy consumption.

Better value, faster adoption.

More efficient models will enable organizations to get more value for their AI running spend, helping to accelerate adoption. AI usage helped drive companies’ cloud computing costs up by an average of 30% last year. Cloud providers typically charge based on computational resources consumed, including processing power, memory, and storage. Companies that opt for private deployments, which give them more security and control, face the need to buy GPUs whose cost has surged to up to $40,000 for the most powerful chips. A model like Command A allows organizations to deploy privately with fewer, more energy-efficient chips.

Wider accessibility.

Lower hardware and energy needs will make advanced AI systems more accessible globally. Organizations such as academic institutions or start-up companies that may have been unable to afford high-end models will be increasingly able to access powerful AI tools. This will foster more innovation and collaboration, as well as lowering barriers to AI in developing countries, many of which lack access to cutting-edge chips helping to unlock new economic and social opportunities.

Expanded edge AI.

The ability to build powerful models with smaller chips will boost the development of edge AI applications – where models are deployed on devices, reducing dependence on cloud infrastructure, improving security, and improving latency. Over time, this will enable a wider range of on-device AI applications in sectors such as healthcare, manufacturing, and retail, where real-time data processing is critical.

This shift to efficiency over size in AI isn’t a passing fad; it’s a long-term trend that demands a new approach from government and enterprise leaders. Those who grasp the opportunity to adopt efficient, practical AI tailored to their unique needs will give themselves a clear long-term advantage.

Building enterprise AI agents

A comprehensive blueprint for developing scalable AI agents in regulated industries.

Download the eBook

Cohere GAPP