In the wake of every technological breakthrough, the initial buzz eventually fades, making room for a more measured analysis of both the costs and benefits. For many AI leaders, this shift in perspective has sparked more questions, particularly around the challenge of measuring the return on investment (ROI).

Conventional ROI metrics are no longer enough. So, how do we quantify the impact of such a transformative technology?

To answer this question, we must first unpack a couple of other metrics.

How do we count the cost?

Generative AI (GenAI) is no small investment. Between the high costs of GPUs, cloud infrastructure, data storage, and training and fine-tuning models, the technical costs of GenAI production at scale can be staggering. A Goldman Sachs report projected that businesses will invest around $1 trillion in AI over the coming years, underscoring the financial commitment required.

In addition to these steep technical costs, GenAI brings a unique set of uncertainties.

While the initial promise of GenAI was its potential to drive substantial productivity gains, realizing those benefits hasn’t been as straightforward as anticipated. Identifying the right use case and scaling solutions demands more than just deploying models – it requires a clear understanding of target goals, widespread workforce adoption, continuous model refinement, infrastructure expansion, and significant computing power.

Despite these challenges, the potential productivity gains from AI solutions remain promising. While some predictions, like MIT’s Daron Acemoglu, suggest a modest 0.5% increase in U.S. productivity over the next decade, others offer a more optimistic outlook.

Goldman Sachs senior global economist Joseph Briggs forecasts a 9% productivity gain, largely due to new tasks and business models that AI can enable, aligning with historical trends seen with other disruptive technologies.

The debate over AI’s ROI ultimately hinges on this balance: can the productivity gains from its use cases and applications outpace the rising technical costs of production at scale? This remains the key question for businesses looking to harness GenAI’s full potential.

What are the benefits?

The real but often hard-to-measure benefits of GenAI range from reducing or removing the burden of trivial tasks to improving decision making. We’re only in the infancy of agentic AI, which organizations can use to automate complex, multi-step workflows, which many experts believe will unlock billions of dollars in new value. Then, there are external-facing uses such as customer-service agents, personalized recommendations, and multilingual support.

It’s not surprising that it can be a challenge to plug the value of this output into conventional measures of ROI.

Some organizations are finding ways to quantify the gains. Amazon, for example, recently announced that adopting GenAI had saved its developers more than 4,500 years spent on work, which amounts to up to $260 million a year.

The Federal Reserve sees this happening on a wider scale. It has released a study suggesting that GenAI adoption is proceeding at twice the pace of the Internet and that this could boost long-stagnant productivity.

“AI, and generative AI in particular, is likely to become a general-purpose technology—one that spreads throughout the economy, sparks downstream innovation and continues to improve over time,” said Fed Governor Lisa Cook.

What ROI metrics matter?

From CFOs to AI product leads, executives are facing rising pressure from investors to embrace the technology while they're being pressed to illustrate its benefits, whether it’s more revenue, improved productivity, or better talent retention.

And it’s not always clear which metric should come first.

A recent survey, for example, found that productivity and employee satisfaction were the top AI ROI priorities for business leaders in Q1 of 2024. However, by Q2 those priorities had fallen away and been replaced by revenue and decision-making.

The right ROI metrics to track could and should vary widely between businesses. As Louise Herring, a partner at McKinsey, points out, organizations must be clear about the value they're trying to achieve and target metrics accordingly. Those metrics, whether focused on user engagement and adoption, scaling, or time savings, need to be constantly reassessed.

“Companies should have different success indicators for each phase and ensure that there’s room for course correction along the way,” Herring says.

She says companies need to target specific problems with their tools to really make an impact with this technology. This strategy is paying off for a growing number of businesses.

Seeing results

As an example of targeting problems this way, the IT management company Atomicwork discovered that its customers were struggling to align a wide range of different applications. This created unnecessary complexity, forcing IT teams to spend about 50% of their time on repetitive tasks.

Atomicwork identified a specific problem – the need to integrate knowledge and search functions into the collaboration tools already being used by employees – then found a specific solution: a tailored GenAI tool that provides accurate and fast answers, enhancing operational efficiency.

CPA and consulting firm, Johnson Lambert, aimed their AI solution at optimizing audit processes. With their targeted approach, they were able to achieve a 50% reduction in audit hours, allowing teams to focus on high-impact solutions for clients.

Conclusion

Away from the noisy headlines and speculation, there are a growing number of companies that are delivering solutions for once-intractable problems using GenAI.

They’re seeing clear gains in productivity, more engaged employees, and an improved competitiveness that are positioning them as industry leaders in the years to come.

Mastering private AI deployments

Is your business ready for a secure, private AI deployment?

Download the eBook

Ryan Lewis