If you can’t measure, you can’t improve. That’s where benchmarks come in as a vital part of successful AI adoption for enterprises.

Before deploying, companies need a systematic way to assess a model’s abilities and how well it will adapt to their unique needs and business environment. Dozens of public benchmarks — essentially exam scores for AI — have emerged to measure a model’s abilities and are usually touted as proof of model superiority.

But the true picture of how a model will perform is far more nuanced and heavily dependent on the specific use cases a business has in mind. A model that looks great on a benchmark’s leaderboard may perform poorly in practice if you haven’t tested the skills that matter most to your business and assessed how it performs in real-world conditions. The key for enterprises is to go beyond passively following benchmark scores to become active builders of their own evaluation processes.

To build an AI model that truly meets their needs, companies need to build benchmarks into a broader evaluation process, starting by focusing on a few core goals, and creating a robust testing framework around them that puts a model through its paces in real-world conditions.

What is a benchmark?

In the simplest terms, a benchmark is a test given to AI models to measure their capabilities — along the lines of the SAT or bar exam. Many are taken directly from human tests like the AIME math competition, which gave rise to the AIME benchmark. Others like the ARC-AGI benchmark are developed specifically with AI models in mind. Scores will often include a human baseline, which represents the score a human being would typically get on the test.

Developers typically want to test models at scale, trying out different fine-tunings or subtle variations in approach to see what scores best. Since benchmarks are usually graded in a testing environment without human input, it’s easy to try out dozens or even hundreds of variations; the only limitation is your budget.

Broadly, benchmark scores tend to correlate with each other; a model that scores well on one benchmark is likely to score well on others. This is generally taken as a sign of general intelligence in state-of-the-art AI models. Rather than focusing on specific skills, today’s foundation models are focused on general-purpose intelligence that can be put to any number of uses. But benchmarks that test specific skills are still an important tool to focus a model’s general intelligence on the specific abilities you need for your deployment.

Where benchmarks can be misleading

Keep in mind, benchmarks are only a window into a model’s overall performance, and it’s important to understand their limitations. For instance, public benchmark scores typically reflect a model’s peak performance. As a result, those scores are poor measures of consistency or worst-case behavior, which are important factors for practical deployments. Another risk is that models may learn to optimize for high benchmark scores but then underperform in more “messy” real-world use cases, a problem known as “overfitting.”

Even reliability percentages can be misleading. Consider a customer service assistant with a 5 percent error rate. If that meant that one in 20 customers would be transferred to a human backup, it might be an acceptable error rate. But if it meant issuing invalid refunds at that frequency, it would clearly be unacceptable. The only way to avoid that kind of risk is to go beyond the simple number and assess it in the context of real business conditions.

Simply measuring accuracy isn't enough because developers can inflate those scores by selecting the best answer from multiple attempts, which raises costs. Therefore, many benchmarks plot accuracy against cost to evaluate the efficiency of a model. The result is a two-dimensional leaderboard, like the ARC-AGI benchmark.

But a benchmark isn’t just a measure; it’s also a tool. Training against specific benchmarks will help produce a model specialized in the skills you need. If your deployment requires strong mathematical reasoning, for instance, the model should be trained against a math-focused benchmark, using tuned versions that do particularly well in that setting. It’s therefore critical to identify the specific capabilities that are necessary for your AI deployment, and to use benchmarks that reflect those priorities.

Choosing the right benchmarks

Once you can measure a skill, you can optimize for it. That’s particularly true when training AI models, since they come with trillions of parameters that can be subtly tweaked to improve performance. Benchmarks are a key tool in that optimization process, since a higher benchmark score is the primary indication that your optimizations are working.

Below are a few of the most important capabilities for enterprise use cases, along with the specific benchmarks that are most often used to optimize for them.

Retrieval-augmented generation (RAG)

RAG systems present unique benchmarking challenges because they are made up of two distinct processes:

the retrieval of relevant documents based on a query,
and the generation of a response based on that material.

For this reason they are often evaluated using separate benchmarking techniques. The MTEB benchmarking suite, for example, tests the retrieval performance of embeddings in finding the most relevant information. On the generation side, question answering (QA) datasets like TechQA can be used to compare generated answers with a ground truth.

RAG capabilities can have a greater impact on overall accuracy for some use cases than other capabilities like model reasoning or mathematical logic. Think of an HR model answering the question, “how many employees are taking time off between now and the end of the month?” Producing the right answer might require navigating multiple systems that change in real time, then performing basic mathematical operations on those figures. In the case of the “time off” question, it might mean navigating one system for hourly employees (which tracks time in hours worked) and a separate system for salaried management, without double-counting any employees or confusing hours with days. A model with well-trained RAG skills will be able to navigate that kind of complexity, which makes it an essential skill for many deployments.

Tooling or function calling

Tooling refers to a model’s ability to interact with other online systems, whether it’s a public API or a back-end system. Typically, it involves teaching the model to access and use an existing application, like an inventory system or HR database.

One of the most popular tooling benchmarks is TAU-bench, which focuses on tool-agent-user interactions. In these instances, the AI model is serving as an intermediary between an existing tool (like an airline ticketing system) and a user with a specific request. TauBench’s questions deal with two different sets of customer service interactions — one for airline tickets and another for retail returns. In each case, the benchmark is testing whether the model can follow a basic script to gather information and enforce policies.

Another major tooling benchmark is the Berkeley Function Calling Leaderboard, also known as BFCL. This benchmark tests a broader set of potential tools, including examples involving SQL, JSON queries, and API calls. It also includes subset tests for parallel calls and sequential calls, allowing developers to optimize models to the specific structure of their planned deployment.

Language understanding

Strong language understanding is a critical aspect of most deployments. Human language is extremely complex, so the associated benchmarks are some of the largest and most sophisticated in the field. The current state of the art is the Massive Multitask Language Understanding benchmark (MMLU), which contains more than 15,000 multiple-choice questions across 57 fields like abstract algebra, macroeconomics, and philosophy. The benchmark was first released in 2021, and several variants like MMLU-Pro have been released in the years since. Still, the core test remains the gold standard for general language capabilities.

Most foundation models were trained in English, and most benchmarks (including MMLU) test skills through English-language interactions. For a deployment involving other languages, companies will either train a model’s multilingual capacity or use a benchmark specific to the language required. Another valuable resource is Cohere’s Aya project, which includes an open-access dataset of 513 million instruction queries across 119 different languages, as well as a custom evaluation suite for training multi-language models.

Safety

What a model won’t say can sometimes be more important than what it will. Toxic or biased responses are a huge concern in many deployments, and there’s an entire set of benchmarks that focus on how well models can avoid those pitfalls.

The ALERT benchmark looks at six categories in particular, all of which would be unwelcome in most enterprise deployments: hate speech, self harm, criminal planning, sexual content, illegal weapons, and illegal drugs. Others like TruthfulQA or DecodingTrust look specifically at a model’s propensity for deception or misalignment. Still, each deployment will come with its own sensitivities, making it vital to use a benchmark that addresses the particular safety risks you’re concerned about.

How to build a real-world evaluation suite

In most enterprise deployments, benchmarks are used as part of a broader “evaluation suite” customized to the needs of your particular project. Some of that evaluation suite will include specific examples from your use case. A customer service deployment, for instance, might evaluate against questions posed by real customers, optimizing for the scenario in which it will be put to work. But you also want to train against general benchmarks to build up broader capabilities, making your model more robust.

Broadly, building an evaluation suite falls into five steps:

1. Choosing a foundation model

Given the current pace of progress, you’ll want to get your foundation model from an outside provider rather than trying to develop it yourself. That provider will be your most important partner in the deployment — not just because of their underlying model, but because of the broader development support that comes with it. In particular, you can expect any serious foundation model provider to have a large bank of in-house enterprise evaluations. Those evaluations will serve as the basis for your custom evaluation suite, giving your model a strong base from which to grow.

2. Decide on key capabilities, then optimize for them

Not every model needs every capability. If your system is for employee use only, for instance, you might be less sensitive to surface-level errors or inappropriate content. If your model is primarily working with customer data, you may want to focus on RAG capabilities above all else. So you’ll want to decide up front which capabilities are most important to you. Then, you’ll want to optimize for those capabilities, using the benchmarks laid out above.

This phase of training is particularly important for building a robust customized model. If you train entirely on data that’s specific to your deployment, the model may struggle when it encounters a situation it hasn’t seen before, causing it to fail in unexpected ways. Training against benchmarks lets you build up broader capabilities in the specific skills your deployment needs.

3. Build a custom test set

Remember, general-purpose benchmarks give only a high-level view, and aren’t a good way to assess how well your model will perform on your specific task. To get that information, you’ll want to create a targeted test set — usually 100 to 200 samples that map as closely as possible to how your model will actually be used. Even a small set of test cases, grounded in your domain, can reveal more than a leaderboard ever will. For privacy reasons, companies may use modified versions of real interactions or concoct new tasks from scratch, depending on the privacy and security issues in play.

This part of the suite will let you optimize for your specific task, but it will also give you crucial information about the model’s strengths and weaknesses — and most critically, whether it’s ready to be deployed. In particular, having your own test set will let you look at the full range of outcomes that single-number benchmark scores often avoid.

4. Consider human evaluators

Human evaluation can be a valuable element of your evaluation suite. Human testers are typically used to A/B test outputs, ranking which response is preferable, and they often uncover issues that fully automated benchmarks miss. But while the contribution can be valuable, human testing can add significant time and expense to the process, which may limit the scope of testing. If the logistical burden of human evaluation prevents you from testing the full range of possible models and prompts, it’s probably doing more harm than good.

5. Iterate

With these individual tests in place, you’ll have everything you need for a robust testing framework — often called a “harness.” Designed properly, the framework will let you easily test new models or prompts to see how they affect outcomes. You can also evaluate each version of your model for consistency, worst-case performance, and output quality — not just the average score or the best.

It’s important to understand this framework as a living part of your process, not a static asset. You’ll want to refresh the suite regularly, adding new test sets to replace sets that are no longer serving their purpose. Foundation models are improving rapidly, and you want to make sure your evaluations reflect those new capabilities, and the risks that come with them.

How agentic AI changes the picture

The rise of agentic AI presents complex new challenges around benchmarking and evaluation that the industry is just beginning to address. Agentic workflows can plan and undertake a multi-step process, using tools, and adapting as it goes, which make them significantly more difficult to evaluate. Effectively evaluating agentic AI systems therefore demands not just new benchmarks, but new thinking.

Cascading errors are a particular concern, with an apparently minor error at an early stage potentially compounding into a completely inaccurate result later in the process. The need for agentic AI to call up the right tools at the right time raises another critical new area for testing and evaluation.

A multi-layered testing strategy is needed to evaluate both how agents perform on narrow tasks and how well they integrate and meet expectations across a complete workflow.

For example,

Unit tests measure specific tasks, such as whether an agent can retrieve a certain file type.
Scenario-based evaluations can measure how well it performs a real-world task, such as searching internal knowledge basis to answer certain questions.
End-to-end integration testing can then measure its overall performance.

While these evaluations can be supported by benchmarks like TAU-bench and Berkeley, it’s particularly crucial to base them on real-world scenarios to understand how an agentic system will really act, adapt, and recover across complex tasks.

RAG and tool-calling benchmarks are particularly relevant for agentic evaluation, as these skills often provide the basis for how agents interact with each other. Multi-turn conversation benchmarks like MultiChallenge can be helpful for tracking overall coherence. Reasoning benchmarks have also been found to predict agentic success.

Moving beyond benchmarks

Ultimately, the right AI model for any organization isn’t going to be the one with the most impressive benchmark scores — it will be the one that performs reliably and accurately in their secure and unique environment.

Think of a successful evaluation as having three levels. At the bottom, there’s the foundation model with core training in relevant skills. Above that comes an assessment based on broad public benchmarks to sharpen those skills. The final layer is a private, deployment-specific evaluation that gives a full picture of a model’s capabilities and enables you to tune it to the business world in which it needs to operate.

Enterprises need to treat these evaluation stages as an integral part of their AI deployment, rather than as a one-off exam based on benchmark scores. That’s particularly important in light of the growing importance of agentic AI, for which reliance only on academic benchmarks risks missing the mark entirely.

Building enterprise AI agents

A comprehensive blueprint for developing scalable AI agents in regulated industries.

Download the eBook

Alex Wang

Manager of Technical Staff in Data & Eval