The practical benefits of generative AI for businesses are piling up, from getting better, faster insights out of their data, to enhanced decision-making, improved customer experience and reduced costs. But all of these gains risk being undercut if companies don’t focus on making trust and worker acceptance central to their rollout of the technology.

This is where citations have an increasingly vital role to play.

Citations provide users with a bridge between the AI-generated responses to their queries and the underlying data that the model is drawing from. This adds a crucial layer of trust to working with GenAI systems, which while powerful and accurate are still not perfect. Not every AI model comes with citations, nor are they all optimized to run them.

Example of citations for customer support solution.

In this article, we’ll discuss why citations are important, some of the challenges and questions that come up around using them, and how organizations can get the most out of them. Let’s jump in.

Why citations are important

Nearly two years since GenAI arrived on the scene, the technology’s tendency to occasionally veer away from the facts and make things up remains a concern. In the corporate setting where accuracy and trustworthiness are paramount, this AI “hallucination” problem risks undermining its whole value proposition.

The arrival of retrieval-augmented generation (RAG) systems has helped reduce hallucination risks because their responses are grounded in relevant information in knowledge bases, leaving less room for the LLM to simply make up the answers.

Still, the top 25 most used large-language models (LLMs) “hallucinate” at rates of between 1.3% to 4.2%, according to a hallucination leaderboard maintained on Github.

Given this persistent issue, citations provide a valuable additional layer of trust and utility, enabling users to see exactly which source a given part of a response is based on and to explore and verify further with their own research into source documents. While citations don’t guarantee that an AI model is always correct or truthful, they do empower users to quickly check the accuracy of a response.

This is especially important in fields like healthcare, manufacturing, and finance, where accuracy is crucial for decision-making and compliance. In these industries, citations aren’t just an added bonus; they’re essential.

Citations can also be a valuable element of customer-facing AI applications, helping companies to build trust by providing links to factual sources. Take the example of Naq Cyber, an automated compliance startup. It uses Cohere models to automatically generate answers to users’ questions about compliance standards such as GDPR, using citations to link back to trustworthy sources from their own databases. This ability to audit LLM responses opens the door to new and more varied industry use cases.

Making citations work for users

To perform well on citations, LLMs need to be specifically trained to generate them, which isn’t the case out of the box with every model.

The way they are designed to enhance rather than hinder the user experience is also vital. Citations need to support productivity while avoiding swamping users with unnecessary detail or so many sources that it slows workflows and makes it difficult to know which ones are important.

Users are demanding control over the type and quantity of citations they’re presented with, depending on their industry, job role, and workflow. Companies in industries such as legal compliance or financial analysis, for example, often need more comprehensive citations to comply with industry standards and regulations.

The two key variables for citations are 1. Granularity: The quantity of citations for a given length of text, and, 2. Volume: The number of sources included in each citation – one, two, several, or as many as are relevant?

It’s important to listen to user feedback on what type of citations work best for them and adjust accordingly.

At Cohere, for example, we’ve introduced the following two citation modes based on what we’ve heard customers want.

Post-response citations: This mode maximizes citation accuracy but only generates citations at the very end of the model response. It's suitable for use cases when a slight delay in receiving citations is acceptable.
Inline citations: This mode generates citations during the model response, reducing latency while maintaining a similar citation quality. It’s a desirable option for organizations that need real-time citations.

For organizations in certain industries, such as financial markets, it will be essential to have real-time citations that pull from the most up-to-date information. Cohere models optimized for RAG solutions, for example, allow users to link to real-time data through APIs, whether that’s to the latest stock market levels or today’s weather. Companies in other industries may be able to rely solely on pre-curated knowledge bases, though these still need to be kept “fresh” with regular updates.

In agentic solutions, citations allow users to understand where the AI response is coming from. They clarify which tools the AI relies on and which results it references to form its answers.

Another vital part of creating a good user experience lies in how citations are formatted. Hyperlinks, footnotes, and more traditional in-text references are three of the main options. Again, the right choice will largely depend on the user base and the industry an organization is in. Consumer-facing companies may opt for less intrusive hyperlink citations while legal firms may prefer in-text citations in line with industry standards, for example.

Ensuring data quality

GenAI systems are only as good as the data they access, and the same applies to citations. If the model is referencing outdated or low-quality sources, the citations will reflect that. This makes it crucial for organizations to maintain and regularly update their knowledge bases to ensure that citations remain relevant and useful.

There’s an ethical aspect to data and citations too. Businesses need to make sure they’re referencing a diverse range of sources to provide balanced and accurate information. Otherwise, citations could reflect and reinforce any underlying biases.

User trust can also be built up through transparency by allowing users to explore the data sources via citations and building confidence in the reliability and accuracy of the answers.

Ensuring citation success

Ensuring that citations are meeting accuracy and relevance standards and raising trust in AI tools is an ongoing, interactive process.

At Cohere, for example, our global team of annotators is constantly fine-tuning our Command model to ensure the data they retrieve and the passages being cited are as relevant as possible to users’ queries. This enables users to then dive deeper into the source documents for further checking and research, confident that they are relevant to their initial query.

The expert annotators teach the model by rating each snippet it retrieves from source databases as “highly relevant,” “relevant,” or “not relevant.” Those ratings are then peer reviewed to ensure consistency and accuracy. Once the model has been trained in this way by annotators, Cohere engineers A/B test it against other versions to assess how well its citations perform.

The relevance and accuracy of citations can be measured through two key metrics: citation recall and citation precision.

Citation recall measures the percentage of key factual statements made by the model that have proper citations, ensuring users are being presented with sources they might want to verify. For each key statement, human reviewers check if the model provided a citation. The Citation Recall score is then calculated by dividing the number of statements that have citations by the number of verification-worthy statements.

The citation precision score assesses the accuracy of citations by measuring the proportion that correctly supports their associated statements.

Delivering trustworthy AI

With AI’s hallucination problem not going away any time soon and compliance requirements rising across industries, citations will continue to play a major role in unlocking the technology’s true potential. Currently still in their early days, they are likely to develop in ways that build users’ trust and ease of use, while adapting according to the needs of different audiences and industries.

Building enterprise AI agents

A comprehensive blueprint for developing scalable AI agents in regulated industries.

Download the eBook

Payal Singh