Overview

When using tools in conversational AI, it's important to trace where information comes from in the generated responses. Citations provide transparency and accountability by linking specific parts of the model's output back to the exact tool results that informed them.

This chapter demonstrates how to implement citations when using tools with Cohere's API, covering both streaming and non-streaming approaches.

By the end, you will be able to retrieve and display citations that connect model responses to their underlying tool outputs.

What we'll cover

Implementing non-streaming tool citations
Implementing streaming tool citations
Using document IDs
Enabling fast citation mode

import cohere
import os
co = cohere.ClientV2(os.getenv("COHERE_API_KEY"))

import json

Non-streaming

We'll look at examples of using citations in both streaming and non-streaming mode.

Let's start with non-streaming mode. Here, the citations are provided in the message.citations field of the response object.

Each citation object contains:

start and end: the start and end indices of the text that cites a source(s)
text: its corresponding span of text
sources: the source(s) that it references

def get_weather(location):
    temperature = {
        "bern": "22°C",
        "madrid": "24°C",
        "brasilia": "28°C",
    }
    loc = location.lower()
    if loc in temperature:
        return [{"temperature": {loc: temperature[loc]}}]
    return [{"temperature": {loc: "Unknown"}}]


functions_map = {"get_weather": get_weather}

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "gets the weather of a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "the location to get the weather, example: San Francisco.",
                    }
                },
                "required": ["location"],
            },
        },
    }
]

messages = [
    {
        "role": "user",
        "content": "What's the weather in Madrid and Brasilia?",
    }
]

response = co.chat(
    model="command-a-03-2025", messages=messages, tools=tools
)

if response.message.tool_calls:
    messages.append(
        {
            "role": "assistant",
            "tool_plan": response.message.tool_plan,
            "tool_calls": response.message.tool_calls,
        }
    )

    for tc in response.message.tool_calls:
        tool_result = functions_map[tc.function.name](
            **json.loads(tc.function.arguments)
        )
        tool_content = []
        for data in tool_result:
            tool_content.append(
                {
                    "type": "document",
                    "document": {"data": json.dumps(data)},
                }
            )
        messages.append(
            {
                "role": "tool",
                "tool_call_id": tc.id,
                "content": tool_content,
            }
        )

response = co.chat(
    model="command-a-03-2025", messages=messages, tools=tools
)

messages.append(
    {"role": "assistant", "content": response.message.content[0].text}
)

print(response.message.content[0].text)

for citation in response.message.citations:
    print(citation, "\n")

The temperature in Madrid is 24°C and the temperature in Brasilia is 28°C.
start=29 end=33 text='24°C' sources=[ToolSource(type='tool', id='get_weather_vfcg8068s4tt:0', tool_output={'temperature': '{"madrid":"24°C"}'})] type='TEXT_CONTENT' 

start=69 end=74 text='28°C.' sources=[ToolSource(type='tool', id='get_weather_7yerfzmfy9t8:0', tool_output={'temperature': '{"brasilia":"28°C"}'})] type='TEXT_CONTENT'

Streaming

You can stream Chat API responses in both the tool calling and the response generation steps. This allows your application to receive token streams as the model plans and executes tool calls and finally generates its response.

To enable response streaming in tool use, use the chat_stream endpoint instead of chat.

In a streaming scenario, the citations are provided in the citation-start events.

Each citation object contains the same fields as the non-streaming scenario.

response = co.chat_stream(
    model="command-a-03-2025", messages=messages, tools=tools
)

response_text = ""
citations = []
for chunk in response:
    if chunk:
        if chunk.type == "content-delta":
            response_text += chunk.delta.message.content.text
            print(chunk.delta.message.content.text, end="")
        if chunk.type == "citation-start":
            citations.append(chunk.delta.message.citations)

messages.append({"role": "assistant", "content": response_text})

for citation in citations:
    print(citation, "\n")

What's the weather in Madrid and Brasilia?

Document ID

When passing the tool results from the tool execution step, you can optionally add custom IDs to the id field in the document object. These IDs will be used by the endpoint as the citation reference.

If you don’t provide the id field, the ID will be auto-generated in the the format of <tool_call_id>:<auto_generated_id>. Example: get_weather_1byjy32y4hvq:0.

Here is an example of using custom IDs. To keep it concise, let’s start with a pre-defined list of messages with the user query, tool calling, and tool results are already available.

messages = [
    {
        "role": "user",
        "content": "What's the weather in Madrid and Brasilia?",
    },
    {
        "role": "assistant",
        "tool_plan": "I will search for the weather in Madrid and Brasilia.",
        "tool_calls": [
            {
                "id": "get_weather_dkf0akqdazjb",
                "type": "function",
                "function": {
                    "name": "get_weather",
                    "arguments": '{"location":"Madrid"}',
                },
            },
            {
                "id": "get_weather_gh65bt2tcdy1",
                "type": "function",
                "function": {
                    "name": "get_weather",
                    "arguments": '{"location":"Brasilia"}',
                },
            },
        ],
    },
    {
        "role": "tool",
        "tool_call_id": "get_weather_dkf0akqdazjb",
        "content": [
            {
                "type": "document",
                "document": {
                    "data": '{"temperature": {"madrid": "24°C"}}',
                    "id": "1",
                },
            }
        ],
    },
    {
        "role": "tool",
        "tool_call_id": "get_weather_gh65bt2tcdy1",
        "content": [
            {
                "type": "document",
                "document": {
                    "data": '{"temperature": {"brasilia": "28°C"}}',
                    "id": "2",
                },
            }
        ],
    },
]

When document IDs are provided, the citation will refer to the documents using these IDs.

response = co.chat(
    model="command-a-03-2025", messages=messages, tools=tools
)

print(response.message.content[0].text)

for citation in response.message.citations:
    print(citation, "\n")

The temperature in Madrid is 24°C and in Brasilia it is 28°C.
start=29 end=33 text='24°C' sources=[ToolSource(type='tool', id='1', tool_output={'temperature': '{"madrid":"24°C"}'})] type='TEXT_CONTENT' 

start=56 end=61 text='28°C.' sources=[ToolSource(type='tool', id='2', tool_output={'temperature': '{"brasilia":"28°C"}'})] type='TEXT_CONTENT'

Fast citation mode

The model generates citations inline, as the response is being produced. In streaming mode, you will see citations injected at the exact moment the model uses a particular piece of external context. This approach provides immediate traceability at the expense of slightly less precision in citation relevance.

You can specify it by adding the citation_options={"mode": "fast"} argument in the API call.

With the citation_options mode set to fast, we get the citations inline as the model generates the response.

response = co.chat_stream(
    model="command-a-03-2025",
    messages=messages,
    tools=tools,
    citation_options={"mode": "fast"},
)

response_text = ""
for chunk in response:
    if chunk:
        if chunk.type == "content-delta":
            response_text += chunk.delta.message.content.text
            print(chunk.delta.message.content.text, end="")
        if chunk.type == "citation-start":
            print(
                f" [{chunk.delta.message.citations.sources[0].id}]",
                end="",
            )

The temperature in Madrid is 24°C [1] and the temperature in Brasilia is 28°C. [2]

Conclusion

This chapter covered how to implement citations with tool use in Cohere's API, including both streaming and non-streaming approaches, document ID management, and fast citation mode for real-time applications.

Tool Use Citations