Overview

Tool use in LLMs involves deciding when and how to call external functions to retrieve information or perform actions. It can be applied across a few interaction patterns and conversation contexts.

This chapter demonstrates various patterns for implementing tool usage with Cohere's chat API.

By the end, you will have working examples of parallel tool calls, direct responses, forced tool usage, and multi-turn conversations.

What we'll cover

Making parallel tool calls
Responding directly without tools
Forcing tool usage patterns
Building multi-turn chatbots

import cohere
import os
co = cohere.ClientV2(os.getenv("COHERE_API_KEY"))

We’ll use the same get_weather tool as in the previous chapter.

def get_weather(location):
    # Implement any logic here
    return [{"temperature": "20C"}]
    # Return a list of objects e.g. [{"url": "abc.com", "text": "..."}, {"url": "xyz.com", "text": "..."}]

functions_map = {"get_weather": get_weather}

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "gets the weather of a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "the location to get the weather, example: San Francisco.",
                    }
                },
                "required": ["location"],
            },
        },
    },
]

Parallel tool calls

The model can determine that more than one tool call is required, where it will call multiple tools in parallel. This can be calling the same tool multiple times or different tools for any number of calls.

In the example below, the user asks for the weather in Toronto and New York. This requires calling the get_weather function twice, one for each location. This is reflected in the model’s response, where two parallel tool calls are generated.

messages = [
    {
        "role": "user",
        "content": "What's the weather in Toronto and New York?",
    }
]

response = co.chat(
    model="command-a-03-2025", messages=messages, tools=tools
)

if response.message.tool_calls:
    messages.append(
        {
            "role": "assistant",
            "tool_plan": response.message.tool_plan,
            "tool_calls": response.message.tool_calls,
        }
    )
    print(response.message.tool_plan, "\n")
    for tool_call in response.message.tool_calls:
        print(tool_call, "\n")

I will use the get_weather tool to find the weather in Toronto and New York. 

id='get_weather_8j2f1mr51z5d' type='function' function=ToolCallV2Function(name='get_weather', arguments='{"location":"Toronto"}') 

id='get_weather_eytxekgxeh9p' type='function' function=ToolCallV2Function(name='get_weather', arguments='{"location":"New York"}')

When tools are called in parallel, we append the messages list with one single assistant message containing all the tool calls and one tool message for each tool call.

import json

if response.message.tool_calls:
    for tc in response.message.tool_calls:
        tool_result = functions_map[tc.function.name](
            **json.loads(tc.function.arguments)
        )
        tool_content = []
        for data in tool_result:
            # Optional: the "document" object can take an "id" field for use in citations, otherwise auto-generated
            tool_content.append(
                {
                    "type": "document",
                    "document": {"data": json.dumps(data)},
                }
            )
        messages.append(
            {
                "role": "tool",
                "tool_call_id": tc.id,
                "content": tool_content,
            }
        )

for message in messages:
    print(message)
    print("-"*50)

{'role': 'user', 'content': "What's the weather in Toronto and New York?"}
--------------------------------------------------
{'role': 'assistant', 'tool_plan': 'I will use the get_weather tool to find the weather in Toronto and New York.', 'tool_calls': [ToolCallV2(id='get_weather_8j2f1mr51z5d', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"Toronto"}')), ToolCallV2(id='get_weather_eytxekgxeh9p', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"New York"}'))]}
--------------------------------------------------
{'role': 'tool', 'tool_call_id': 'get_weather_8j2f1mr51z5d', 'content': [{'type': 'document', 'document': {'data': '{"temperature": "20C"}'}}]}
--------------------------------------------------
{'role': 'tool', 'tool_call_id': 'get_weather_eytxekgxeh9p', 'content': [{'type': 'document', 'document': {'data': '{"temperature": "20C"}'}}]}
--------------------------------------------------

Directly answering

A key attribute of tool use systems is the model’s ability to choose the right tools for a task. This includes the model’s ability to decide to not use any tool, and instead, respond to a user message directly.

In the example below, the user asks for a simple arithmetic question. The model determines that it does not need to use any of the available tools (only one, get_weather, in this case), and instead, directly answers the user.

messages = [{"role": "user", "content": "What's 2+2?"}]

response = co.chat(
    model="command-a-03-2025", messages=messages, tools=tools
)

if response.message.tool_calls:
    print(response.message.tool_plan, "\n")
    print(response.message.tool_calls)

else:
    print(response.message.content[0].text)

2+2 is 4.

print(response)

id='e0c59423-7293-496c-bcc2-6439c8216a9c' finish_reason='COMPLETE' message=AssistantMessageResponse(role='assistant', tool_calls=None, tool_plan=None, content=[TextAssistantMessageResponseContentItem(type='text', text='2+2 is 4.')], citations=None) usage=Usage(billed_units=UsageBilledUnits(input_tokens=31.0, output_tokens=7.0, search_units=None, classifications=None), tokens=UsageTokens(input_tokens=1453.0, output_tokens=9.0)) logprobs=None

Forcing tool usage

As shown in the previous examples, during the tool calling step, the model may decide to either:

make tool call(s)
or, respond to a user message directly.

You can, however, force the model to choose one of these options. This is done via the tool_choice parameter.

response = co.chat(
    model="command-a-03-2025",
    messages=messages,
    tools=tools,
    tool_choice="REQUIRED" # optional, to force tool calls
    # tool_choice="NONE" # optional, to force a direct response
)

Chatbots (multi-turn)

Building chatbots requires maintaining the memory or state of a conversation over multiple turns. To do this, we can keep appending each turn of a conversation to the messages list.

As an example, here’s the messages list from the first turn of a conversation.

messages = [
    {"role": "user", "content": "What's the weather in Toronto?"},
    {
        "role": "assistant",
        "tool_plan": "I will search for the weather in Toronto.",
        "tool_calls": [
            {
                "id": "get_weather_1byjy32y4hvq",
                "type": "function",
                "function": {
                    "name": "get_weather",
                    "arguments": '{"location":"Toronto"}',
                },
            }
        ],
    },
    {
        "role": "tool",
        "tool_call_id": "get_weather_1byjy32y4hvq",
        "content": [
            {
                "type": "document",
                "document": {"data": '{"temperature": "20C"}'},
            }
        ],
    },
    {"role": "assistant", "content": "It's 20°C in Toronto."},
]

messages.append({"role": "user", "content": "What about London?"})

response = co.chat(
    model="command-a-03-2025", messages=messages, tools=tools
)

if response.message.tool_calls:
    messages.append(
        {
            "role": "assistant",
            "tool_plan": response.message.tool_plan,
            "tool_calls": response.message.tool_calls,
        }
    )
    print(response.message.tool_plan, "\n")
    print(response.message.tool_calls)

I will search for the weather in London. 

[ToolCallV2(id='get_weather_hs2at9cqe6te', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"London"}'))]

for message in messages:
    print(message)
    print("-" * 50)

{'role': 'user', 'content': "What's the weather in Toronto?"}
--------------------------------------------------
{'role': 'assistant', 'tool_plan': 'I will search for the weather in Toronto.', 'tool_calls': [{'id': 'get_weather_1byjy32y4hvq', 'type': 'function', 'function': {'name': 'get_weather', 'arguments': '{"location":"Toronto"}'}}]}
--------------------------------------------------
{'role': 'tool', 'tool_call_id': 'get_weather_1byjy32y4hvq', 'content': [{'type': 'document', 'document': {'data': '{"temperature": "20C"}'}}]}
--------------------------------------------------
{'role': 'assistant', 'content': "It's 20°C in Toronto."}
--------------------------------------------------
{'role': 'user', 'content': 'What about London?'}
--------------------------------------------------
{'role': 'assistant', 'tool_plan': 'I will search for the weather in London.', 'tool_calls': [ToolCallV2(id='get_weather_hs2at9cqe6te', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"London"}'))]}
--------------------------------------------------

Conclusion

This chapter covered the main patterns for implementing tool usage in conversational applications, including parallel execution, response control, and maintaining conversation state across multiple turns.

Tool Use Usage Patterns