Today we’re releasing updated versions of our Chat, Embed, Rerank, and Classify APIs. We aim to improve the developer experience with these new versions, making it easier and faster for developers to build with Cohere.

We’re streamlining our APIs to adhere more closely to industry standards. This will enable developers switching from other model providers to develop apps faster with Cohere. The new API version will also match features offered by Cohere’s models hosted on private cloud providers, making it easier for developers to upgrade from using Cohere’s endpoints hosted on Cohere to private cloud.

V2 endpoints are now available on the Cohere platform, accompanied by a comprehensive set of SDKs. Key updates to the APIs include:

Required model version: model as a required field for Embed, Rerank, Classify, and Chat APIs
Usability improvements to the Chat API including:
- Using a single array for messages instead of separate message, chat_history, and preamble parameters
- Ability to define tools in JSON schema
- Introducing tool call IDs to match tool results with tool calls
- Using server sent events (SSE) for streaming

Explore the details below to see what we’ve implemented.

Get started with V2

API V2 supports all existing Cohere models. We’re continuing to improve V2 and if you have any feedback, please share it with us on Discord.

We are releasing a V2 SDK alongside the major endpoint version update. With the new SDKs we are publishing an OpenAPI spec for our V2 endpoint. To get started using our APIs, visit our API Reference. See our V2 Chat Migration Guide for instructions to migrate your Chat API implementation.

Update model versions with certainty

`model` is now a required field across our Embed, Rerank, Classify, and Chat APIs. In the previous V1 APIs, developers can send requests without specifying which model they want to use. This can lead to unwanted behavior changes for developers when Cohere releases new models and updates the underlying default model for our endpoints. This is especially relevant for our Embed models, where using a different embed model across queries will result in performance degradation.

Easily switch to the Cohere API

The Chat V2 API will receive a number of new updates to improve usability and model performance for users. See below for an explanation of the changes.

Send all messages in a single array

Cohere’s Chat V2 API now combines all chat message inputs into a `messages` array. By contrast, the Chat V1 API accepts the latest user message in the `message` parameter, a user instruction in the `preamble` parameter, and all previous chat messages in a `chat_history` parameter.

Shown in the example below, with Chat V1, the `message` parameter has no assignable role and is limited to only the `user` role. With Chat V2, we are enabling use cases that require the `system` or `assistant` role to be the latest chat turn. Additionally, we are replacing `chatbot` with `assistant`.

# Chat V1: Prompt using Message + Chat History
chat_history = [
{"message": "hi", "role": "user"},
{"message": "hello, how can i help you", "role": "chatbot"},
]
message = "count to three",
preamble = "You are a helpful assistant"

co.chat(model="command-r", message=message, chat_history=chat_history, preamble=preamble)

# Chat V2: Messages
messages = [
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": "hi"},
{"role": "assistant", "content": "hello, how can i help you"},
{"role": "user", "content": "count to three"},
]
co.chat(model="command-r", messages=messages)

Define tools in JSON schema

Chat V1 uses Python types to define tools. Chat V2 allows users to define tools in JSON schema, instead of Python types.

# Chat V1: Tools are defined in Python types

"tools": [{
  "name": "get_weather",
  "description": "Gets the weather for a given location",
  "parameter_definitions": {
    "location": {
      "description": "The city and state, e.g. San Francisco, CA",
      "type": "str",
      "required": true
    }
  }
}]

# Chat V2: Tools are defined in JSON schema 

"tools": [{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description" : "gets the weather of a given location",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {
          "type" : "string",
          "description": "the location to get weather, example: San Fransisco, CA"
        }
      },
      "required": ["location"]
    }
  }
}]

Tool calls emit IDs that must be returned by tool results

In a scenario where a user is returning multiple tool results in follow-on messages, the model does not have a way to identify the correctly associated tool call. Tool calls in Chat V1 did not emit tool IDs, but now in Chat V2 there are tool call IDs. This will help the model match tool results to the right tool call.

# Chat V1: Tool results (No tool call IDs) 
"tool_results" = [
  {
    "call": {
      "name": <tool name>,
      "parameters": {
        <param name>: <param value>
      }
    },
    "outputs": [{
      <key>: <value>
    }]
  },
]


# Chat V2: Tool results (With tool call IDs) 
"messages": [
    {
        "role": "tool",
        "tool_call_id": <tool call id>,
        "content": [
            {
                "type": "document",
                "document": {
                    "id": "123",
                    "data": {"<key>": "<value>"},
                },
            }
        ],
    }
]

Server sent stream events

Chat V2 will use server sent events (SSE) for streaming, whereas Chat V1 uses JSON-stream events. See our migration guide for more details on the new stream events.

Support for V1 APIs and SDKs

The V1 suite of APIs will continue to be supported by Cohere and cloud providers with the currently available models. There will be no breaking changes to the V1 API or SDK. We recommend updating your implementation to use the V2 API for enforced model version stability and more common Chat API features such as server sent events for streaming.

Helpful information

API V2 supports all existing Cohere models. We are releasing a V2 SDK alongside the major endpoint version update. With the new SDKs we are publishing an OpenAPI spec for our V2 endpoint. To get started using our APIs, visit our API Reference. See our V2 Chat Migration Guide for instructions to migrate your Chat API implementation.

We’re continuing to improve V2 and if you have any feedback, please share it with us on Discord.

Billy Trend