We’ll use Cohere’s Python SDK for the code examples. Follow along in this notebook.

Understanding text generation is the first step to creating your own chatbot, but the real learning takes place when you start building one from scratch. In this chapter, you’ll learn how to use Cohere’s Chat endpoint to build a simple chatbot that can respond to user messages and maintain the context of the conversation.

Additionally, the API reference page contains a detailed description of the Chat endpoint’s input parameters and response objects.

Step-by-Step Guide

Step 1: Quickstart

To set up, we first import the Cohere module and create a client.

import cohere
co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key: https://dashboard.cohere.com/api-keys

At its most basic, we only need to pass two parameters to the Chat endpoint: the model type and the user message using the messages parameter.

At its most basic, the endpoint takes a user message and generates a response

Here’s an example. We use command-r-plus as the model and pass the user message as the user role in the messages array.

Right now, we’re interested in the main content of the response, which is stored in the response.message.content[0]['text'] value of the response.

# Add the user message
message = "I'm joining a new startup called Co1t today. Could you help me write an introduction message for my teammates."

# Generate the response
response = co.chat(model="command-r-plus-08-2024",
                   messages=[{'role':'user', 'content': message}])

print(response.message.content[0].text)

# RESPONSE

Sure! Here is a short introduction message:

"Hi everyone! My name is [Your Name] and I am thrilled to be joining the Co1t team today. I am excited to contribute to the company's mission and to collaborate with all of you. I look forward to getting to know you and working together to achieve success. Let's make Co1t the best it can be!"

Step 2: Defining a Custom System Message

A conversation starts with a system message, or a preamble, to help steer a chatbot’s response toward certain characteristics. For example, if we want the chatbot to adopt a formal style, the preamble can be used to encourage the generation of more business-like and professional responses.

The custom system message and the current user message form the prompt to the chatbot

In the earlier example, we didn’t have to define a custom system message because a default one was used. We can, however, define our own by adding a message with the system role in the messages array.

Here’s an example. Here, we are telling the chatbot to generate the text in rhymes as much as possible. As a result, we get a response that adopts that persona.

# Add the user message
message = "I'm joining a new startup called Co1t today. Could you help me write an introduction message for my teammates."

# Create a custom system message
system_message="""## Task and Context
You are an assistant who assist new employees of Co1t with their first week.

## Style Guide
Try to speak in rhymes as much as possible. Be professional."""

# Add the messages
messages = [{'role': 'system', 'content': system_message},
            {'role': 'user', 'content': message}]

# Generate the response
response = co.chat(model="command-r-plus-08-2024",
                   messages=messages)

print(response.message.content[0].text)

# RESPONSE

Sure, let's get you started,
And to your teammates, be warmly welcomed! 

Hello, my name is [Your Name],
And I'm thrilled to join the crew,
At Co1t, I'm new,
But my enthusiasm is true.

I look forward to working with you all,
Learning and growing, taking the ball,
Together, we'll innovate and create,
A successful path, we'll navigate.

My skills and experience, I bring to the table,
Ready to take on any challenge, able and stable,
So let's collaborate and synergize,
And watch our startup soar to new heights, surprise by surprise. 

Cheers to a great journey ahead,
With Co1t, we'll leave a mark, bold and red! 

Your enthusiastic colleague, 
[Your Name]

Step 3: Streaming the Chatbot Response

Our examples so far generate responses in a non-streamed manner. This means that the endpoint would return a response object only after the model has generated the text in full. The longer the text is, the longer it takes to get back the response. If you are building an application, this directly impacts the user’s perception of the application’s latency.

The Chat endpoint solves this problem by supporting streamed responses. In a streamed response, the endpoint would return a response object for each token as it is being generated. This means you can display the text incrementally without having to wait for the full completion.

To activate it, use co.chat_stream() instead of co.chat().

In streaming mode, the endpoint will generate a series of objects. To get the actual text contents, we take stream events whose type is content-delta.

# Add the user message
message = "I'm joining a new startup called Co1t today. Could you help me write an introduction message for my teammates."

# Generate the response by streaming it
response = co.chat_stream(model="command-r-plus-08-2024",
                          messages=[{'role':'user', 'content': message}])

for event in response:
    if event.type == "content-delta":
        print(event.delta.message.content.text, end="")

# RESPONSE (Streamed)

Hello, my name is [Your Name] and I'm thrilled to join the Co1t team today as the newest member!

Do you want help drafting a more detailed introduction or any other messages you might need for your first day at work?

Step 4: Building the Chat History

At the core of a conversation is a multi-turn dialog between the user and the chatbot. This requires the chatbot to have a “memory” of all the previous turns to maintain the state of the conversation.

The messages list is used to store the conversation history, including both user messages and chatbot responses. It is structured as an array of objects, with each object representing a message in the conversation.

By appending the previous response and new user message to messages, the model can access the chat history and generate contextually-aware responses. This allows the chatbot to refer to previous messages and maintain a consistent conversation flow.

To illustrate this, let's look at an example. We start with the first conversation turn.

Let's also add a custom system message for generating concise response, just to keep the outputs brief for this tutorial.

# Add the user message
message = "I'm joining a new startup called Co1t today. Could you help me write an introduction message for my teammates."

# Create a custom system message
system_message="""## Task and Context
Generate concise responses, with maximum one-sentence."""

# Add the messages
messages = [{'role': 'system', 'content': system_message},
            {'role': 'user', 'content': message}]

# Generate the response
response = co.chat(model="command-r-plus-08-2024",
                   messages=messages)

print(response.message.content[0].text)

"Hi everyone, I'm thrilled to join Co1t as a new team member, and I look forward to collaborating and contributing to our shared success!"

Now, we want the model to refine the earlier response. This requires the next generation to have access to the state, or memory, of the conversation.

To do this, we append the message object from the previous response to the messages list.

Then we append the next user message to the messages list.

Looking at the response, we see that the model is able to get the context from the chat history. The model is able to capture that "it" in the user message refers to the introduction message it had generated earlier.

# Append the previous response
messages.append({'role': 'assistant', 'content': response.message.content})

# Add the user message
message = "Make it more upbeat and conversational."

# Append the user message
messages.append({'role': 'user', 'content': message})

# Generate the response with the current chat history as the context
response = co.chat(model="command-r-plus-08-2024",
                   messages=messages)

print(response.message.content[0].text)

"Hey there, super excited to be a part of the Co1t family! Can't wait to meet you all and dive into some awesome projects together!"

You can continue doing this for any number of turns by repeating the same steps of appending the chatbot and user messages.

# Append the previous response
messages.append({'role': 'assistant', 'content': response.message.content})

# Add the user message
message = "Thanks. Could you create another one for my DM to my manager."

# Append the user message
messages.append({'role': 'user', 'content': message})

# Generate the response with the current chat history as the context
response = co.chat(model="command-r-plus-08-2024",
                   messages=messages)

print(response.message.content[0].text)

"Hey [Manager's Name], just wanted to express my excitement about starting at Co1t today, and I'm grateful for the opportunity to work with you and the team!"

To look at the current chat history, you can print the messages list.

# Append the previous response
messages.append({'role': 'assistant', 'content': response.message.content})
# View the chat history
for message in messages:
    print(message,"\n")

{'role': 'system', 'content': '## Task and Context\nGenerate concise responses, with maximum one-sentence.'} 

{'role': 'user', 'content': "I'm joining a new startup called Co1t today. Could you help me write an introduction message for my teammates."} 

{'role': 'assistant', 'content': [TextAssistantMessageResponseContentItem(type='text', text='"Hi everyone, I\'m thrilled to join Co1t as a new team member, and I look forward to collaborating and contributing to our shared success!"')]} 

{'role': 'user', 'content': 'Make it more upbeat and conversational.'} 

{'role': 'assistant', 'content': [TextAssistantMessageResponseContentItem(type='text', text='"Hey there, super excited to be a part of the Co1t family! Can\'t wait to meet you all and dive into some awesome projects together!"')]} 

{'role': 'user', 'content': 'Thanks. Could you create another one for my DM to my manager.'} 

{'role': 'assistant', 'content': [TextAssistantMessageResponseContentItem(type='text', text='"Hey [Manager\'s Name], just wanted to express my excitement about starting at Co1t today, and I\'m grateful for the opportunity to work with you and the team!"')]}

And with that, we have built a simple chatbot that can respond to user messages and maintain the context of the conversation.

Conclusion

This chapter showed how to build a simple chatbot using the Chat endpoint and how to configure the chatbot, such as overriding the preamble, building the chat history, streaming the response, and modifying the parameters.

Building a Chatbot