We’ll use Cohere’s Python SDK for the code examples. Follow along in this notebook.

The Chat endpoint is a versatile tool that empowers developers with an extensive array of options and parameters.

As you’ll learn, the Command model has many variations to select from, where each has been carefully crafted to suit different needs. Additionally, you will see how to use parameters to control the creativity of model responses.

Setup

To set up, we first import the Cohere module and create a client.

import cohere
co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key: https://dashboard.cohere.com/api-keys

Model Type

With the Chat endpoint , you can choose from several variations of the Command model . Different models may produce different output profiles, so you may want to experiment with different models to get the best output for your use case.

Use the model parameter to select a variation that suits your requirements. In the code cell, we select command-r-plus-08-2024.

# Add the user message
message = "Hello."

# Generate the response
response = co.chat(model="command-r-plus-08-2024",
                   messages=[{'role':'user', 'content': message}])

print(response.message.content[0].text)

Hello! How can I help you today?

Randomness

Often, you’ll need to control the level of randomness of the model. There are a number of factors to keep in mind when tuning model randomness, including:

Task type: You’ll likely want to decrease the randomness of the model when performing structured tasks that have a correct answer, like for question answering or summarization, or for generating technical text. In these cases, we want the model to yield a safe and predictable response. On the other hand, if you’re generating poetry or brainstorming ideas, you might want to increase the randomness to produce more diverse and creative responses.
Model behavior: We likely need to increase the randomness of the model if it gets stuck in a loop and starts repeating itself, or if it is producing overly generic phrases. Increasing the randomness will expand the set of words that the model can use when generating responses.
Controlling style and tone: If generating text that needs to have a specific tone or style, like for a company blog post or customer support responses, a low level of randomness might be desirable, to keep the model from generating unusual words.

Modifying the temperature parameter changes the extent to which the model considers incorporating unlikely tokens (can be words, parts of words, or punctuation) in its response, which can make the output more random and creative.

To understand this, we’ll look at an example. The model would likely predict that the token cookies has a much higher likelihood than chair for appearing after the phrase I like to bake.

Before these likelihoods can be used to select the next token, they first need to be converted to probabilities. The temperature parameter controls how this conversion is done.

At low temperature, low likelihood tokens are assigned very low probabilities, and high likelihood tokens are assigned very high probabilities.
At high temperature, the probabilities will look roughly similar for each token, with high likelihood tokens assigned only slightly higher probability.

Building off the example above,

At low temperature, there’s a probability that chair is selected, but the probability is significantly lower than cookies.
At high temperature, the probability that chair is selected is only slightly lower than cookies.

The temperature parameter is a value between 0 and 1. As you increase the temperature, the model gets more creative and random. Temperature can be tuned for different problems, and most people will find that the default temperature of 0.3 is a good starting point.

Here's an example of using a low temperature value and running three generations. The result is the same each time.

# Add the user message
message = "I like learning about the industrial revolution and how it shapes the modern world. How can I introduce myself in two words."

# Generate the response multiple times by specifying a low temperature value
for idx in range(3):
    response = co.chat(model="command-r-plus-08-2024",
                       messages=[{'role':'user', 'content': message}],
                       temperature=0)

    print(f"{idx+1}: {response.message.content[0].text}\n")

1: Enthusiastic Historian.

2: Enthusiastic Historian.

3: Enthusiastic Historian.

However, if we increase the temperature to the maximum value of 1, the model gives different generations.

# Add the user message
message = "I like learning about the industrial revolution and how it shapes the modern world. How can I introduce myself in two words."

# Generate the response multiple times by specifying a high temperature value
for idx in range(3):
    response = co.chat(model="command-r-plus-08-2024",
                       messages=[{'role':'user', 'content': message}],
                       temperature=1)

    print(f"{idx+1}: {response.message.content[0].text}\n")

1: Here are two words to introduce yourself based on your interest:

**Revolutionary Historian**

This introduction captures your passion for the Industrial Revolution and its impact on history. It suggests that you have a deep understanding and appreciation for this transformative era, which revolutionized industries and societies worldwide.

2: Two words that could describe you could be:

**History Enthusiast**

3: History Enthusiast.

Conclusion

In this chapter, you learned how to call different variations of the Command model when using the Chat endpoint. You worked with a code example using the temperature parameter to control the Command model’s level of randomness.

Parameters for Controlling Outputs