We’ll use Cohere’s Python SDK for the code examples. Follow along in this notebook.

Introduction

Large language models (LLMs) enable a long list of text generation use cases. For instance, in business writing, they can assist with creating compelling narratives or drafting precise reports. LLMs are also useful in answering questions accurately and efficiently, drawing from a vast knowledge base. Additionally, they can summarize vast amounts of information, making it easier to glean key insights. Another noteworthy application is rewriting text to, for example, enhance readability or alter the tone to suit different audiences.

Cohere's Command family of models, available on Amazon Bedrock, are powerful LLMs that offer these capabilities. This includes cutting-edge models like Command R and Command R+, which we’ll be using in our code examples in this chapter.

For example, a major luxury goods retailer wanted to help its client advisors, reducing the time advisors needed to find and research products. It engaged Cohere to create a tool that could help advisors suggest products to customers based on demographics, purchase history, and personal preferences.

Using retrieval-augmented generation (RAG), Cohere Command and Rerank were integrated into the customer’s website, product catalog, inventory, and customer service apps to build a conversational virtual shopping assistant.

Implementing a virtual shopping assistant at a major luxury goods retailer

In this chapter, we'll explore how to use Cohere's Command R+ model on Amazon Bedrock. For the most updated list of Cohere models available on Amazon Bedrock, visit the Amazon Bedrock website.

We also have a supplementary video on this topic for further reference:

0:00

/7:00

Code Walkthrough

Let’s walk through how to use Command R+ to power a range of tasks in a customer support agent scenario, including text generation, summarization, rewriting, and extraction. We’ll also use the Chat endpoint to build a simple chatbot that can maintain the memory of a conversation that runs over multiple turns.

Setup

First, let's install and import the necessary libraries and set up our Cohere client using the cohere SDK. To use Bedrock, we create a BedrockClient by passing the necessary AWS credentials.

pip install cohere

import cohere
co = cohere.BedrockClient(
    aws_region="YOUR_AWS_REGION",
    aws_access_key="YOUR_AWS_ACCESS_KEY_ID",
    aws_secret_key="YOUR_AWS_SECRET_ACCESS_KEY",
    aws_session_token="YOUR_AWS_SESSION_TOKEN",
)

Create Contextual Information

Before we begin, let's create some context to use in our text generation tasks. In this example, we'll use a set of technical support frequently asked questions (FAQs) as our context.

# Technical support FAQ
faq_tech_support = """- Question: How do I set up my new smartphone with my mobile plan?
- Answer:
  - Insert your SIM card into the device.
  - Turn on your phone and follow the on-screen setup instructions.
  - Connect to your mobile network and enter your account details when prompted.
  - Download and install any necessary apps or updates.
  - Contact customer support if you need further assistance.

- Question: My internet connection is slow. How can I improve my mobile data speed?
- Answer:
  - Check your signal strength and move to an area with better coverage.
  - Restart your device and try connecting again.
  - Ensure your data plan is active and has sufficient data.
  - Consider upgrading your plan for faster speeds.

- Question: I can't connect to my mobile network. What should I do?
- Answer:
  - Check your SIM card is inserted correctly and not damaged.
  - Restart your device and try connecting again.
  - Ensure your account is active and not suspended.
  - Check for any network outages in your area.
  - Contact customer support for further assistance.

- Question: How do I set up my voicemail?
- Answer:
  - Dial your voicemail access number (usually provided by your carrier).
  - Follow the prompts to set up your voicemail greeting and password.
  - Record your voicemail greeting and save it.
  - Test your voicemail by calling your number and leaving a message.

- Question: I'm having trouble sending text messages. What could be the issue?
- Answer:
  - Check your signal strength and move to an area with better coverage.
  - Ensure your account has sufficient credit or an active plan.
  - Restart your device and try sending a message again.
  - Check your message settings and ensure they are correct.
  - Contact customer support if the issue persists."""

Create the Function to Generate Text

Now, let's define a function to generate text using the Command R+ model on Bedrock. We’ll use this function a few times throughout.

This function takes a user message and generates the response via the chat endpoint. We also define the model as cohere.command-r-plus-v1:0. Note that this is how the model is named on Bedrock, compared to command-r-plus for the same model via the native Cohere platform.

def generate_text(message):
    response = co.chat(message=message,
                       model="cohere.command-r-plus-v1:0")
    return response

Text Generation

Let’s look at the first use case, which is in the broad category of text generation. Given a prompt, the model will generate a corresponding response.

Suppose we have a customer support agent who responds to customer inquiries using the help of an LLM to craft accurate responses. For this, we pass the technical support FAQs as context to the prompt for the LLM. We then structure the prompt to include the instruction, context (FAQs), and the customer inquiry.

We then run the generate_text function on the prompt and get the response. The textual content of the response is stored in the response.text field.

inquiry = "I've noticed some fluctuations in my mobile network's performance recently.The connection seems stable most of the time, but every now and then, I experience brief periods of slow data speeds. It happens a few times a day and is quite inconvenient."

prompt = f"""Use the FAQs below to provide a concise response to this customer inquiry.

# Customer inquiry
{inquiry}

# FAQs
{faq_tech_support}"""

response = generate_text(prompt)

print(response.text)

And here’s the response. It uses the information available from the FAQs and adds its own style for responding.

Response to customer inquiry: 

Brief slowdowns in data speed can be frustrating. Here are some quick steps to improve your mobile data performance: 

1. Check your signal strength: Move to an area with better coverage if needed. 
2. Restart your device: A simple restart can often improve connectivity. 
3. Data plan status: Ensure your data plan is active and you have sufficient data allocated for faster speeds. 
4. Consider a plan upgrade: If the issue persists, consider contacting your provider to discuss faster speed plans. 

Additionally, checking for any network outages in your area and ensuring your SIM card is inserted correctly can also help maintain a stable connection. 

If the issue continues, reach out to your mobile provider's customer support for further guidance and troubleshooting.

Text Summarization

Another type of use case is text summarization. Now, let's summarize the customer inquiry into a single sentence. We add an instruction to the prompt and then pass the inquiry to the prompt.

prompt=f"""Summarize this customer inquiry into one short sentence.

Inquiry: {inquiry}"""

response = generate_text(prompt)

print(response.text)

And here’s the response. As specified in the instruction, it generates a one-sentence summary about the inquiry.

I experience intermittent periods of slow data speeds a few times a day on my mobile network.

Text Rewriting

Another useful use case is rewriting or transforming a piece of text to suit a different need. Typically, we want to modify some aspects of the text while maintaining its overall meaning.

In our example, we want to turn the response from the customer support agent, which is done via chat, into an email format that can be sent to the customer as a follow-up. It follows a similar format for building the prompt: specifying the instruction followed by adding the necessary context.

prompt=f"""Rewrite this customer support agent response into an email format, ready to send to the customer.

If you're experiencing brief periods of slow data speeds or difficulty sending text messages and connecting to your mobile network, here are some troubleshooting steps you can follow:

1. Check your signal strength - Move to an area with better coverage.
2. Restart your device and try connecting again.
3. Ensure your account is active and not suspended.
4. Contact customer support for further assistance. (This can include updating your plan for better network performance.)

Did these steps help resolve the issue? Let me know if you need further assistance."""

response = generate_text(prompt)

print(response.text)

And here’s the response, which is an email version of the customer support agent’s response.

Subject: Troubleshooting Guide for Slow Data and Network Issues

Dear [Customer Name], 

I hope this email finds you well. I understand that experiencing slow data speeds and network connectivity issues can be frustrating. Here are some detailed troubleshooting steps to help resolve these problems: 

- Signal Strength: 
   - Please check your device's signal strength by looking at the signal bars displayed on your screen. 
   - If you're in an area with weak coverage, try moving to a different location with better reception. You can also try going outdoors or near a window to improve signal strength. 

- Device Restart: 
   - Simply restart your device. This step can often resolve minor glitches and connectivity issues. Turn your device off, wait for a few seconds, and ... 
   
(truncated for brevity)

Text Extraction

With text extraction, we can pull out certain pieces of information from a document. In our example, we want to generate a support ticket and populate it with information about the interaction, such as the category of inquiry, product type, and the ticket status. Additionally, we can ask the model to generate the output in a specific format, such as markdown, JSON, HTML, and others. In our example, we ask the model to generate a JSON string.


prompt=f"""Extract the following as a JSON from the text provided below:
- category (Options: technical support, promotions, billing)
- product (Options: broadband, mobile, TV)
- status (Options: open, closed)

# Customer Inquiry
{inquiry}

# Customer Support Agent's Response
If you're experiencing brief periods of slow data speeds or difficulty sending text messages and connecting to your mobile network, here are some troubleshooting steps you can follow:

1. Check your signal strength - Move to an area with better coverage.
2. Restart your device and try connecting again.
3. Ensure your account is active and not suspended.
4. Contact customer support for further assistance. (This can include updating your plan for better network performance.)

Did these steps help resolve the issue? Let me know if you need further assistance."""

response = generate_text(prompt)

print(response.text)

And here’s the response, formatted in JSON as instructed.

{
  "category": "technical support",
  "product": "mobile",
  "status": "open"
}

Building a Chatbot

All our examples so far run on a single-turn interaction. But with the Chat endpoint, we can build a chatbot that keeps a memory of all previous interactions. This allows developers to build chatbot-style applications that maintain the state of a conversation.

The following is an implementation of a simple chatbot in a customer support setting, where the chatbot acts as a helpful customer support agent. We create a function called run_chatbot to implement these steps and print out the key events and messages along the way. This function also optionally accepts the chat history as an argument to keep the state in a multi-turn conversation.

For this, we introduce a couple of additional parameters to the Chat endpoint:

preamble: A preamble contains instructions to help steer a chatbot’s response toward specific characteristics, such as a persona, style, or format. Here, we are using a simple preamble: “You are a helpful customer support agent that assists customers of a mobile network service.”
chat_history: We store the history of a conversation between a user and the chatbot as a list, append every new conversation turn, and pass this information to the next endpoint call.

# Define a preamble
preamble = """## Task and Context
You are a helpful customer support agent that assists customers of a mobile network service."""

# Run the chatbot
def run_chatbot(message, chat_history=[]):
    response = co.chat(message=message,
                       model="cohere.command-r-plus-v1:0",
                       preamble=preamble,
                       chat_history=chat_history)
    
    print(response.text)
    
    chat_history = response.chat_history
            
    return chat_history

And here’s an example conversation that runs over a few turns.

Turn #1:

chat_history = run_chatbot("Hi. I've noticed some fluctuations in my mobile network's performance recently.")

I'm sorry to hear that you've been experiencing issues with your mobile network's performance. Can you tell me a little more about the specific problems you've been facing? For example, are you experiencing dropped calls, slow data speeds, or something else entirely? The more information you can provide, the better I'll be able to assist you in troubleshooting the issue.

Turn #2:

chat_history = run_chatbot("At times, the data speed is very poor. What should I do?", chat_history)

I'm sorry to hear that you're experiencing issues with your data speed. Here are a few troubleshooting steps you can try to improve your mobile data performance: 

- Check your device's signal: Weak or fluctuating signals can lead to poor data speeds. Try moving to a different location or adjusting your device's position to see if that improves your signal strength. 

- Restart your device: Sometimes, ...

(truncated for brevity)

Turn #3:

chat_history = run_chatbot("Thanks, I'll try these.", chat_history)

You're welcome! If you have any other questions or concerns, feel free to reach out. I'm here to help.

Here's what is contained in the chat history after a few turns.

print("Chat history:")
for turn in chat_history:
    print(turn, "\n")

Chat history:
message="Hi. I've noticed some fluctuations in my mobile network's performance recently." tool_calls=None role='USER' 

message="I'm sorry to hear that you've been experiencing issues with your mobile network's performance. Can you tell me a little more about the specific problems you've been facing? For example, are you experiencing dropped calls, slow data speeds, or something else entirely? The more information you can provide, the better I'll be able to assist you in troubleshooting the issue." tool_calls=None role='CHATBOT' 

message='At times, the data speed is very poor. What should I do?' tool_calls=None role='USER' 

message="I'm sorry to hear that you're experiencing issues with your data speed. Here are a few troubleshooting steps you can try to improve your mobile data performance: \n\n- Check your device's signal: Weak or fluctuating signals can lead to poor data ... (truncated for brevity) " tool_calls=None role='CHATBOT' 

message="Thanks, I'll try these." tool_calls=None role='USER' 

message="You're welcome! If you have any other questions or concerns, feel free to reach out. I'm here to help." tool_calls=None role='CHATBOT'

Conclusion

In this chapter, we explored how to use Command R+ on Amazon Bedrock for various text generation and manipulation tasks. We covered text generation, summarization, rewriting, and extraction, demonstrating how these capabilities can be applied to real-world scenarios.

If you want to learn more LLM use cases, visit our LLMU chapter on use case patterns.

In Chapter 4, you’ll learn how to perform semantic search using the Cohere Embed model on Amazon Bedrock.

Contents

Text Generation Using Cohere Command on Amazon Bedrock