Contents

Introduction
Code Walkthrough
Conclusion

We’ll use Cohere’s Python SDK for the code examples. Follow along in this notebook.

Introduction

Enterprises have a lot of data at their disposal, but much of it is hard to tap into. This is because a good chunk of this data is unstructured, and digging into it to find insights isn't straightforward. Traditional methods, like keyword matching, can only do so much and they often fail to capture the full context and depth of a document.

This is where we introduce the concept of text embeddings. Essentially, text embeddings are numerical representations created by language models that convert text into vectors. They capture and encode the context of a document. These vectors store a wealth of context about the documents they represent, opening up the possibility of various applications, from semantic search and retrieval-augmented generation (RAG) to topic modeling and text classification.

Cohere's Embed model, available on Amazon Bedrock, is a powerful text embeddings model that offers these capabilities. This model supports over 100 languages and is unique among text embedding models due to its emphasis on document quality for applications like semantic search.

For example, a global network security company wanted to build an AI assistant to help its IT professionals respond to queries on network security setup and security best practices. Its security policy documentation repository includes tens of thousands of articles consisting of unstructured data, and its legacy search tools often produce inaccurate results.

They built the AI assistant with retrieval-augmented generation (RAG). They used Cohere Embed for semantic search to generate embeddings of the security policy documentation and Cohere Rerank to further improve retrieval accuracy by selecting the most relevant documents for answer generation. The team then fine-tuned Cohere Command to optimize how it answered user questions.

Implementing an AI assistant at a global network security company

In this chapter, we'll explore how to use Cohere's Embed model on Amazon Bedrock. At the time of writing, users can choose between two models on Amazon Bedrock: Embed English and Embed Multilingual which covers 100+ languages. For the most updated list of models, visit the Amazon Bedrock website.

We also have a supplementary video on this topic for further reference:

0:00

/16:11

Code Walkthrough

Let’s walk through how to use Command R+ to power a range of tasks in a customer support agent scenario, including text generation, summarization, rewriting, and extraction. We’ll also use the Chat endpoint to build a simple chatbot that can maintain the memory of a conversation that runs over multiple turns.

An overview of what we'll cover in the code walkthrough

Setup

First, let's install and import the necessary libraries and set up our Cohere client using the cohere SDK. To use Bedrock, we create a BedrockClient by passing the necessary AWS credentials.

pip install cohere

import cohere
co = cohere.BedrockClient(
    aws_region="YOUR_AWS_REGION",
    aws_access_key="YOUR_AWS_ACCESS_KEY_ID",
    aws_secret_key="YOUR_AWS_SECRET_ACCESS_KEY",
    aws_session_token="YOUR_AWS_SESSION_TOKEN",
)

Create Contextual Information

Before we begin, let's create some context to use in our text generation tasks. In this example, we'll use a set of technical support frequently asked questions (FAQs) as our context.

# Technical support FAQ
faq_tech_support = """- Question: How do I set up my new smartphone with my mobile plan?
- Answer:
  - Insert your SIM card into the device.
  - Turn on your phone and follow the on-screen setup instructions.
  - Connect to your mobile network and enter your account details when prompted.
  - Download and install any necessary apps or updates.
  - Contact customer support if you need further assistance.

- Question: My internet connection is slow. How can I improve my mobile data speed?
- Answer:
  - Check your signal strength and move to an area with better coverage.
  - Restart your device and try connecting again.
  - Ensure your data plan is active and has sufficient data.
  - Consider upgrading your plan for faster speeds.

- Question: I can't connect to my mobile network. What should I do?
- Answer:
  - Check your SIM card is inserted correctly and not damaged.
  - Restart your device and try connecting again.
  - Ensure your account is active and not suspended.
  - Check for any network outages in your area.
  - Contact customer support for further assistance.

- Question: How do I set up my voicemail?
- Answer:
  - Dial your voicemail access number (usually provided by your carrier).
  - Follow the prompts to set up your voicemail greeting and password.
  - Record your voicemail greeting and save it.
  - Test your voicemail by calling your number and leaving a message.

- Question: I'm having trouble sending text messages. What could be the issue?
- Answer:
  - Check your signal strength and move to an area with better coverage.
  - Ensure your account has sufficient credit or an active plan.
  - Restart your device and try sending a message again.
  - Check your message settings and ensure they are correct.
  - Contact customer support if the issue persists."""

Create the Function to Generate Text

Now, let's define a function to generate text using the Command R+ model on Bedrock. We’ll use this function a few times throughout.

This function takes a user message and generates the response via the chat endpoint. We also define the model as cohere.command-r-plus-v1:0. Note that this is how the model is named on Bedrock, compared to command-r-plus for the same model via the native Cohere platform.

def generate_text(message):
    response = co.chat(message=message,
                       model="cohere.command-r-plus-v1:0")
    return response

Text Generation

Let’s look at the first use case, which is in the broad category of text generation. Given a prompt, the model will generate a corresponding response.

Suppose we have a customer support agent who responds to customer inquiries using the help of an LLM to craft accurate responses. For this, we pass the technical support FAQs as context to the prompt for the LLM. We then structure the prompt to include the instruction, context (FAQs), and the customer inquiry.

We then run the generate_text function on the prompt and get the response. The textual content of the response is stored in the response.text field.

inquiry = "I've noticed some fluctuations in my mobile network's performance recently.The connection seems stable most of the time, but every now and then, I experience brief periods of slow data speeds. It happens a few times a day and is quite inconvenient."

prompt = f"""Use the FAQs below to provide a concise response to this customer inquiry.

# Customer inquiry
{inquiry}

# FAQs
{faq_tech_support}"""

response = generate_text(prompt)

print(response.text)

And here’s the response. It uses the information available from the FAQs and adds its own style for responding.

Response to customer inquiry: 

Brief slowdowns in data speed can be frustrating. Here are some quick steps to improve your mobile data performance: 

1. Check your signal strength: Move to an area with better coverage if needed. 
2. Restart your device: A simple restart can often improve connectivity. 
3. Data plan status: Ensure your data plan is active and you have sufficient data allocated for faster speeds. 
4. Consider a plan upgrade: If the issue persists, consider contacting your provider to discuss faster speed plans. 

Additionally, checking for any network outages in your area and ensuring your SIM card is inserted correctly can also help maintain a stable connection. 

If the issue continues, reach out to your mobile provider's customer support for further guidance and troubleshooting.

Text Summarization

Another type of use case is text summarization. Now, let's summarize the customer inquiry into a single sentence. We add an instruction to the prompt and then pass the inquiry to the prompt.

prompt=f"""Summarize this customer inquiry into one short sentence.

Inquiry: {inquiry}"""

response = generate_text(prompt)

print(response.text)

And here’s the response. As specified in the instruction, it generates a one-sentence summary about the inquiry.

I experience intermittent periods of slow data speeds a few times a day on my mobile network.

Text Rewriting

Another useful use case is rewriting or transforming a piece of text to suit a different need. Typically, we want to modify some aspects of the text while maintaining its overall meaning.

In our example, we want to turn the response from the customer support agent, which is done via chat, into an email format that can be sent to the customer as a follow-up. It follows a similar format for building the prompt: specifying the instruction followed by adding the necessary context.

prompt=f"""Rewrite this customer support agent response into an email format, ready to send to the customer.

If you're experiencing brief periods of slow data speeds or difficulty sending text messages and connecting to your mobile network, here are some troubleshooting steps you can follow:

1. Check your signal strength - Move to an area with better coverage.
2. Restart your device and try connecting again.
3. Ensure your account is active and not suspended.
4. Contact customer support for further assistance. (This can include updating your plan for better network performance.)

Did these steps help resolve the issue? Let me know if you need further assistance."""

response = generate_text(prompt)

print(response.text)

And here’s the response, which is an email version of the customer support agent’s response.

Subject: Troubleshooting Guide for Slow Data and Network Issues

Dear [Customer Name], 

I hope this email finds you well. I understand that experiencing slow data speeds and network connectivity issues can be frustrating. Here are some detailed troubleshooting steps to help resolve these problems: 

- Signal Strength: 
   - Please check your device's signal strength by looking at the signal bars displayed on your screen. 
   - If you're in an area with weak coverage, try moving to a different location with better reception. You can also try going outdoors or near a window to improve signal strength. 

- Device Restart: 
   - Simply restart your device. This step can often resolve minor glitches and connectivity issues. Turn your device off, wait for a few seconds, and ... 
   
(truncated for brevity)

Text Extraction

With text extraction, we can pull out certain pieces of information from a document. In our example, we want to generate a support ticket and populate it with information about the interaction, such as the category of inquiry, product type, and the ticket status. Additionally, we can ask the model to generate the output in a specific format, such as markdown, JSON, HTML, and others. In our example, we ask the model to generate a JSON string.


prompt=f"""Extract the following as a JSON from the text provided below:
- category (Options: technical support, promotions, billing)
- product (Options: broadband, mobile, TV)
- status (Options: open, closed)

# Customer Inquiry
{inquiry}

# Customer Support Agent's Response
If you're experiencing brief periods of slow data speeds or difficulty sending text messages and connecting to your mobile network, here are some troubleshooting steps you can follow:

1. Check your signal strength - Move to an area with better coverage.
2. Restart your device and try connecting again.
3. Ensure your account is active and not suspended.
4. Contact customer support for further assistance. (This can include updating your plan for better network performance.)

Did these steps help resolve the issue? Let me know if you need further assistance."""

response = generate_text(prompt)

print(response.text)

And here’s the response, formatted in JSON as instructed.

{
  "category": "technical support",
  "product": "mobile",
  "status": "open"
}

Building a Chatbot

All our examples so far run on a single-turn interaction. But with the Chat endpoint, we can build a chatbot that keeps a memory of all previous interactions. This allows developers to build chatbot-style applications that maintain the state of a conversation.

The following is an implementation of a simple chatbot in a customer support setting, where the chatbot acts as a helpful customer support agent. We create a function called run_chatbot to implement these steps and print out the key events and messages along the way. This function also optionally accepts the chat history as an argument to keep the state in a multi-turn conversation.

For this, we introduce a couple of additional parameters to the Chat endpoint:

preamble: A preamble contains instructions to help steer a chatbot’s response toward specific characteristics, such as a persona, style, or format. Here, we are using a simple preamble: “You are a helpful customer support agent that assists customers of a mobile network service.”
chat_history: We store the history of a conversation between a user and the chatbot as a list, append every new conversation turn, and pass this information to the next endpoint call.

# Define a preamble
preamble = """## Task and Context
You are a helpful customer support agent that assists customers of a mobile network service."""

# Run the chatbot
def run_chatbot(message, chat_history=[]):
    response = co.chat(message=message,
                       model="cohere.command-r-plus-v1:0",
                       preamble=preamble,
                       chat_history=chat_history)
    
    print(response.text)
    
    chat_history = response.chat_history
            
    return chat_history

And here’s an example conversation that runs over a few turns.

Turn #1:

chat_history = run_chatbot("Hi. I've noticed some fluctuations in my mobile network's performance recently.")

I'm sorry to hear that you've been experiencing issues with your mobile network's performance. Can you tell me a little more about the specific problems you've been facing? For example, are you experiencing dropped calls, slow data speeds, or something else entirely? The more information you can provide, the better I'll be able to assist you in troubleshooting the issue.

Turn #2:

chat_history = run_chatbot("At times, the data speed is very poor. What should I do?", chat_history)

I'm sorry to hear that you're experiencing issues with your data speed. Here are a few troubleshooting steps you can try to improve your mobile data performance: 

- Check your device's signal: Weak or fluctuating signals can lead to poor data speeds. Try moving to a different location or adjusting your device's position to see if that improves your signal strength. 

- Restart your device: Sometimes, ...

(truncated for brevity)

Turn #3:

chat_history = run_chatbot("Thanks, I'll try these.", chat_history)

You're welcome! If you have any other questions or concerns, feel free to reach out. I'm here to help.

Here's what is contained in the chat history after a few turns.

print("Chat history:")
for turn in chat_history:
    print(turn, "\n")

Chat history:
message="Hi. I've noticed some fluctuations in my mobile network's performance recently." tool_calls=None role='USER' 

message="I'm sorry to hear that you've been experiencing issues with your mobile network's performance. Can you tell me a little more about the specific problems you've been facing? For example, are you experiencing dropped calls, slow data speeds, or something else entirely? The more information you can provide, the better I'll be able to assist you in troubleshooting the issue." tool_calls=None role='CHATBOT' 

message='At times, the data speed is very poor. What should I do?' tool_calls=None role='USER' 

message="I'm sorry to hear that you're experiencing issues with your data speed. Here are a few troubleshooting steps you can try to improve your mobile data performance: \n\n- Check your device's signal: Weak or fluctuating signals can lead to poor data ... (truncated for brevity) " tool_calls=None role='CHATBOT' 

message="Thanks, I'll try these." tool_calls=None role='USER' 

message="You're welcome! If you have any other questions or concerns, feel free to reach out. I'm here to help." tool_calls=None role='CHATBOT'

Conclusion

In this chapter, we explored how to use Command R+ on Amazon Bedrock for various text generation and manipulation tasks. We covered text generation, summarization, rewriting, and extraction, demonstrating how these capabilities can be applied to real-world scenarios.

If you want to learn more LLM use cases, visit our LLMU chapter on use case patterns.

In Chapter 4, you’ll learn how to perform semantic search using the Cohere Embed model on Amazon Bedrock.

Sign up for our newsletters

I consent to receiving email communications fromCohere

By submitting this form, you agree to our terms of use and privacy policy

©Cohere 2024

Code Walkthrough

Using Cohere Embed in a financial services scenario, let’s build an application that can perform semantic search across financial news in different languages.

Financial analysts need to digest a lot of content, such as financial publications and news media, to stay informed. A semantic search system helps analysts quickly search across numerous article titles in multiple languages for the most relevant articles, saving an enormous amount of time and effort. Unlike traditional keyword-based search, semantic search goes beyond matching exact terms and instead focuses on understanding the underlying concepts and relationships between words and phrases.

An overview of what we'll cover in the code walkthrough

Setup

First, let's install and import the necessary libraries and set up our Cohere client using the cohere SDK. To use Bedrock, we create a BedrockClient by passing the necessary AWS credentials.

# ! pip install cohere pandas hnswlib -q

import pandas as pd
import hnswlib
import re
import cohere
co = cohere.BedrockClient(
    aws_region="YOUR_AWS_REGION",
    aws_access_key="YOUR_AWS_ACCESS_KEY_ID",
    aws_secret_key="YOUR_AWS_SECRET_ACCESS_KEY",
    aws_session_token="YOUR_AWS_SESSION_TOKEN",
)

Download Dataset

We use a dataset (MultiFIN) containing a list of real-world article headlines covering 15 languages (English, Turkish, Danish, Spanish, Polish, Greek, Finnish, Hebrew, Japanese, Hungarian, Norwegian, Russian, Italian, Icelandic, and Swedish). This is an open-source dataset curated for financial natural language processing (NLP) and is available on a GitHub repository.

In our case, we’ve created a CSV file with MultiFIN’s data, as well as a column with translations. We don’t use this column to feed the model; we use it to help us follow along when we print the results for those who don’t speak Danish or Spanish. We point to that CSV to create our dataframe.

url = "https://raw.githubusercontent.com/cohere-ai/cohere-aws/main/notebooks/bedrock/multiFIN_train.csv"
df = pd.read_csv(url)

# Inspect dataset
df.head(5)

Pre-Process Dataset

MultiFIN has over 6,000 records in 15 different languages. For our example use case, we focus on three languages: English, Spanish, and Danish.

For this, we’ll need to do some pre-processing steps. First, we remove the duplicates, remove the languages other than the three we need, and pick the top 80 articles for demonstration purposes.

# Ensure there is no duplicated text in the headers
def remove_duplicates(text):
    return re.sub(r'((\b\w+\b.{1,2}\w+\b)+).+\1', r'\1', text, flags=re.I)

df ['text'] = df['text'].apply(remove_duplicates)

# Keep only selected languages
languages = ['English', 'Spanish', 'Danish']
df = df.loc[df['lang'].isin(languages)]

# Pick the top 80 longest articles
df['text_length'] = df['text'].str.len()
df.sort_values(by=['text_length'], ascending=False, inplace=True)
top_80_df = df[:80]

# Language distribution
top_80_df['lang'].value_counts()

Embed and Index Documents

Now, we want to embed our documents and store the embeddings. The embeddings are very large vectors that encapsulate the semantic meaning of our document. In particular, we use Cohere’s embed-multilingual-v3.0 model, which creates embeddings with 1,024 dimensions.

With the v3.0 embeddings models, we need to specify the input_type parameter to indicate the nature of the document. In semantic search applications, this is either search_document, which is for the documents to search, or search_query, which is for the search query that we’ll define later.

We also keep track of the language and translation of the document to enrich the display of the results.

Next, we create a search index using the hnsw vector library. This stores the embeddings in an index, which makes searching the documents more efficient.

# Embed documents
docs = top_80_df['text'].to_list()
docs_lang = top_80_df['lang'].to_list()
translated_docs = top_80_df['translation'].to_list() #for reference when returning non-English results
doc_embs = co.embed(texts=docs,
                    model="cohere.embed-multilingual-v3",
                    input_type='search_document').embeddings

# Create a search index
index = hnswlib.Index(space='ip', dim=1024)
index.init_index(max_elements=len(doc_embs), ef_construction=512, M=64)
index.add_items(doc_embs, list(range(len(doc_embs))))

Send Query and Retrieve Documents

Next, we build a function that takes a query as input, embeds it, and finds the four headers more closely related to it.

# Retrieval of 4 closest docs to query
def retrieval(query):
    # Embed query and retrieve results
    query_emb = co.embed(texts=[query],
                         model="cohere.embed-multilingual-v3",
                         input_type="search_query").embeddings
    doc_ids = index.knn_query(query_emb, k=3)[0][0] # we will retrieve 4 closest neighbors
    
    # Print and append results
    print(f"QUERY: {query.upper()} \n")
    retrieved_docs, translated_retrieved_docs = [], []
    
    for doc_id in doc_ids:
        # Append results
        retrieved_docs.append(docs[doc_id])
        translated_retrieved_docs.append(translated_docs[doc_id])
    
        # Print results
        print(f"ORIGINAL ({docs_lang[doc_id]}): {docs[doc_id]}")
        if docs_lang[doc_id] != "English":
            print(f"TRANSLATION: {translated_docs[doc_id]} \n----")
        else:
            print("----")
    print("END OF RESULTS \n\n")
    return retrieved_docs, translated_retrieved_docs

Let’s now try to query the index with a couple of examples, one each in English and Danish.

queries = [
    "Can data science help meet sustainability goals?", # English example
    "Hvor kan jeg finde den seneste danske boligplan?" # Danish example - "Where can I find the latest Danish property plan?"
]

for query in queries:
    retrieval(query)

Here’s the response for the English query, showing semantic search in action. Notice how the retrieval system was able to surface documents similar in meaning, i.e., data science vs. AI. Keyword-based search systems would not be able to capture this.

QUERY: CAN DATA SCIENCE HELP MEET SUSTAINABILITY GOALS? 

ORIGINAL (English): Using AI to better manage the environment could reduce greenhouse gas emissions, boost global GDP by up to 38m jobs by 2030
----
ORIGINAL (English): Quality of business reporting on the Sustainable Development Goals improves, but has a long way to go to meet and drive targets.
----
ORIGINAL (English): Only 10 years to achieve Sustainable Development Goals but businesses remain on starting blocks for integration and progress
----
END OF RESULTS

Here’s the response for the Danish query. This example highlights the ability to perform cross-lingual searches with the Embed Multilingual model. You can enter a query in one language and get relevant search results in other languages.

Another observation here is that the English acronym “PP&E” stands for “property, plant, and equipment,” and the model was able to connect it to the query.

QUERY: HVOR KAN JEG FINDE DEN SENESTE DANSKE BOLIGPLAN? 

ORIGINAL (Danish): Nyt fra CFOdirect: Ny PP&E-guide, FAQs om den nye leasingstandard, podcast om udfordringerne ved implementering af leasingstandarden og meget mere
TRANSLATION: New from CFOdirect: New PP&E guide, FAQs on the new leasing standard, podcast on the challenges of implementing the leasing standard and much more 
----
ORIGINAL (Danish): Lovforslag fremlagt om rentefri lån, udskudt frist for lønsumsafgift, førtidig udbetaling af skattekredit og loft på indestående på skattekontoen
TRANSLATION: Bills presented on interest -free loans, deferred deadline for payroll tax, early payment of tax credit and ceiling on the balance in the tax account 
----
ORIGINAL (Danish): Nyt fra CFOdirect: Shareholder-spørgsmål til ledelsen, SEC cybersikkerhedsguide, den amerikanske skattereform og meget mere
TRANSLATION: New from CFOdirect: Shareholder questions for management, the SEC cybersecurity guide, US tax reform and more 
----
END OF RESULTS

Conclusion

Semantic search applications, enabled by text embeddings, offer a significantly more effective approach to retrieving and analyzing information. Cohere's Embed model can do this across over 100 languages. Its application in fields like financial analysis, as demonstrated in this chapter, shows how it can transform data retrieval and processing tasks, saving time and improving accuracy.

In Chapter 5, we’ll switch to Amazon SageMaker and look at an example using the Cohere Rerank model on SageMaker.

Semantic Search Using Cohere Embed on Amazon Bedrock