Contents

Introduction
Cohere's Model Deployment Options
What Is Amazon Bedrock?
Features of Amazon Bedrock
Cohere Models on Amazon Bedrock
- List of Models
- Pricing Mechanism
Example Use Cases
Getting Started with Amazon Bedrock
Conclusion

Introduction

The most common way to access Cohere’s large language models (LLMs) is through the Cohere platform, which is fully managed by Cohere and accessible through an API.

But that’s not the only way to access Cohere’s models. In an enterprise setting, companies might require more control over where and how the models are hosted. The good news is that Cohere provides enterprises with flexibility and choice around how they want to deploy the models.

In this module, Cohere on AWS, you’ll learn how to deploy Cohere’s LLMs through Amazon Web Services (AWS). In this first chapter of the module, you’ll get an overview of Amazon Bedrock, AWS’s fully managed service for deploying foundational models (FMs).

Cohere's Model Deployment Options

Specifically, Cohere offers four types of deployment options:

Cohere’s SaaS Platform and API
- This is the fastest and easiest way to start using Cohere’s models. The models are hosted on Cohere infrastructure and available on our public SaaS platform (which provides an API data opt-out), which is fully managed by Cohere.
Cloud AI Services (Managed)
- These managed services enable enterprises to access Cohere’s models while easily integrating the service and dealing with sensitive data. In this scenario, Cohere’s models are hosted on the cloud provider’s infrastructure. Cohere is cloud-agnostic, meaning you can deploy our models through any cloud provider. Examples include Amazon Bedrock, Amazon SageMaker, Google Vertex AI, OCI Generative AI, and Azure AI.
Private Deployment - Cloud
- Private deployments (cloud) allow enterprises to deploy the Cohere stack privately on cloud platforms. With AWS, Cohere’s models can be deployed in an enterprise’s AWS Cloud environment via their own VPC (EC2, EKS).
Private Deployment - On-Premises
- Private deployments (on-premises) allow enterprises to deploy the Cohere stack privately on their own compute.

Amazon Bedrock (introduced in this chapter) and Amazon SageMaker (introduced in the next chapter) fall under the second category (Cloud AI Services).

An overview of Cohere’s four deployment options

What Is Amazon Bedrock?

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs).

Using Amazon Bedrock, you can customize the models with your data using techniques such as fine-tuning and retrieval-augmented generation (RAG), and build agents that execute tasks (tool use) using your enterprise systems and data sources.

Amazon Bedrock is serverless, so you don't have to manage any infrastructure. You can securely integrate generative AI capabilities into your applications using the AWS services that you are already familiar with.

Features of Amazon Bedrock

The following are some key features that come with Bedrock.

Model fine-tuning
- Bedrock allows companies to adapt Cohere’s models to meet their individual needs. It enables developers to fine-tune a model’s performance and optimize it for specific use cases, thus empowering you to build and scale generative AI applications.
Data security and privacy
- Amazon Bedrock helps to ensure that your data stays under your control. When you fine-tune a foundation model, we base it on a private copy of that model. This means that your data is not shared with model providers, and it is not used to improve the base models. You can use AWS PrivateLink to establish private connectivity from your Amazon Virtual Private Cloud (VPC) to Amazon Bedrock without exposing your VPC to internet traffic. Finally, Bedrock is in scope for common compliance standards, including ISO, SOC, and CSA STAR Level 2, it is HIPAA eligible, and customers can use Bedrock in compliance with the GDPR.
Provisioned throughput
- Bedrock makes it easy to get started with Cohere’s models without worrying about the underlying infrastructure as that is handled by AWS. That said, if you want greater control over a model’s performance, you can opt for the provisioned throughput option. When you configure Provisioned Throughput for a model, you receive a level of throughput at a fixed cost.
Knowledge bases
- This fully managed capability helps you implement the entire retrieval-augmented generation (RAG) workflow, from ingestion to retrieval and prompt augmentation, without having to build custom integrations to data sources and manage data flows. Session context management is built in, so your app can readily support multi-turn conversations.
Guardrails
- Guardrails for Amazon Bedrock evaluates user inputs and FM responses based on use case-specific policies, and it provides an additional layer of safeguards regardless of the underlying FM. You can create multiple guardrails, each configured with a different combination of controls, and use them across different applications and use cases.
Model evaluation
- Amazon Bedrock supports model evaluation jobs. The results of a model evaluation job allow you to compare model outputs and then choose the model best suited for your downstream generative AI applications.

Note: the list of features will continue to evolve and expand. To keep up with the latest, visit the Bedrock website.

Cohere Models on Amazon Bedrock

List of Models

The most up-to-date list of available Cohere’s models is on the Bedrock website, but here is the list at the time of writing:

Generative models:
- Command R+
- Command R
- Command
- Command Light
Embeddings models:
- Embed (English)
- Embed (Multilingual)

Pricing Mechanism

The pricing for Cohere models on Bedrock at the time of writing falls under two types:

On-demand and batch
- This applies to default Cohere models on Bedrock. Pricing is based on tokens and broken down into:
  - Price per 1,000 input tokens
  - Price per 1,000 output tokens
Customization (fine-tuning)
- This applies to fine-tuned Cohere models on Bedrock. Pricing is broken down into:
  - Price to train 1,000 tokens
  - Price to store each custom model per month
  - Price to infer from a custom model per model unit per hour (with no-commit Provisioned Throughput pricing)

To get the most updated pricing mechanism with further details, visit the Bedrock pricing page.

Example Use Cases

The following are some example use cases that you can build on top of Cohere’s models on Bedrock.

Generative AI: Build applications that can write product descriptions, help draft emails, suggest example press releases, and much more.
Chat: Combine the power of text generation with a conversational interface. Build powerful chatbots and interactive knowledge assistants that give users accurate answers from connected enterprise data, citing sources if required.
Semantic Search and Retrieval: Easily build powerful search solutions in English and over 100 other languages using Cohere’s industry-leading Embed model. Cohere provides the Embed model, connectors to common enterprise data sources, and customization tools to maximize search relevance by domain.
RAG: Enhance the performance of generative AI applications by enabling a model to access and utilize supplementary information from external documents, thereby improving the accuracy of its responses. The RAG approach significantly reduces the hallucination issue common in LLMs.

Getting Started with Amazon Bedrock

We’ll start implementing actual use cases in later chapters, but for now, let’s first get set up with Amazon Bedrock.

Step 1: Set Up Amazon Bedrock

To get started, follow the steps outlined on the Bedrock page, and in particular, be sure to do the following:

Step 2: Choose Your Models

After completing the step above, you’ll reach the page where you can select models to access. To access Cohere’s models, locate the list of Cohere’s models and select the ones you want.

A screenshot of Bedrock’s model selection page

Step 3: Test on the Playground

In Step 1, you established console and playground access. The Bedrock playground provides a console environment that allows you to experiment with running inference on different models and with different configurations before deciding to use them in your application. You can test out the models, such as Command, and get a feel for their capabilities. Enter a prompt, adjust the configurations (if needed), and you’ll get the output generated.

Conclusion

This chapter introduced Cohere’s deployment options: its SaaS platform, Cloud AI platforms, virtual private cloud, and private deployments. We also introduced Amazon Bedrock, a fully managed service by AWS that hosts foundational models, including Cohere’s LLMs. We also went through the steps to get set up with Bedrock.

In Chapter 2, you’ll learn about another way to access Cohere’s models on AWS: Amazon SageMaker.

Introduction to Cohere on Amazon Bedrock