Contents

Introduction
Features of Amazon SageMaker
Cohere Models on Amazon SageMaker
- List of Models
- Pricing Mechanism
Getting Started with Amazon SageMaker
Conclusion

Introduction

In the previous chapter, we discussed using Amazon Bedrock to access Cohere’s models. Amazon SageMaker provides another option.

Amazon SageMaker is a fully managed service where you can build, train, and deploy ML models at scale using tools like notebooks, debuggers, profilers, pipelines, MLOps, and more — all in one integrated development environment (IDE).

For each of the available models, Cohere ships to Amazon Elastic Container Registry (AWS ECR) the elements required to run a model replica. This includes a container image and the model weights.

You can subscribe to Cohere’s models via AWS Marketplace. Once subscribed, you can create a new inference endpoint deployment from SageMaker. This will create a deployment of a model replica and an associate compute instance that runs the replica within your VPC. The model will be available for communication through your inference endpoint.

When you use a VPC interface endpoint, communication between your VPC and the SageMaker API or Runtime is conducted entirely and securely within an AWS network.

An overview of the Amazon SageMaker solution

Differences Between Amazon Bedrock and SageMaker

What is the difference between SageMaker and Bedrock? While Bedrock is a platform that’s focused on foundational models (FMs), SageMaker is a platform that caters to a much broader range of machine learning (ML) models.

Another key difference is that Bedrock offers a simpler way to get started using Cohere’s models, while SageMaker offers greater control over the underlying infrastructure hosting the models.

The table below summarizes the key differences between Bedrock and SageMaker.

Module	Title	Colab
Scope	Focused on foundation models and generative AI applications	Caters to a much broader range of machine learning models and applications
Ease of Use	Simple API for foundation models, easy to integrate and experiment	Relatively more involved setup requiring more steps to get started
Control	Limited control over infrastructure and model updates, managed entirely by AWS	Greater control over model versions, endpoints, and security configurations
Pricing	Per-token pricing based on tokens processed, as well as separate compute/storage time used for fine-tuning	Per-hour pricing based on software and infrastructure licensing

Features of Amazon SageMaker

SageMaker offers a wide range of services for building, training, and deploying machine learning models. It provides fully managed infrastructure, tools, and workflows to help engineers and data scientists.

Here are some of the relevant features in the context of working with foundational models:

SageMaker Studio: A single, web-based interface for end-to-end ML development, SageMaker Studio provides a fully integrated development environment (IDE) for machine learning, including a visual interface for all ML development activities.
SageMaker Jumpstart: An ML hub that forms a part of the SageMaker Studio experience, SageMaker Jumpstart accelerates ML with built-in algorithms, pre-trained models, and pre-built solutions for common use cases, enabling one-click deployment. Jumpstart also allows users to share ML models and notebooks within their organizations.
Automatic scaling: SageMaker supports automatic scaling for hosted models, dynamically adjusting the number of instances provisioned based on workload changes to optimize resource usage.
Data labeling: SageMaker Ground Truth is a service that helps build high-quality training datasets for machine learning models by combining human and automatic labeling.

Cohere Models on Amazon SageMaker

List of Models

At the time of writing, the following Cohere models are available on SageMaker.

Generative models:
- Command R+
- Command R
- Command
- Command Light
Embeddings models:
- Embed v3 (English & Multilingual)
- Embed Light v3 (English & Multilingual)
- Rerank 2 and 3 (English & Multilingual)
- Classification (English & Multilingual)

To get the most updated list of available models on SageMaker, visit Cohere’s listing on the AWS Marketplace.

Pricing Mechanism

The pricing for Cohere models on SageMaker is broken down into two portions:

Software: Hourly pricing that can vary by instance type
Infrastructure: Hourly pricing that can vary by region and instance type

Getting Started with Amazon SageMaker

Step 1: Set Up Amazon SageMaker

To get set up with SageMaker, follow the steps outlined in the Cohere documentation, which take you through the prerequisites and getting access to Cohere models.

In order to successfully subscribe to Cohere’s offerings on SageMaker, you’ll need the right identity and access management (IAM) permissions, which is described in the documentation linked above.

Step 2: Get Model Access

The guide above also includes the steps for subscribing to Cohere’s models.

Step 3: Test on SageMaker Studio

You can test out the models on SageMaker Studio, which offers a hosted environment in which to access the models on Jupyter notebooks. To do this, follow these steps:

In the AWS Console, go to Amazon SageMaker and click Studio.
Then, click Open Studio. If you don't see this option, you must first set up a SageMaker domain.
A new JupyterLab tab will open. Look for Prebuilt and automated solutions and click JumpStart.
A list of models will appear. In the Foundation Models: Text Generation category, look for Cohere Command and then click View notebook.
This will open up a sample notebook to get started with the model.

Conclusion

This chapter explored Amazon SageMaker, another platform that hosts Cohere's models. Unlike Amazon Bedrock, which focuses on foundational models, SageMaker caters to a wide range of machine learning models, providing greater control over infrastructure. We also looked at SageMaker's features, including SageMaker Studio, SageMaker Jumpstart, automatic scaling, and data labeling. The chapter concluded with a guide on getting started with SageMaker, including setting up, subscribing to Cohere's models, and testing on SageMaker Studio.

In Chapter 3, you’ll learn how to use Cohere Command on Amazon Bedrock for text generation use cases.

Introduction to Cohere on Amazon SageMaker