Key Contributors: Manoj Govindassamy, Maxime Brunet, Jeremy Pekmez, Inna Shteinbuk, Elliott Choi

Cohere is launching Model Vault, a dedicated, fully isolated SaaS platform for customers to run Cohere models securely, at scale, and with guaranteed performance.

Model Vault combines the reduced operational overhead of fully managed SaaS with the security and performance advantages of self-hosting. North users deploy their application within a secure VPC while offloading the maintenance-heavy model inference and scaling to their secure, cloud-based Model Vault. The result: lower cost of ownership and faster enterprise adoption.

Model Vault marks a milestone in Cohere’s journey to deliver transformative AI that solves the real-world challenges faced by enterprise customers today. Read on to learn more about what’s on offer, or get started now.

The convenience-control tradeoff

At the heart of enterprise AI transformation lies a fundamental constraint: how to balance speed and scale in adoption with the infrastructure control and compliance requirements of a mature, modern-day organization.

This is increasingly a problem of inference. As enterprises work to incorporate models into more workflows, teams, and products, inference is gradually replacing training as the dominant AI workload. McKinsey now projects that inference will account for the majority of AI compute by the end of the decade, even as per-unit costs continue to fall. For businesses, this shifts the core concern from funding episodic training runs to accounting for inference — a recurring cost that scales directly with increased adoption.

For most enterprises, available deployment options clearly reflect this tension. Multi-tenant SaaS platforms optimize for speed and operational simplicity by amortizing infrastructure costs across multiple customers. However, the lack of workload isolation inherent to shared environments introduces problems like noisy-neighbour effects, enforced rate limits during peak usage, and unpredictable latency. These platforms also tend to limit model configurability, provide little visibility into workload-level performance, and often fall short of the compliance standards of highly regulated enterprises.

Self-hosted deployments — whether on-premises or within a customer-managed VPC — address many of these limitations by offering greater control over infrastructure, performance characteristics, and security boundaries. The capital and operational burden underpinning that control, however, can be prohibitively costly, particularly for teams looking to scale.

With North, Cohere’s enterprise platform for building agentic AI applications, this problem of infrastructure becomes even more acute. Model serving still requires provisioning and managing hardware, but agentic workloads by their design are bursty, multifaceted, and unpredictable.

A new framework for scalable, secure AI deployment

Model Vault is the latest addition to Cohere’s suite of secure deployment options. It is designed to address this tradeoff between convenience and control by transferring the operational overheads of model inference to a secure, Cohere-managed cloud environment.

The result is a SaaS deployment model that removes the constraints of shared infrastructure, such as resource contention and unpredictable performance, while preserving the speed, elasticity, and ease of use associated with multi-tenant SaaS.

To be clear, there is no single deployment approach that fits every organization. For some enterprises, multi-tenant SaaS or self-hosted deployments will remain the right choice, depending on internal capabilities, data strategy, and regulatory requirements. Industries with highly standardized workflows, for example, may meet compliance needs through application-level controls alone and continue to favor shared SaaS platforms.

But for many enterprises, infrastructure management has become a growing barrier to production-grade agentic AI. They want to scale model inference capacity without scaling operational load. Model Vault is built for these teams.

Decouple inference from development

Model Vault is a Cohere-managed solution. That means we assume the full operational burden of production inference: deploying models, managing upgrades and dependencies, provisioning and scaling capacity, and ensuring performance and availability. We deliver Model Vault users 99.9%+ guaranteed availability, backed by production-grade SLOs and latency performance that matches or beats leading managed AI/ML deployment platforms.

This materially lowers the total cost of ownership by removing the need for customers to procure, provision, and operate GPU-backed inference infrastructure. North deployments can now run entirely on CPU-only environments, while the highest overhead ML infrastructure work is done within your Model Vault.

Model Vault lets ML teams scale workloads elastically without pre-allocating capacity or absorbing the risk of idle GPUs. Inference capacity expands and contracts with demand, eliminating the tension between overprovisioning for peak performance and underprovisioning that degrades latency and availability.

In simple terms, Model Vault lets Cohere absorb the operational complexity of model serving, freeing your most valuable asset — your engineers — to focus on moving agentic AI applications from experimentation to production. Inference is not your differentiator.

Throughout, enterprise control is retained where it matters most. North customers keep full ownership of the control plane, covering agent logic, workflow orchestration, conversation states, data storage, pipelines. They have full architectural flexibility over which data and workloads they choose to host on-prem or on their existing VPCs, and which can be handled by Model Vault.

Model Vault is deployed as a logically isolated virtual private cloud. No infrastructure components are shared across customers, ensuring strong isolation for performance, reliability, and security. This includes dedicated network load balancers, reverse proxies, serving middleware, inference servers, and the underlying GPU accelerators.

Teams can now use Model Vault to run unlimited production workloads without competing for scarce capacity. The resources you provision are reserved for you — no more rate limits to maintain platform stability. At the same time, Model Vault dynamically scales inference capacity according to customizable logic, so performance remains consistent even for bursty agentic inference.

Observe and optimize in real time

Model Vault gives MLOps teams unique visibility into how inference workloads behave in production. Through a dedicated, real-time dashboard, teams can track request patterns, latency, token throughput, and resource utilization, enabling them to quickly pinpoint bottlenecks and continuously optimize for efficiency, predictability, and performance as demand evolves.

Setting up a new Vault is quick and easy. Select your desired model and performance tier (S, M, L, XL), the replica range, and click Create Vault. You can now run Cohere models in your system.

Manage your Vault directory and all deployed models from your Cohere dashboard.

Model Vault also equips teams with real-time monitoring of model usage and performance.

Getting started

Model Vault is designed for quick and intuitive onboarding, with minimal manual setup. It can be integrated into your existing North deployment, or serve as a private inference platform for Cohere models used individually.

Here’s how to create your Model Vault in minutes:

Without North:

Go to dashboard.cohere.com. Create an account and log in.
Find Vaults in the side bar, then select “+ New Vault.”
Name your Vault.
Select your Cohere model, and the performance tier.
Specify the desired range of replicas for each selected model. This defines your scaling limits.
Confirm your Vault.

That’s it! Your infrastructure is provisioned and you’ll instantly receive your unique subdomain, API endpoint, and access to your private monitoring dashboard.

With North:

Follow the steps above to create a dedicated Model Vault for your organization.
Make sure you select the models that you need for your North deployment.
Once you have your API endpoint, update the Helm chart in your North deployment, so that the desired models point to that endpoint.
After setup, customers can manage the primary and secondary inference endpoints for Command models in North directly from the admin interface.
Your North deployment is now configured with Model Vault.

Supported models

Model Vault currently supports all of Cohere’s latest state-of-the-art embedding, reranker, and generative models. This includes bundled offerings for our North customers (see table below).

What’s more, Rerank and Embed models, which underpin Cohere’s state-of-the-art AI search and retrieval capabilities, can also be self-served through your Model Vault. That means on-demand access for seamless setup and faster experimentation. Customers interested in self-serve access to our Command models or integrated model bundles can request to join the waitlist.

Model Vault pricing plans are flexible and can be tailored to your team’s workload requirements and budgeting preferences. Speak with one of our team to find what works best for you.

You can also read our documentation for full, step-by-step instructions, as well as our model specs and how-to guides for building agentic AI applications in North.

Build fast. Stay in control.

Start shipping secure, high-performance enterprise AI with Model Vault today. Get started.

Introducing Model Vault: Your private platform for secure and scalable model inference