Today, Cohere is introducing Rerank 3 Nimble: the newest foundation model in our Cohere Rerank model series, built to enhance enterprise search and RAG systems, that is ~3x faster than Rerank 3 while maintaining a high level of accuracy.

This graph illustrates the tradeoffs between Rerank 3 models. Accuracy on Enterprise Tasks is an average score on various external and internal benchmarks. Requests Per Second (RPS) was measured internally based on the capacity of a single instance of Rerank 3 Nimble to process a request profile of 50 documents with 1024 tokens.

Functionally, reranking models take a query and a set of text documents as input. They assess each text document according to its relevance to the provided query and return a reordered list with the most relevant documents at the top. They’re used in search systems to increase the relevancy of results.

We see customers using Cohere Rerank for two primary purposes. It can be added to existing search systems to improve the relevancy of results or it can be added to RAG systems to increase the relevancy of documents retrieved (and reduce operating costs associated with processing irrelevant information).

Rerank 3 Nimble is available in both an English and a Multilingual version that supports 100+ languages. It can be implemented with just a few lines of code, offers the ability to rerank very long documents (tens of thousands of tokens), and excels with multi-aspect (e.g. email) and semistructured data (e.g. JSON documents, tabular data).

0:00

/0:33

3x Improved Throughput & Lower Latency

Earlier this year, we released Rerank 3 to help businesses build more advanced search systems. Customers like Bluedot, an infectious disease intelligence platform, are using this model in production to advance their search capabilities. We heard from businesses, however, that certain high volume workloads require a reranking model which optimizes primarily for throughput. Rerank 3 Nimble meets that need, offering ~3x greater throughput versus Rerank 3.

*This graph illustrates the results of an internal evaluation using a request profile of 50 documents at 1024 tokens.*

Similarly, Rerank 3 Nimble cuts latency by ⅓ versus to Rerank 3. In industries such as retail, where website drop-off increases with every 100ms added to search response time, having a faster AI model powering your search system translates to higher conversion rates and more revenue.

Rerank 3 Nimble can also be combined with our efficient Command R generative model series to build RAG applications. The addition of Cohere Rerank allows developers to pass fewer, more relevant documents to the generative language model when producing grounded generations.

(left) RAG workflow without Rerank. Documents are retrieved from an existing search system and are passed directly to the LLM for grounded generation. (right) RAG workflow with Rerank. Documents are retrieved from an existing search system and are passed directly to Rerank. High precision semantic reranking allows fewer, higher quality documents to be passed to the LLM for grounded generation.

Rerank 3 Nimble helps enterprises scale their advanced search systems to meet the most demanding production workloads.

Enterprise-Ready Performance & Capabilities

Companies across many industries use Cohere Rerank to improve their enterprise search and RAG systems. Rerank 3 Nimble can be used to…

Enhance existing search systems using BM25 or other traditional search algorithms
Enable customers to accurately search complex documentation
Build applications that understand 100+ languages
Retrieve the most relevant information from various datastores for RAG systems

For example, Atomicwork, an IT service management company, improved the accuracy of their RAG-enabled AI digital assistant (Atom AI) by +20% by adding Cohere Rerank.

While Rerank 3 Nimble prioritizes speed and efficiency, the model continues to offer a high level of accuracy across important enterprise tasks.

**BEIR** is a benchmark focused on out-of-domain information retrieval. Originally it consisted of 18 datasets, but now, just 14 are publically available (and due to license changes for the Twitter API, one dataset can no longer be accessed). We benchmarked all 18 datasets, but focused on the 14 publicly available datasets to allow easier reproduction. **Code** *evaluation accuracy based on nDCG@10 on Codesearchnet, Stackoverflow, CosQA, Human Eval, MBPP, DS1000 (higher is better).* **Semi-structured (JSON)** *retrieval accuracy based Recall@5 on TMDB-5k-Movies, WikiSQL, nq-tables, and Cohere annotated datasets (higher is better).* **Long context** *accuracy based on nDCG@10 on TREC 2019-2022, Conditional QA , NQ-Hard, Qasper, Genomics, QMSum, FinanceBench (higher is better)*.

Getting Started

Rerank 3 Nimble is only available on Amazon SageMaker and for on-premise deployments. It is priced the same as existing Rerank 3 models. You can access the two model versions at the following links:

Rerank 3 Nimble will be available on Amazon Jumpstart on July 30, 2024

To understand how your company can start deploying with Rerank 3 at production-scale, reach out to our sales team. If you are a developer with feedback about your experience using Cohere Rerank, we would love to hear from you. Please send us an email at support@cohere.com.

Martin Hentschel