Today, we’re launching a set of improved fine-tuning features alongside fine-tuning support for our latest Command R 08-2024 large language model (LLM). These new features and functionality are intended to offer enterprises and developers increased flexibility, control over their data, and visibility into the fine-tuning process.

Impactful in an enterprise setting

Our customers have successfully applied fine-tuning to achieve two key benefits: domain expertise and content customization.

"Cohere's fine-tuning service allows for easy model tuning simply by preparing the data, making it extremely convenient. Even without specialized knowledge, it can be operated intuitively, delivering high-precision results in a short time. This service greatly contributes to business efficiency and project success, and its application is expected across various industries. It will help us develop specialized models based on our original LLM, Takane." - Masahiro Fukuyori, Research Director, Generative AI CPJ, Artificial Intelligence Laboratory, Fujitsu Research, Fujitsu

We have seen customers use fine-tuning to deliver tangible business outcomes across various sectors. For example, companies have improved the accuracy of question-answering in their financial documents, clarified regulatory summaries by simplifying legal jargon and highlighting key points, and tightened email communications with better length control.

We fine-tuned Command R 08-2024 on the ConvFinQA dataset, a popular benchmark focused on a model’s ability to process and answer complex financial queries accurately. We found that Command R 08-2024 fine-tuned delivered near state-of-the-art accuracy, and users are able to do this completely privately in a secure environment.

Performance comparison of Command R 08-2024 with fine-tuning, GPT-4o (2024-08-06), GPT-4o mini (2024-07-18), GPT-4o mini (2024-07-18) with Fine-tuning, and Claude 3.5 Sonnet on ConvFinQA, showing accuracy scores for financial query processing.

Fine-tuned Command R 08-2024 delivers higher efficiency, offering faster time to first token and greater token throughput compared to larger models, making it a performance-efficient choice for many enterprise use cases.

Latency and throughput comparison between Command R 08-2024 fine-tuned deployed on a p5d.48xlarge Amazon SageMaker instance as compared to reported numbers for OpenAI’s GPT-4o (2024-08-06), GPT-4o mini, and Claude 3.5 Sonnet for the same profile.

Feature updates

We are introducing improvements to Cohere’s fine-tuning capabilities, including full control over fine-tuning with a bring your own fine-tune option, support for longer context lengths for fine-tuning training, increased visibility with an integration to Weights & Biases, and parameter-efficient training and inference using LoRA.

Bring your own fine-tune

We are releasing a bring your own fine-tune (BYOFT) option for users. We recognize some users want to use a framework of their choice to fine-tune and with BYOFT we are giving them full flexibility. It enables inference over fine-tuned Command R models exported in standard Hugging Face export format. This is currently only available on Amazon SageMaker and private deployments. We’ve created this developer cookbook to help get you started.

Longer context length

We’ve doubled the supported context length to 16k during training. By being able to handle larger inputs, the fine-tuned model accuracy will improve for long-form tasks like text summarization, analysis, and general retrieval-augmented generation (RAG) use cases.

LoRA support

To enable efficient fine-tuning, we have introduced support for LoRA (Low-Rank Adaptation) fine-tuning. LoRA adds a small set of trainable parameters while keeping the original model weights frozen. This approach reduces computational overhead and memory usage during training, and enables simultaneous inference across multiple fine-tunes reducing the hardware footprint for inference by orders, and enabling scalable model deployments.

Easily monitor performance with Weights & Biases

We've integrated Cohere models with Weights & Biases (W&B) to streamline the monitoring and evaluation of fine-tuning experiments. Users can now customize their models while also enhancing visibility into training efficiency and outcomes using W&B Models, the industry standard system of record to train, fine-tune, and manage AI models.

With W&B, Cohere users can now monitor training and validation loss curves during fine-tuning. This real-time monitoring eliminates the need to wait for jobs to finish before analyzing results, enabling faster iteration cycles and accelerating your workflow. To get started quickly, we've created a developer cookbook. For more details, check out our documentation and the W&B guide.

Pricing and availability

Fine-tuning for Command R 08-2024 is available for businesses and developers today on the Cohere Platform (find pricing here) and Amazon SageMaker, with additional platforms coming soon. Contact sales@cohere.ai for fine-tuning on-prem or within your VPC.

Ye Shen