Chatbots brought large language models (LLMs) into the mainstream. LLMs have been around for a few years, but their adoption was largely limited to the AI community. The launch of AI-powered consumer chatbots has made LLMs accessible to the everyday user, and now they're a hot topic in tech and enterprise circles alike.

This text generation module teaches you how to build LLM chatbots using Cohere’s Chat endpoint.

Command Model

Command is Cohere’s family of text generation models. It generates a response given a prompt or message from a user. It is trained to follow user commands and to be instantly useful in practical business applications, like summarization, copywriting, extraction, and question answering.

Command models are trained with large volumes of instruction-following and conversational data so they can handle the nuances of multi-turn language across a wide range of business use cases.

How an LLM Chatbot Works

To understand how LLM chatbots work, it’s important to develop an understanding of their building blocks. This section focuses on how to build the generative part of a chatbot by looking at how to use a foundational model and added layers of context to generate answers in a conversation style.

The Foundation of an LLM Chatbot

The foundation of an LLM chatbot is an LLM that has been fine-tuned to follow instructions. It can generate a response given a prompt or message from a user. This type of model is tuned to follow instructions and questions, such as “Write a headline for my homemade jewelry product” or “What is the capital of Canada?”.

A message or prompt returning a model response

However, the LLM’s context is limited to only the last message it receives, and it does not consider any previous messages and responses.

Yet, chatbots are characterized by their ability to maintain a conversation with a user, which takes place over multiple interactions.

A baseline LLM’s context is limited to only the last message it receives

The goal of a chatbot is to solve this problem by linking a sequence of interactions into a single instance, allowing the chatbot to hold an ongoing conversation. In doing so, the model’s response can keep a memory of all the previous interactions instead of having to start from scratch every time.

How to Build a Chatbot's Memory

Working off of the baseline generation model above, we can pass multiple interactions to the model as a chronological list of messages and create a memory of the conversation.

First, we can add a message with the system role. A system message contains instructions that steer the chatbot’s response toward specific characteristics, such as a persona, style, or format. For example, if we want the chatbot to adopt a formal style, the system message could be something like "You are a helpful chatbot, trained to assist human users by providing responses in a formal and professional tone."

Then, we append the current user message to the messages list. After the model responds, the application appends the assistant response and the next user message to the same list before making the next Chat API call.

We can repeat this step for any number of interactions until we reach the model’s maximum context length. Context length is the total number of tokens taken up by the prompt and response, and each model has a maximum context length that it can support.

Building a conversation by stitching multiple prompt-response pairs together

This multi-turn framework is what gives chatbots the ability to hold the full context of conversation from start to finish.

Multi-turn conversations can happen when the full context is available

However, building on top of a baseline LLM alone is not sufficient.

Chatbots need to perform well in a wide range of scenarios. To create a robust chatbot that consistently generates high-quality and reliable output, the baseline LLM needs to be adapted specifically to conversations. This means taking the baseline model and fine-tuning it further with a large volume of conversational data.

This is what forms the foundation of Cohere’s Chat endpoint — let’s take a closer look.

Cohere's Chat Endpoint

The Chat endpoint provides a consistent, simplified, and structured way to organize inputs through a messages array, making it easier for developers to build chatbot applications. This added layer includes a stable abstraction and schema, providing more stability to scale and build applications on top of the foundation model.

The Chat endpoint includes all the elements required for an LLM chatbot (as discussed in the previous sections), exposing a simple interface for developers. It consists of the following key components:

System messages: Developers can include a system message to steer the assistant’s behavior, tone, or response format.
Multi-turn conversations: The Chat endpoint builds upon the Command model by accepting conversation history in the messages array.
Application-managed memory: Applications preserve conversation memory by storing prior user and assistant turns and passing the relevant history back to the Chat endpoint.
Structured conversation input: The Chat endpoint gives developers a consistent schema for sending system, user, assistant, and tool-related messages while still allowing control over chatbot configuration.

The Chat endpoint takes care of the underlying logic, exposing a simple interface for developers — The Chat endpoint provides a structured interface for developers

Conclusion

Cohere's Command model and Chat endpoint offer powerful tools for developers looking to harness the potential of LLMs in their applications. Command's capabilities, coupled with the Chat endpoint's developer-friendly interface, allow developers to build text generation applications across diverse business applications.

Introduction to Text Generation