Going from test-driving AI to delivering it with sustained value is bound to involve a degree of trial and error for every company. But starting that journey with a strong, strategic cast of people in place can minimize those errors while increasing the likelihood of building an effective, scalable, and ethical AI platform.

There’s no one dream-team template to follow – different company sizes, industry sectors, and goals will each bring their own demands. Smaller firms, for instance, may need to hire externally for certain roles or bring on outside contractors (with appropriate safeguards for sensitive data), while larger companies, with their own machine-learning and data-science teams, may already have most of their starting lineup.

But at any scale, a good hypothetical org chart should have the same basic boxes filled in. A recent McKinsey white paper about bringing generative AI to scale emphasized the need for creating a tactical and strategic team, including such common roles as data-ops and dev-ops staffers, site reliability engineers, and cloud and data architects. In addition to these essential team members, here are some other roles and responsibilities that businesses ought to consider.

Do you need a Head of AI?

The rapid development of generative AI has given rise to a relatively new executive: the head of AI. But do you really need one? To be sure, it can be beneficial to have someone who owns the overall AI project, can champion it inside the organization, and be held responsible for its outcomes. But that responsibility doesn’t have to fall to one person. Some organizations are opting for AI committees that bring together leaders from different departments to coordinate AI projects. Other companies are creating “AI centers of excellence” as a way to centralize AI expertise and coordinate its development across departments.

For smaller organizations, it is more common for the CTO or CIO to oversee AI projects as part of their remit and to ensure that they align with the overall business and technology strategy. But for companies that intend AI to be a core engine of growth, appointing a head of AI may make sense in order to have a dedicated leader who has the deep, specialized expertise these complex systems demand.

Who is responsible for safe AI?

Given the high stakes involved, ensuring trust and traceability in an AI engine’s output must be top of mind when creating a team. A large language model’s answers are only as good as the data it draws from and how well that data is organized and connected. Safety-proofing a system against inaccurate results or leaking sensitive data demands a strong team of data scientists and data engineers.

The McKinsey paper endorses having data scientists for such work as fine-tuning foundational models and retrieval-augmented generation (RAG) systems and monitoring how LLMs comply with output guidelines. Data engineers are needed to oversee how these models ingest data and to continuously test their responses against defined pass-fail parameters.

“Language models power many products. Ideally, catching safety issues early on at the model level would be the answer. Unfortunately, so far, upstream checks don’t reliably provide downstream safety,” says Seraphina Goldfarb-Tarrant, Head of AI Safety, Cohere, and author of “The Enterprise Guide to AI Safety.”

To emphasize the importance of safe AI development, companies might benefit from adopting best practices from other industries. For example, newspapers hire an ombudsperson to review reader complaints and investigate what might have gone wrong.

If end users find themselves staring at a screen in anger, dismay, or offense over an AI’s output, they need to know where they can bring a complaint. And there should be a clear path for forwarding that complaint to the developers responsible. An AI ombudsperson can ensure that happens and ward off potential reputational damage, while at the same time provide oversight so that biases in data and models are identified and ethical guidelines followed.

What does a red team do?

Another vital player is a security expert (or experts) who act as a “red team,” probing AI applications for vulnerabilities and risks. They can also act as a prompt-engineering hacker, finding ways to induce the system to generate unhelpful, inaccurate, or harmful responses. AI “hallucinations” can hurt a company’s reputation. Air Canada, for example, recently attracted unwanted coverage when its AI chatbot offered a customer incorrect assurances of a bereavement-fare policy.

Cohere recommends hiring application security engineers who can integrate security into AI products.

“Machine learning and generative AI have so many different and unique attack vectors, and some of them overlap with traditional application security,” security expert Adam (Ads) Dawson says. He also stresses the importance of building a diverse red team that includes perspectives from areas such as machine learning, research, and academia.

This person or team should also be prepared to deal with third-party security researchers. Those outside experts are your allies: They can help flag troublesome outputs, such as disclosures of confidential data, before they cause reputational damage. It’s not enough to post a vulnerability disclosure program; someone must have fielding those reports in their core job description.

Who tracks ROI?

AI can get expensive, often in surprising ways. It’s worth factoring in a project management role or responsibility that keeps a close check on costs and the impact they could have on the company’s business model.

McKinsey’s report highlighted that “run costs are greater than build costs” for GenAI applications. Specifically, the cost of an LLM may add up to no more than 15% of the total project, it found. The report said that change management — orchestrating adjustments to everything “from training your people to role modeling to active performance tracking” — leaves the biggest financial footprint, typically at a 3x multiple of development costs.

“Tracking success for AI implementations requires a multi-layered approach, as the metrics will change depending on the stage of maturity” says McKinsey’s Louise Herring. She goes on to advise, “Companies should have different success indicators for each phase and ensure that there’s room for course correction along the way.”

How do you build strong partnerships?

Even the biggest and most AI-savvy companies need strong external partnerships to complete their dream team, from cloud service providers and vector database platforms to software vendors and academic researchers.

Choosing the right LLM provider may be the single most crucial decision, since building your own can be prohibitively expensive. Based on specific use cases, companies need to choose between open-source and proprietary LLMs. While open-source solutions may have a low sticker price, companies also need to consider the additional cost and internal resource requirements of implementing, launching, and supporting them.

It’s wise to choose an LLM partner that acts as a true business partner (rather than just a vendor). Selecting a provider that works with you to refine systems, identify areas for improvement, and keep up with rapid changes in required knowledge can be the equivalent of having one or more additional team members.

Conclusion

By assessing their current team and considering the key roles outlined above, companies can start putting the right people in place to ensure their AI systems are robust, scalable, and responsible. Add in strong partnerships and a commitment to diversity that reflects the communities your AI will serve, and the dream team will be complete.

Mastering private AI deployments

Is your business ready for a secure, private AI deployment?

Download the eBook

Vinod Devan