When Michael Neumann was posted to Libya as a CIA operative, gathering intelligence was essential to deploy robust counter-intelligence methods.

Even though that was only a decade ago, Neumann, now chief data scientist at national security software provider Second Front, recalls having to use paper maps to track people of interest.

“We didn’t even have basic digital mapping technology,” he said. “That’s ridiculous in this day and age.”

Fast forward to today, and Neumann is at the forefront of efforts to ensure that national security staff and other government workers have far more advanced tools at their disposal, including by leveraging the power of generative AI (GenAI). Second Front delivers mission-critical software to democracies globally. In June, Cohere announced a partnership with Second Front, enabling it to leverage our secure, state-of-the-art models and our North platform to deliver powerful agentic AI solutions through 2F Game Warden, its fully accredited development, security, and operations (DevSecOps) platform.

We recently spoke with Neumann on the challenges facing governments adopting AI and where he sees great potential. Below are some highlights from that conversation.

Porous AI security boundaries are a ‘non-starter’

Neumann’s understanding of security after 15 years with the CIA has helped enable Second Front to better address core government concerns about data security – the most common impediment he sees to faster AI adoption in the national security sphere.

“We're behind in getting cutting-edge SaaS into the hands of mission users and to that end we really started to think about what the impediments are to delivering software securely into these environments,” he says.

“Given the sensitivity of these workloads, oftentimes the idea of data transiting outside of a security boundary, or going to some third-party services providing an LLM, it's a non-starter.”

Before software is deployed in U.S. public agencies, it needs to obtain an Authority to Operate (ATO) – a formal sign-off from the agency’s risk owner that it meets specific security, compliance, and risk-management standards for its intended use. The process of obtaining an ATO can be lengthy and expensive, making it vital for providers to present a robust body of evidence that enumerates vulnerabilities and remediations inherent to the given application. 2F Game Warden automates much of this, speeding up the software ATO process so that technology companies can deliver much-needed AI innovation to government end-users

Cohere’s modular AI solutions with security built in, including through privately deployed LLMs, have further expanded Second Front’s ability to embed secure AI into its solutions to address this fundamental requirement confidently.

“The question is always about data and data flows – data security. I think that’s actually the beauty of Cohere, which is that we can absolutely answer those questions. We own it end to end … It just removes this whole set of questions around the security piece, which is massive,” Neumann says.

U.S. Department of Defense officials say that AI is increasingly central to its digital modernization plans, including integrating the technology into military operations to enhance commanders’ decision-making and responsiveness capabilities. More broadly, the White House has announced a sweeping roadmap aimed at cementing the U.S. as the world leader in AI, calling on federal agencies to remove barriers to its development.

The UK defense minister said in May that AI will increasingly power the country’s military to put it on the “leading edge of innovation” in NATO. Canada’s military has committed to becoming “AI-enabled” by 2030.

In June, Cohere announced agreements with the UK and Canadian governments to collaborate on implementing secure AI solutions and cutting-edge research to strengthen the public sector and national security.

Going forward, Neumann sees the ability to provide secure end-to-end solutions as key to bridging the gap between the jump in public-sector interest in GenAI and lingering institutional and leadership caution.

Addressing ‘black box’ questions

The ability to run everything in a single, secure environment means that agencies can customize AI foundation models on the highly specialized data sets they use without worrying about leaks, Neumann says.

“I think it's a huge value proposition to be able to do that fine-tuning within your boundary, and have the flexibility to do that in a way that your data is remaining under your control.”

Neumann sees a lack of explainability in how foundational models work as another key factor holding back adoption. Although an AI model is really a sophisticated statistical predictor, the fact that it “looks like magic” feeds into leadership concerns about risks, he says. This is where retrieval-augmented generation (RAG) architecture brings advantages. Because RAG draws on vetted and pre-defined sources, it helps to provide users with clarity over the provenance and lineage of data, alleviating some government concerns over the “black box” nature of AI models.

Neumann recognizes that the path to broader government GenAI adoption and more advanced use cases needs to be a steady and cautious one. It would take just a few high-profile failures to erode confidence significantly.

“There’s a huge cultural shift that I think is going to take time. We're going to have to have systems that have repeatedly proved their value.”

The next AI frontier in national security

Still, for Neumann, the speed of transformation that AI has brought to everyday tasks that used to dominate his work at the CIA has been “totalling mind-blowing.” Time-consuming manual processes such as translation, and analyzing and summarizing large documents can now be handled by LLMs in a fraction of the time and with greater accuracy, helping to save lives.

The next frontier, Neumann believes, will be the “operational usage of models” for real-time mission tasks such as triaging and assessing satellite images. Computer vision could be used to help agents identify specialized objects, for example.

These edge-based use cases will need advanced analytics capabilities and have GenAI inference taking place close to where the data is collected while keeping it within a secure boundary.

“The ability to be able to do inference from a given set of endpoints in a secure boundary where they know that they have complete control of their data. That's huge.”

Government AI adoption

Learn how to accelerate secure AI adoption in government with our latest guide.

Read now!

AI adoption for national security: Modernizing defense