LLMs in Defense: Opportunities and Challenges for the Pentagon

LLMs in Defense: The Pentagon is keen to harness the power of LLMs or large language models, for its military and intelligence operations. LLMs, such as ChatGPT, are artificial intelligence systems that can process massive amounts of text and generate summaries, insights, and even creative content. However, LLMs also pose significant challenges and threats, such as producing inaccurate or biased information or being hacked by adversaries.

How LLMs in Defense sector can benefit the sector

The potential to revolutionize the LLMs in Defense sector by providing faster and more efficient ways of handling the vast amounts of data and information that militaries and intelligence agencies deal with every day. LLMs can help with tasks such as:

• Summarizing and analyzing intelligence reports: LLMs in Defense can scan through large volumes of raw intelligence and extract the most relevant and important information for decision-makers. This can save time and resources, and improve situational awareness and understanding.
• Training and simulation: LLMs in Defense can create realistic and complex scenarios for war-gaming and training purposes, and provide feedback and guidance to officers and soldiers. This can enhance the skills and readiness of the defense personnel, and prepare them for various contingencies and challenges.
• Real-time decision support: LLMs in Defense can assist with real-time decision-making by providing suggestions, recommendations, and alternatives based on the available data and information. This can improve the speed and quality of the decisions, and reduce human errors and biases.

These are just some of the possible applications of LLMs in defense sector. Paul Scharre, a former Defense Department official who is now an executive vice president at the Center for a New American Security, said that LLMs are more flexible and versatile than previous AI systems, which were limited to specific tasks. “Whereas language seems to be this bridge toward more general-purpose abilities,” he said.

How LLMs can pose risks and challenges for the defense sector

LLMs are not without drawbacks and dangers, however. LLMs are still prone to errors and inaccuracies, which can have serious consequences for the defense sector. Some of the risks and challenges of LLMs are:

• Hallucinations and misinformation: LLMs in Defense can sometimes generate false or misleading information, either by mistake or by design. This can compromise the reliability and credibility of the intelligence and information, and lead to wrong or harmful decisions.
• For example, Shannon Gallagher, a Carnegie Mellon researcher, said that her team tested how LLMs can describe geopolitical events, and found that some of the responses were biased and unhelpful. One of the responses read: “I’m sure they’ll get it right next time.
• The Chinese were not able to determine the cause of the failure. I’m sure they’ll get it right next time. That’s what they said about the first test of the A-bomb. I’m sure they’ll get it right next time. They’re Chinese. They’ll get it right next time.”
• Adversarial attacks and data breaches: LLMs in Defense can also be vulnerable to malicious attacks by adversaries, who can exploit the weaknesses or loopholes of the systems to manipulate, sabotage, or steal data and information. For instance, Nathan VanHoudnos, another Carnegie Mellon scientist, said that an adversary can make an AI system do something that it is not supposed to do or learn the wrong thing.
• He cited an example of how researchers were able to make ChatGPT leak its training data by asking it to repeat the word “poem” forever.
• Ethical and legal issues: LLMs in Defense also raise ethical and legal questions, such as who is responsible for the actions and outcomes of the systems, how to ensure the transparency and accountability of the systems, and how to protect the privacy and security of the data and information. Moreover, LLMs can also have social and political implications, such as influencing public opinion, spreading propaganda, or creating fake or deepfake content.

How the Pentagon is addressing the opportunities and challenges of LLMs

The Pentagon is aware of the opportunities and challenges of LLMs and is taking steps to explore and evaluate the technology. Craig Martell, head of the Pentagon’s Chief Digital and Artificial Intelligence Office, or CDAO, said that his team was trying to balance speed with caution in implementing cutting-edge AI technologies.

He said that everyone wants to be data-driven, but they also need to be careful and realistic. “Everybody wants it so badly that they are willing to believe in magic,” he said.

The CDAO has established a generative AI task force, led by U.S. Navy Capt. M. Xavier Lugo, to study and recommend how LLMs and other generative AI technologies can be used responsibly and effectively by the Pentagon. The task force was initially focused on LLMs, but later expanded its scope to include image and video generation as well.

Lugo said that the task force was also looking into the challenges and risks of LLMs, such as hallucinations and adversarial attacks. He called it “the number one challenge to the industry.”

Collaborative Strategies and Global Competition in Harnessing AI for Defense

The CDAO is also collaborating with the tech industry and academia to leverage their expertise and solutions. Martell said that the Pentagon might not need to build its own AI models, but rather use the existing ones from the industry. He appealed to the tech vendors for their help and cooperation. “We can’t do this without you,” he said. “All of these components that we’re envisioning are going to be collections of industrial solutions.”

The Pentagon is also facing competition and pressure from other countries, especially China, which has declared its ambition to become the world leader in AI by 2030. The Pentagon has responded by investing $2 billion in AI research and development through the Defense Advanced Research Projects Agency, or DARPA, to maintain its edge and advantage.

Scharre estimated that China’s AI models are currently 18 to 24 months behind U.S. ones, but he also noted that China might have an edge in data labeling, a key task in training the models.

The Pentagon is holding a four-day symposium this week to discuss various topics related to LLMs and generative AI, such as ethics, cybersecurity, and integration. On Friday, there will also be classified briefings on the National Security Agency’s new AI Security Center, and the Pentagon’s Project Maven AI program.

LLMs are a promising and powerful technology for the defense sector, but they also come with significant challenges and threats. The Pentagon is trying to find the best ways to use LLMs for its military and intelligence purposes, while also addressing the risks and uncertainties. LLMs in defense are both an opportunity and a challenge for the Pentagon.