Implications for strategic stability of attacks on military AI systems

What is the issue?

Strategic competition in artificial intelligence (AI), including in the military domain, is intensifying, while global governance has not kept pace. The tactical implications of military AI are well documented: autonomous targeting, navigation in robotic systems, and decision-support tools for commanders are relatively well understood. Growing attention is now directed toward AI’s strategic effects and its influence on strategic stability.

Although research on the strategic implications of AI and governance options is expanding, it largely focuses on the potential benefits of effective AI systems. Little attention has been paid to how the disruption or denial of AI systems could threaten strategic stability. As a result, governance proposals rarely address these risks. Yet as AI is increasingly integrated in decision making and reliance on these systems grows, the consequences of adversarial interference will likewise intensify.

How did we help?

This project, commissioned by the UK Foreign Commonwealth and Development Office (FCDO), offers insights into how adversarial AI attacks targeting AI systems integrated in the military or critical national infrastructure may affect strategic stability, and offers recommendations to decrease the probability and impact of adversarial attacks on AI systems for the UK.

What did we find?

We analysed five scenarios to illustrate the breadth of implications of adversarial AI attacks for strategic stability, resulting in the following findings:

The (perceived) vulnerability of AI systems to adversarial attacks undermines the credibility of UK defence to adversaries and allies.
Adversarial AI may tip the offence-defence balance in favour of the offence.
Ambiguity, distinguishing ‘normal’ AI malfunction due to model fragility and deliberate adversarial attacks, and the difficulty of attribution create opportunities for miscalculation and mistrust.
Empowerment of non-state actors through democratisation of technology, and specifically adversarial AI, further complicates attribution.
Entanglement of conventional and nuclear C3 increases the risk of an accidental attack on NC3, risking escalation.

Comprehensive findings can be found in the report.

What can be done?

Governments should treat adversarial AI as a strategic stability risk, rather than only as a technical challenge for cybersecurity or AI assurance. While improving the robustness of AI systems remains important, policy responses also need to address how states would recognise, respond to, and recover from adversarial AI attacks, particularly where these could affect military decision making, critical infrastructure or crisis stability.

Based on our research, concerned nations and wider government agencies should prioritise the following actions:

Embedding adversarial AI risks in wider AI governance debates and international forums, so these risks are considered alongside broader questions of military AI development and use.
Improving information-sharing with allies and partners on threats, incidents, attribution challenges and emerging adversarial AI techniques.
Conducting after-action reviews after significant adversarial AI incidents to identify causes, assess strategic effects and reduce the likelihood of recurrence.
Building national expertise across foreign policy, defence, security, cyber and AI assurance communities, so risks are understood beyond technical specialist teams.
Strengthening the robustness and resilience of AI-enabled military systems, NC3, dual-use C3 and critical infrastructure, including recovery plans and fallback options.
Developing clear red lines, response options and communication channels with competitors and adversaries to reduce the risk of miscalculation, misattribution or escalation.