Jan 3, 2024
The Department of the Air Force has become increasingly interested in the potential for artificial intelligence (AI) to revolutionize different aspects of warfighting. But in the context of cybersecurity, distributional shift—the divergence of current data from the data on which an AI system was trained—can significantly reduce the longevity of AI applications. This could be especially dangerous as the threat of cyberattacks continues to grow.
Volume 2, Distributional Shift in Cybersecurity Datasets
Published Jan 3, 2024
|Add to Cart
The Department of the Air Force has become increasingly interested in the potential for artificial intelligence (AI) to revolutionize different aspects of warfighting. For this project, the U.S. Air Force asked RAND Project AIR FORCE to consider broadly what AI cannot do—to understand the limits of AI for warfighting applications. This report presents a discussion of the application of AI systems to perform two common cybersecurity tasks—detecting network intrusions and identifying malware—and the effect of distributional shift on those tasks, a phenomenon that can significantly limit AI effectiveness. Distributional shift occurs when the data that an AI system encounters after it is deployed differ appreciably from the data on which it was trained and tested.
This report describes the importance of distributional shift, how it can and does significantly limit AI effectiveness in detecting network intrusions and identifying malware, how to test for and quantify its effects, and how those effects could be mitigated. This work is aimed primarily at larger organizations, such as headquarters facilities, that have the bandwidth and computing power to implement AI-enabled cybersecurity systems and to update their systems regularly.
This report is the second in a five-volume series addressing how AI could be employed to assist warfighters in four distinct areas: cybersecurity, predictive maintenance, wargames, and mission planning. This volume is intended for a technical audience; the series as a whole is designed for those who are interested in warfighting and AI applications more generally