Understanding the Limits of Artificial Intelligence for Warfighters
Volume 3, Predictive Maintenance
ResearchPublished Jan 3, 2024
The U.S. Air Force deploys flying units with readiness spares packages (RSPs) to try to ensure that the units are stocked with enough parts to be self-sufficient for 30 days. This report presents an assessment of the use of statistical distributions for predicting aircraft parts failure and an evaluation of how artificial intelligence can be used to determine the content of RSPs.
The U.S. Air Force (USAF) deploys flying units with readiness spares packages (RSPs) to try to ensure that the units are stocked with enough parts to be self-sufficient for 30 days. Predicting which parts are likely to fail—and, therefore, which parts should be included in the RSPs—is important because overstocking can be expensive and understocking can threaten mission readiness. This report presents a discussion of whether and when artificial intelligence (AI) methods could be used to improve parts failure analysis, which currently uses a model that assumes a probability distribution. To do this, several machine-learning models were developed and tested on historical data to compare their performance with the optimization and prediction software currently employed by the USAF, using A-10C aircraft data as a test case.
This report is the third in a five-volume series addressing how AI could be employed to assist warfighters in four distinct areas: cybersecurity, predictive maintenance, wargames, and mission planning. This report is aimed primarily at those with an interest in predictive maintenance, RSPs, and AI applications more generally.
Key Findings
- AI can improve failure analysis for RSPs on a case-by-case basis. The current probability based prediction process is a poor predictor of the performance of many parts. The AI model not only made better predictions but also, as a result, was much more cost effective. Updating the current prediction process with data can achieve a performance level that is quite close to the AI model used in this research.
- It is necessary to establish a complex and labor-intensive data operations pipeline to USAF maintenance databases before any large-scale AI implementation can occur. Historical data are essential to train and test AI models, but pulling this data from the relevant systems is a complex, manual process that involves scripting, drop-down lists, and nested menus. Moreover, considerable data cleaning is also needed. Given this situation, leveraging AI is practical only as a proof-of-concept model.
- AI cannot alleviate the scarcity of wartime data. It is unclear whether RSPs developed using peacetime data will be adequate for wartime operations. Moreover, one of the main limitations of AI for this application is its inability to estimate truly rare events, which might be more likely during wartime operations. As a result, different approaches to modeling AI could be required to deal with these changing circumstances. However, regular retraining and updating, which is possible with an AI model, can ensure the adaptability of these models during wartime.
Recommendations
- Air Force Materiel Command (AFMC) should work with USAF Logistics to build a data operations pipeline to conduct retrospective analysis of aircraft maintenance and RSP efficiency. Aircraft maintenance programs and databases function effectively for the purposes for which they were designed, but they clearly were not designed for retrospective analysis or to train AI models. Unless the data can be properly conditioned and pulled for this analysis, none of the following recommendations can be implemented.
- AFMC should experiment with AI to improve failure analysis for RSPs. Extend the proof-of-concept models to all aircraft. This will likely have to be done on a part-by-part, platform-by-platform basis. Automated or partially automated data extraction will likely be necessary if AI is relied on to conduct these analyses. For RSP parts with hard-to-predict rare failures, the AI cost function can be modified to prefer overpredictions or rely on overpredictions via a Poisson distribution or the problem can be modeled as survival analysis (predict time to failure).
- AFMC should limit AI to failure analysis within the RSP process. The ASM software tackles a large and complex operations research problem of selecting which parts to send from which depot to which base. Current AI capabilities are data hungry and better suited to solving narrowly scoped problems. Splitting parts failures across multiple depots and bases will fragment the data too much for algorithms to learn anything useful.
Related Content
Topics
Document Details
- Copyright: RAND Corporation
- Availability: Available
- Year: 2024
- Print Format: Paperback
- Paperback Pages: 47
- Paperback Price: $26.00
- Paperback ISBN/EAN: 9781977412805
- DOI: https://doi.org/10.7249/RRA1722-3
- Document Number: RR-A1722-3
Citation
RAND Style Manual
Chicago Manual of Style
Zhang, Li Ang, Yusuf Ashpari, and Anthony Jacques, Understanding the Limits of Artificial Intelligence for Warfighters: Volume 3, Predictive Maintenance, RAND Corporation, RR-A1722-3, 2024. As of October 16, 2024: https://www.rand.org/pubs/research_reports/RRA1722-3.html
Zhang, Li Ang, Yusuf Ashpari, and Anthony Jacques, Understanding the Limits of Artificial Intelligence for Warfighters: Volume 3, Predictive Maintenance. Santa Monica, CA: RAND Corporation, 2024. https://www.rand.org/pubs/research_reports/RRA1722-3.html. Also available in print form.
Research conducted by
This research was prepared for the Department of the Air Force and conducted within the Force Modernization and Employment Program of RAND Project AIR FORCE.
This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.