In this report, RAND researchers explore an application of artificial intelligence that would enable the U.S. Air Force to leverage existing data for improved human resource management policies and practices. The main purpose of this report is to serve as a worked example for Air Force policymakers as they consider how to approach the potential ways in which artificial intelligence can improve human resource management processes.
Can Artificial Intelligence Help Improve Air Force Talent Management?
An Exploratory Application
Download
Download eBook for Free
| Format | File Size | Notes |
|---|---|---|
| PDF file | 2.6 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Purchase
Purchase Print Copy
| Format | List Price | Price | |
|---|---|---|---|
| Add to Cart | Paperback76 pages | $20.00 | $16.00 20% Web Discount |
Research Questions
- How could an AI-enabled performance-scoring system that mimics the process by which human experts judge performance narratives help senior leaders take full advantage of performance records when making talent management decisions?
- What are the potential consequences and implications of implementing such an AI system?
Both private and public organizations are increasingly taking advantage of improvements in computing power, data availability, and analytic capabilities to improve business processes. These trends have prompted U.S. Department of Defense policymakers to become more interested in whether adopting data-enabled methods would facilitate more-effective management of department personnel. In this report, RAND researchers explore one such application that would enable the U.S. Air Force to leverage existing data for improved human resource management (HRM) policies and practices. Specifically, the researchers develop a performance-scoring system that uses artificial intelligence (AI) and machine learning, which would enable the expanded use of performance narratives in HRM processes. The main purpose of this report is to serve as a worked example (i.e., a step-by-step solution to a problem) for Air Force policymakers as they consider how to approach the potential ways in which AI can improve HRM processes.
Key Findings
- The researchers apply the cross-industry standard process for data-mining to the AI application explored in this analysis. This task requires two primary data inputs: (1) a sufficiently large sample of officer performance narratives and (2) outcome labels that provide information about which narratives indicate the best job performance.
- The researchers find that there is a business need for the AI performance-scoring system. The system could be used as a tool to facilitate policy analysis, assist development teams, enable professional development, or aid in competitive selection decisions.
- Extracting and digitizing large amounts of officer evaluation data from the existing archive is feasible, but, because the process of digitizing the text from older documents requires extensive tuning and computation time, the process could present a potential challenge to future efforts by Air Force practitioners.
- Initial model results are promising: Standard machine-learning algorithms accurately predicted an officer's performance quality by identifying known signals in the performance narrative text without explicit programming.
- Implementation concerns regarding privacy, fairness, explainability, and other unintended consequences are greatest if the system were used to make decisions that would alter officers' careers. These considerations could be less of a barrier to implementing the system for other purposes.
Recommendations
- Policymakers should position systems and policies to take advantage of analytic uses of data that HRM processes generate.
- Natural language processing techniques can reduce the need to pre-quantify information at the data collection stage.
- Human resources managers should consider AI applications to be an enabler of better policies rather than a substitute for human decisionmaking.
Table of Contents
Chapter One
Introduction
Chapter Two
The Business Need for Performance Metrics in Human Resource Management Decisions
Chapter Three
Understanding Officer Evaluation System Data
Chapter Four
Processing Performance Narratives to Create Analytic Data
Chapter Five
Modeling the Relationship Between Performance Narratives and Promotion
Chapter Six
Evaluating Other Implementation Considerations
Chapter Seven
Conclusion and Policy Implications
Research conducted by
Funding for this research was made possible by the independent research and development provisions of RAND’s contracts for the operation of its U.S. Department of Defense federally funded research and development centers. The research was conducted within the Manpower, Personnel, and Training Program of RAND Project AIR FORCE.
This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.