Cover: Can Artificial Intelligence Help Improve Air Force Talent Management?

Can Artificial Intelligence Help Improve Air Force Talent Management?

An Exploratory Application

Published Jan 19, 2021

by David Schulker, Nelson Lim, Luke J. Matthews, Geoffrey E. Grimm, Anthony Lawrence, Perry Shameem Firoz

Download

Download eBook for Free

FormatFile SizeNotes
PDF file 2.5 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Purchase

Purchase Print Copy

 Format Price
Add to Cart Paperback76 pages $20.00

Research Questions

  1. How could an AI-enabled performance-scoring system that mimics the process by which human experts judge performance narratives help senior leaders take full advantage of performance records when making talent management decisions?
  2. What are the potential consequences and implications of implementing such an AI system?

Both private and public organizations are increasingly taking advantage of improvements in computing power, data availability, and analytic capabilities to improve business processes. These trends have prompted U.S. Department of Defense policymakers to become more interested in whether adopting data-enabled methods would facilitate more-effective management of department personnel. In this report, RAND researchers explore one such application that would enable the U.S. Air Force to leverage existing data for improved human resource management (HRM) policies and practices. Specifically, the researchers develop a performance-scoring system that uses artificial intelligence (AI) and machine learning, which would enable the expanded use of performance narratives in HRM processes. The main purpose of this report is to serve as a worked example (i.e., a step-by-step solution to a problem) for Air Force policymakers as they consider how to approach the potential ways in which AI can improve HRM processes.

Key Findings

  • The researchers apply the cross-industry standard process for data-mining to the AI application explored in this analysis. This task requires two primary data inputs: (1) a sufficiently large sample of officer performance narratives and (2) outcome labels that provide information about which narratives indicate the best job performance.
  • The researchers find that there is a business need for the AI performance-scoring system. The system could be used as a tool to facilitate policy analysis, assist development teams, enable professional development, or aid in competitive selection decisions.
  • Extracting and digitizing large amounts of officer evaluation data from the existing archive is feasible, but, because the process of digitizing the text from older documents requires extensive tuning and computation time, the process could present a potential challenge to future efforts by Air Force practitioners.
  • Initial model results are promising: Standard machine-learning algorithms accurately predicted an officer's performance quality by identifying known signals in the performance narrative text without explicit programming.
  • Implementation concerns regarding privacy, fairness, explainability, and other unintended consequences are greatest if the system were used to make decisions that would alter officers' careers. These considerations could be less of a barrier to implementing the system for other purposes.

Recommendations

  • Policymakers should position systems and policies to take advantage of analytic uses of data that HRM processes generate.
  • Natural language processing techniques can reduce the need to pre-quantify information at the data collection stage.
  • Human resources managers should consider AI applications to be an enabler of better policies rather than a substitute for human decisionmaking.

Research conducted by

Funding for this research was made possible by the independent research and development provisions of RAND’s contracts for the operation of its U.S. Department of Defense federally funded research and development centers. The research was conducted within the Manpower, Personnel, and Training Program of RAND Project AIR FORCE.

This report is part of the RAND research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.