The authors reviewed relevant bodies of literature on machine learning (ML) and human resource management (HRM), examined Department of the Air Force (DAF) personnel policy documents, and tested ML systems that deliver recommendations to officer developmental education and promotion boards. They were then able to demonstrate the feasibility of applying ML to existing DAF HRM processes.

Leveraging Machine Learning to Improve Human Resource Management

Volume 1, Key Findings and Recommendations for Policymakers

David Schulker, Matthew Walsh, Avery Calkins, Monique Graham, Cheryl K. Montemayor, Albert A. Robbert, Sean Robson, Claude Messan Setodji, Joshua Snoke, Joshua Williams, et al.

ResearchPublished Feb 12, 2024

The national security environment poses strategic challenges for Department of the Air Force (DAF) human resource management (HRM) policies and systems. While the DAF has historically leveraged data analysis to make decisions, such technologies as machine learning (ML) and natural language processing (NLP) could offer opportunities to improve decisionmaking in HRM.

Evidence from other organizations shows that, while adoption of these technologies in general is high, adoption in the area of HRM remains uncommon because of challenges unique to the HRM domain. 

Leveraging data technologies for the kind of strategic impact that senior leaders are calling for requires DAF decisionmakers to systematically select the right mix of projects, effectively execute the development of selected projects, establish procedures for testing decision-support systems to address ethical and legal unknowns, and successfully transition systems into use in a way that is acceptable to the adopting organizations. To address these elements, the authors used a mixed-methods approach that involved (1) reviewing relevant bodies of literature on ML and HRM; (2) examining DAF personnel policy documents; and (3) implementing and testing ML systems that deliver recommendations to officer developmental education and promotion boards. From this review, they extracted evidence-based practices for innovation management and technology transition, along with principles for safe and ethical use of data technologies. From the analysis of DAF personnel policies, they identified high-value use cases. Finally, from their technical case studies, they demonstrated the feasibility of applying ML to existing DAF HRM processes.

Key Takeaways

  • To generate business value by meaningfully contributing to HRM process efficiency and workforce capabilities, the DAF must first grow an ML project portfolio made up of technically feasible projects that address near-term and future HRM needs.
  • To effectively develop ML systems, the DAF must first specify HRM objectives and then select modes of decision support that meet those objectives.
  • To act legally, ethically, and responsibly, the DAF must test candidate systems to ensure that they are safe—that is, accurate, fair, and explainable.
  • To overcome inertia, the DAF must pursue transition pathways that involve gradually increasing the degree of ML influence or, alternatively, gradually increasing the significance of the HRM processes at stake.

Recommendations

  • Manage innovation constantly: Require a well-formulated business case and technical feasibility assessment before projects can move forward. Adopt a portfolio approach to managing complexity.
  • Develop effectively: Begin the design process with priority objectives and consider multiple modes of decision assistance. Prioritize development of ML systems that automatically summarize narrative records as a mode of decision support.
  • Implement safely: Use an accurate-fair-explainable framework to create tailored designs that safely meet objectives. Publish acceptable limits for safety criteria in different classes of use cases to encourage adoption.
  • Transition strategically: Regulate the stakes of the HRM decision and the amount of influence allotted to the ML system to find an implementation that balances value and risk. Apply ML systems to limited cases before gradually expanding their scope and consequence.

Topics

Document Details

Citation

Chicago Manual of Style

Schulker, David, Matthew Walsh, Avery Calkins, Monique Graham, Cheryl K. Montemayor, Albert A. Robbert, Sean Robson, Claude Messan Setodji, Joshua Snoke, Joshua Williams, and Li Ang Zhang, Leveraging Machine Learning to Improve Human Resource Management: Volume 1, Key Findings and Recommendations for Policymakers. Santa Monica, CA: RAND Corporation, 2024. https://www.rand.org/pubs/research_reports/RRA1745-1.html.
BibTeX RIS

Research conducted by

This research was prepared for the Department of the Air Force and conducted within the Workforce, Development, and Health Program of RAND Project AIR FORCE.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.