The Air Force Transformational Capabilities Office aims to foster capabilities across initiatives. To show how human-centered, data-enhanced decision processes can be used to determine which concepts to advance into the capability pipeline, the authors used a multimethod qualitative approach and developed a data science tool to extract information from free-text descriptions. They demonstrate the tool and foresight methods in three case studies.
Data-Enabled Approaches for Enhancing the Air Force Transformational Capability Pipeline
Download
Download eBook for Free
Full Document
| Format | File Size | Notes |
|---|---|---|
| PDF file | 10.7 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Research Summary
| Format | File Size | Notes |
|---|---|---|
| PDF file | 0.1 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Purchase
Purchase Print Copy
| Format | List Price | Price | |
|---|---|---|---|
| Add to Cart | Paperback110 pages | $37.00 | $29.60 20% Web Discount |
Research Questions
- What methods can be used to decide which concepts to advance into the Air Force transformational capability pipeline?
- Can data science tools be used to extract information from vast databases of capability gaps, capability needs, and technology solutions?
- What methods can be used to leverage human expertise and creativity in these efforts?
A key goal for the U.S. Air Force's Transformational Capabilities Office (TCO) is fostering transformational capabilities across a variety of initiatives. To propose, develop, and select which concepts to advance into the transformational capability pipeline, the TCO must extract information from many data sources. Machine learning and natural language processing can be used to extract information from text sources; however, subject matter expertise must also be applied and leveraged effectively to provide creative insight and make the best use of extracted information.
To understand how human-centered, data-enhanced (HCDE) decision processes can be used to determine which concepts to advance into the pipeline, the authors used a multimethod qualitative approach that included a review of the relevant literature on development planning and interviews with senior leaders, technical experts, and subject matter experts from the Air Force and the defense community. The synthesis of their analysis revealed opportunities for the TCO to use data science tools to extract information from vast databases of capability gaps, capability needs, and technology solutions and to use a more diverse set of future-focused decision methods — called foresight methods — to leverage human expertise and creativity. They developed and implemented the proof-of-concept Semantic Clustering Analysis and Thematic Exploration Tool to extract information from free-text descriptions of capability gaps and technologies and combined data extraction with foresight methods as part of an HCDE decision process. The authors demonstrate the data science tool and foresight methods in three case studies.
Key Findings
- The TCO's exceptionally broad mandate calls for tools and methods different from those used by other Department of the Air Force (DAF) and Department of Defense organizations.
- Some data sources for capability gaps are widely referenced, but they are not centrally managed; data sources for science and technology solutions are far more varied and diverse, and the volume of data contained across these sources is vast.
- No software tools are systematically used to parse, extract, and summarize the content of capability gap and technology solution sources.
- Modern data science techniques can be used to extract information from free-text descriptions contained in these sources.
- Development planning is a human-centered endeavor that depends on domain knowledge, creativity, and social networks.
- Foresight methods can be used to leverage human expertise and creativity.
- Data science techniques and foresight methods can be integrated to form an HCDE decision process.
Recommendations
- The Air Force Research Laboratory (AFRL) and the TCO should use the concept development and selection process described in this report.
- The AFRL and the TCO should use a software tool, like the one described in this report, to extract information from natural language data sources. As they do so, they should conduct user testing and validation studies to improve the software tool.
- The AFRL should explore alternate natural language processing methods to maximize the utility of information extracted from free-text sources.
- The DAF should curate and standardize key operational capability gap data sources.
- The AFRL, the DAF, and the TCO should enrich key science and technology data sources by purchasing or developing capabilities to cleanse records and merge them with metadata.
- The TCO should expand the use of creative, interactive, expert-driven, and evidence-based foresight methods.
- As a stepping stone to reach full curation and standardization of HCDE capability development planning, the AFRL and the TCO should record human-generated technology pairings for capability gaps.
Table of Contents
Chapter One
Introduction
Chapter Two
Interviews
Chapter Three
A Human-Centered, Data-Enhanced Approach to Identify and Prioritize Technology Concepts
Chapter Four
Data Science Methods to Support an HCDE Decision Process
Chapter Five
Foresight Methods to Support an HCDE Decision Process
Chapter Six
Case Studies
Chapter Seven
Findings and Recommendations
Appendix A
Interview Protocol
Appendix B
Foresight Methods
Appendix C
Data Science Methods
Appendix D
Semantic Clustering Analysis and the Thematic Exploration Tool
Research conducted by
The research reported here was commissioned by the AFRL's TCO (AFRL/RS) and conducted by the Force Modernization and Employment Program within RAND Project AIR FORCE.
This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.