RAND Corporation researchers developed a framework for assessing the maturity of knowledge products — outputs from health science research. An assessment results in assignment to a knowledge readiness level: scientific foundation, applied knowledge, or application to specific, real-world contexts. Those assignments can help the U.S. Army make better research portfolio decisions.

Development of a Knowledge Readiness Level Framework for Medical Research

Charles C. Engel, Richard Silberglitt, Brian G. Chow, Molly Morgan Jones, Jonathan Grant

ResearchPublished Apr 4, 2019

Given the substantial investments in health research in the United States and globally, a better understanding of health science maturity could have significant influence on the development of research portfolios. To assist the U.S. Army Medical Research and Materiel Command (USAMRMC) in making such investment decisions, RAND Corporation researchers developed knowledge readiness levels (KRLs) for use across different lines of health research. KRLs mark a step forward in thinking about how to assess health research, innovation, and knowledge development.

Military research is requirement-driven and programmed with urgency to resolve priority "gaps" in human performance, operational medicine, training, and care of the ill or injured. A large portion of health research addresses knowledge products (KPs) — knowledge output from research with potential to improve individual or public health.

The goal of this research was to develop and test KRLs to assess KP maturity. The authors defined nine KRLs in three groups: (1) KRLs 1 through 3 provide the scientific foundation for KP development toward practical application. These KPs are the outputs of health research that seeks basic mechanisms and tends to be theoretical or conceptual. (2) KRLs 4 through 6 are for KPs that seek to generate applied knowledge. (3) KRLs 7 through 9 are for KPs resulting from research designed to emphasize external validity of knowledge for use in specific, real-world applications. The authors found that this framework of measuring KP maturity is reliable.

Key Findings

The KRL is a reliable metric of a KP's scientific maturity

  • A KP's scientific maturity can be productively measured in terms of three stages: foundational research, application to human subjects, and application in a real-world context.
  • Each of these three stages of health research has unique characteristics that allow reliable assignment of a specific KP to just one of them.
  • A KP's maturity can be defined using a nine-point Likert-type scale for KRL using a two-step process: (1) Assign the stage and (2) assign the KP's level of maturity within that stage.
  • According to the authors' reliability testing with research publications and real-world testing with research proposals, the KRL and Likert-type scales provide a reliable metric of a KP's scientific maturity.

Recommendations

  • The USAMRMC should adopt and use the KRL as an indicator of scientific maturity. It offers a high degree of conceptual clarity and simplicity, ease of administration, stakeholder satisfaction, and reliable estimates.
  • The KRLs should be adopted for routine use as indicators of KP scientific maturity. However, although the KRL is a sound indicator of scientific maturity, it should not be interpreted as an indicator of health impact: Scientific maturity and health impact are mutually important but orthogonal constructs.

Topics

Document Details

Citation

RAND Style Manual
Engel, Charles C., Richard Silberglitt, Brian G. Chow, Molly Morgan Jones, and Jonathan Grant, Development of a Knowledge Readiness Level Framework for Medical Research, RAND Corporation, RR-2127-OSD, 2019. As of September 11, 2024: https://www.rand.org/pubs/research_reports/RR2127.html
Chicago Manual of Style
Engel, Charles C., Richard Silberglitt, Brian G. Chow, Molly Morgan Jones, and Jonathan Grant, Development of a Knowledge Readiness Level Framework for Medical Research. Santa Monica, CA: RAND Corporation, 2019. https://www.rand.org/pubs/research_reports/RR2127.html.
BibTeX RIS

Research conducted by

This research was sponsored by the the U.S. Army Medical Research and Materiel Command (USAMRMC) with the support of the Defense Health Agency and conducted within the Forces and Resources Policy Center and the Acquisition and Technology Policy Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.