Download Free Electronic Document

FormatFile SizeNotes
PDF file 0.2 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Question

  1. How can the USAMRMC best assess health research, innovation, and knowledge development so that it can create effective and useful research portfolios?

Given the substantial investments in health research in the United States and globally, a better understanding of health science maturity could have significant influence on the development of research portfolios. To assist the U.S. Army Medical Research and Materiel Command (USAMRMC) in making such investment decisions, RAND Corporation researchers developed knowledge readiness levels (KRLs) for use across different lines of health research. KRLs mark a step forward in thinking about how to assess health research, innovation, and knowledge development.

Military research is requirement-driven and programmed with urgency to resolve priority "gaps" in human performance, operational medicine, training, and care of the ill or injured. A large portion of health research addresses knowledge products (KPs) — knowledge output from research with potential to improve individual or public health.

The goal of this research was to develop and test KRLs to assess KP maturity. The authors defined nine KRLs in three groups: (1) KRLs 1 through 3 provide the scientific foundation for KP development toward practical application. These KPs are the outputs of health research that seeks basic mechanisms and tends to be theoretical or conceptual. (2) KRLs 4 through 6 are for KPs that seek to generate applied knowledge. (3) KRLs 7 through 9 are for KPs resulting from research designed to emphasize external validity of knowledge for use in specific, real-world applications. The authors found that this framework of measuring KP maturity is reliable.

Key Findings

The KRL is a reliable metric of a KP's scientific maturity

  • A KP's scientific maturity can be productively measured in terms of three stages: foundational research, application to human subjects, and application in a real-world context.
  • Each of these three stages of health research has unique characteristics that allow reliable assignment of a specific KP to just one of them.
  • A KP's maturity can be defined using a nine-point Likert-type scale for KRL using a two-step process: (1) Assign the stage and (2) assign the KP's level of maturity within that stage.
  • According to the authors' reliability testing with research publications and real-world testing with research proposals, the KRL and Likert-type scales provide a reliable metric of a KP's scientific maturity.

Recommendations

  • The USAMRMC should adopt and use the KRL as an indicator of scientific maturity. It offers a high degree of conceptual clarity and simplicity, ease of administration, stakeholder satisfaction, and reliable estimates.
  • The KRLs should be adopted for routine use as indicators of KP scientific maturity. However, although the KRL is a sound indicator of scientific maturity, it should not be interpreted as an indicator of health impact: Scientific maturity and health impact are mutually important but orthogonal constructs.

This research was sponsored by the the U.S. Army Medical Research and Materiel Command (USAMRMC) with the support of the Defense Health Agency and conducted within the Forces and Resources Policy Center and the Acquisition and Technology Policy Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.

This report is part of the RAND Corporation research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.