RAND Corporation researchers developed a framework for assessing the maturity of knowledge products — outputs from health science research. An assessment results in assignment to a knowledge readiness level: scientific foundation, applied knowledge, or application to specific, real-world contexts. Those assignments can help the U.S. Army make better research portfolio decisions.
- How can the USAMRMC best assess health research, innovation, and knowledge development so that it can create effective and useful research portfolios?
Given the substantial investments in health research in the United States and globally, a better understanding of health science maturity could have significant influence on the development of research portfolios. To assist the U.S. Army Medical Research and Materiel Command (USAMRMC) in making such investment decisions, RAND Corporation researchers developed knowledge readiness levels (KRLs) for use across different lines of health research. KRLs mark a step forward in thinking about how to assess health research, innovation, and knowledge development.
Military research is requirement-driven and programmed with urgency to resolve priority "gaps" in human performance, operational medicine, training, and care of the ill or injured. A large portion of health research addresses knowledge products (KPs) — knowledge output from research with potential to improve individual or public health.
The goal of this research was to develop and test KRLs to assess KP maturity. The authors defined nine KRLs in three groups: (1) KRLs 1 through 3 provide the scientific foundation for KP development toward practical application. These KPs are the outputs of health research that seeks basic mechanisms and tends to be theoretical or conceptual. (2) KRLs 4 through 6 are for KPs that seek to generate applied knowledge. (3) KRLs 7 through 9 are for KPs resulting from research designed to emphasize external validity of knowledge for use in specific, real-world applications. The authors found that this framework of measuring KP maturity is reliable.
The KRL is a reliable metric of a KP's scientific maturity
- A KP's scientific maturity can be productively measured in terms of three stages: foundational research, application to human subjects, and application in a real-world context.
- Each of these three stages of health research has unique characteristics that allow reliable assignment of a specific KP to just one of them.
- A KP's maturity can be defined using a nine-point Likert-type scale for KRL using a two-step process: (1) Assign the stage and (2) assign the KP's level of maturity within that stage.
- According to the authors' reliability testing with research publications and real-world testing with research proposals, the KRL and Likert-type scales provide a reliable metric of a KP's scientific maturity.
- The USAMRMC should adopt and use the KRL as an indicator of scientific maturity. It offers a high degree of conceptual clarity and simplicity, ease of administration, stakeholder satisfaction, and reliable estimates.
- The KRLs should be adopted for routine use as indicators of KP scientific maturity. However, although the KRL is a sound indicator of scientific maturity, it should not be interpreted as an indicator of health impact: Scientific maturity and health impact are mutually important but orthogonal constructs.