Evolving the Practice of Evaluating Research-Portfolios
Nov 12, 2019
RAND Corporation researchers reviewed metrics for evaluating research portfolios used by medical research organizations and appraised their characteristics and trade-offs associated with their use. This report presents the results, categorized using a logic-model approach, to help organizations responsible for research portfolios to select, develop, and revise the metrics they use in their evaluations.
|PDF file||1.1 MB||
Use Adobe Acrobat Reader version 10 or higher for the best experience.
|Add to Cart||Paperback124 pages||$34.00||$27.20 20% Web Discount|
The effectiveness of research, like that of other activities, can be evaluated at different levels — the individual project, a group of projects or program, or a larger grouping that might include multiple programs (a portfolio). Focusing on options for research portfolio evaluation, RAND Corporation researchers found many metrics in use or recommended for federal agencies and private, research-supporting organizations and organized them in a taxonomy. This report presents the characteristics and utility of these metrics, organized by individual stages in a logic-model framework, mapping portfolio metrics to the upstream stages of inputs, processes, and outputs and the downstream stages of outcomes and impacts. At each stage, categories of metrics are composed of sets of metric types, each of which is, in turn, composed of individual metrics. In addition to developing this taxonomy, the authors appraised key attributes of portfolio evaluation metrics and described the trade-offs associated with their use. This structured, annotated compilation can help the Defense Health Agency and other entities that evaluate research portfolios to select, develop, or revise the metrics they use.
General Findings and Considerations
Overview of Identified Metrics
Conclusions for the Psychological Health Center of Excellence, the Defense and Veterans Brain Injury Center, and the Defense Health Agency
Variety in Evaluation Frameworks and Tools
Suggested Prioritization of Metrics
Additional Data on Metrics
Stakeholder Interview Topic Guide
Stakeholders Consulted for the Study
The Research Portfolio Management Data Dictionary of the Former Defense Centers of Excellence for Psychological Health and Traumatic Brain Injury
This research was sponsored by what was then known as the Defense Centers of Excellence for Psychological Health and Traumatic Brain Injury (DCoE) and conducted within the Forces and Resources Policy Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.
This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.