Strengthening Federal Capacity to Conduct Evaluations to Inform Future R&D Program Planning

by Daniel Basco

Download eBook for Free

FormatFile SizeNotes
PDF file 1.9 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

This dissertation identifies strategies for strengthening the Federal Government's capacity to evaluate its research and development (R&D) programs using insights identified from philanthropic foundations. While there is broad recognition that Federal investments in R&D have resulted in breakthrough technologies and scientific achievements, there is little evidence to support whether the Federal Government's vast investments are being made wisely. Few systematic evaluations have been conducted to assess whether the structure of these investments and the requirements put on federally funded scientists bolster or discourage impactful work. Federal science agencies and departments must develop better evaluation capacity to conduct the evaluations necessary to inform the design and structure of R&D programs to spur greater, faster, and cheaper innovation.

Strategies for strengthening Federal R&D evaluation capacity were identified through a two-stage approach. In Stage 1, a range of R&D evaluation approaches were identified through interviews with science program officers, evaluation personnel, and leaders within philanthropic foundations that fund R&D in the United States. In Stage 2, a diverse group of Federal science policy, planning, and evaluation personnel participated in discussions and activities to identify which foundation approaches could benefit the Federal Government and how they would need to be adapted to fit a Federal context. This process resulted in a series of strategies to improve Federal science agencies' capacities to conduct R&D evaluations that can inform future program planning and strategy.

This dissertation finds that the Federal Government can strengthen its R&D evaluation capacity using four strategies adapted from philanthropic foundations. The Federal Government could hire additional evaluation-trained personnel within science agencies to provide the expertise needed to improve existing evaluations and develop new evaluation activities that can inform future program planning. Congress and the White House could explore how to centralize oversight and support for R&D evaluations to provide consistent advice to science agencies on how to strengthen evaluation capacity. The Federal Government could also invest in additional evaluation activities to provide the resources needed to conduct large-scale evaluations that are designed to inform future program planning. Finally, the White House and science agencies could improve the design of and resources for R&D evaluations to improve their effectiveness.

Table of Contents

  • Chapter One

    Literature Review

  • Chapter Two

    Foundation and Firm Findings

  • Chapter Three

    Workshop Findings

  • Chapter Four

    Policy Implications and Conclusions

  • Appendix One

    Exploratory Interview Protocol

  • Appendix Two

    Firm Interview Protocol

  • Appendix Three

    In-Depth Foundation Interview Protocols

  • Appendix Four

    Workshop Survey

  • Appendix Five

    Senior Federal Leader Interview Protocol

Research conducted by

This document was submitted as a dissertation in May 2018 in partial fulfillment of the requirements of the doctoral degree in public policy analysis at the Pardee RAND Graduate School. The faculty committee that supervised and approved the dissertation consisted of Dave Baiocchi (Chair), Matthew Lewis, and Kei Koizumi.

This report is part of the RAND Corporation dissertation series. Pardee RAND dissertations are produced by graduate fellows of the Pardee RAND Graduate School, the world's leading producer of Ph.D.'s in policy analysis. The dissertations are supervised, reviewed, and approved by a Pardee RAND faculty committee overseeing each dissertation.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.