Strengthening Federal Capacity to Conduct Evaluations to Inform Future R&D Program Planning

This dissertation identifies strategies for strengthening the Federal Government's capacity to evaluate its research and development (R&D) programs using insights identified from philanthropic foundations. While there is broad recognition that Federal investments in R&D have resulted in breakthrough technologies and scientific achievements, there is little evidence to support whether the Federal Government's vast investments are being made wisely. Few systematic evaluations have been conducted to assess whether the structure of these investments and the requirements put on federally funded scientists bolster or discourage impactful work. Federal science agencies and departments must develop better evaluation capacity to conduct the evaluations necessary to inform the design and structure of R&D programs to spur greater, faster, and cheaper innovation.

Strategies for strengthening Federal R&D evaluation capacity were identified through a two-stage approach. In Stage 1, a range of R&D evaluation approaches were identified through interviews with science program officers, evaluation personnel, and leaders within philanthropic foundations that fund R&D in the United States. In Stage 2, a diverse group of Federal science policy, planning, and evaluation personnel participated in discussions and activities to identify which foundation approaches could benefit the Federal Government and how they would need to be adapted to fit a Federal context. This process resulted in a series of strategies to improve Federal science agencies' capacities to conduct R&D evaluations that can inform future program planning and strategy.

This dissertation finds that the Federal Government can strengthen its R&D evaluation capacity using four strategies adapted from philanthropic foundations. The Federal Government could hire additional evaluation-trained personnel within science agencies to provide the expertise needed to improve existing evaluations and develop new evaluation activities that can inform future program planning. Congress and the White House could explore how to centralize oversight and support for R&D evaluations to provide consistent advice to science agencies on how to strengthen evaluation capacity. The Federal Government could also invest in additional evaluation activities to provide the resources needed to conduct large-scale evaluations that are designed to inform future program planning. Finally, the White House and science agencies could improve the design of and resources for R&D evaluations to improve their effectiveness.