- What should the Cooperative Biological Engagement Program (CBEP) measure to evaluate program performance?
- How should CBEP systematically identify metrics?
- What metrics do we recommend for CBEP?
The Cooperative Biological Engagement Program (CBEP) is the biological threat component of the Cooperative Threat Reduction program. It grew out of efforts to address risks associated with legacy biological agents, related materials, and technical expertise developed as part of the biological weapon program in the former Soviet Union. CBEP now partners with about 20 countries in different regions around the world and works with them to address diverse threats to international security, including terrorist organizations seeking to acquire pathogens of security concern; human, animal, and agricultural facilities operating with inadequate safety and security safeguards; and the spread of diseases with potential security or economic consequences. As the program has evolved since its inception two decades ago, so too have its content and approaches to performance measurement. The objective of the research reported here was to build on existing work to develop a comprehensive evaluation framework and recommend metrics for assessing and communicating progress toward CBEP's goals. The report ultimately recommends a number of qualitative and quantitative indicators of CBEP performance, some that can be implemented immediately, some to be implemented later.
To Inform Metric Selection, the Cooperative Biological Engagement Program (CBEP) Needs to
- Strengthen enduring partner capabilities for biorisk management
- Strengthen enduring partner capabilities for biosurveillance.
These Objectives Can Be Placed into a Conceptual Framework Built on Logical Models
- Logic models establish the sequence from program inputs to activities, outputs, outcomes, and impacts.
- Separate logic models were developed for biosafety, biosecurity, and biosurveillance, each addressing capacities, capabilities, and sustainability enablers.
This Framework Can Be Used to Identify a Candidate Measures
- Using these models, an initial list of some 300 metrics was identified from the literature. This list was scored and winnowed down into those metrics that scored most strongly in multiple dimensions.
- Two final lists of recommended metrics emerged, one for use now, one for later introduction.
- CBEP should take steps to refine and implement the metrics framework to support internal evaluations and external reporting on program impacts. To refine the framework by identifying implementation challenges and mitigation strategies, conduct pilot tests of the recommended metrics in a small set of CBEP partner countries. This approach would allow CBEP to identify challenges and opportunities for refinement in advance of widespread implementation.
- CBEP should also consider moving evaluation efforts forward by focusing on assessing program outcomes in areas key to ultimate program success. Evaluations of lessons learned and best practices would position CBEP to support effectiveness by improving alignment of program activities with program objectives.
Table of Contents
Introduction and Background
Using a Conceptual Framework to Identify Metrics for CBEP
Conceptual Model and Recommended Metrics for Biorisk Management
Conceptual Model and Recommended Metrics for Biosurveillance
Implementing the Recommended Metrics Framework
Aggregating Proposed Metrics to Report on CBEP Performance to Support Decisionmaking at Several Levels
Conclusions and Next Steps
Implementation Information for Proposed Metrics — Aligned to Logic Model Framework
Implementation Information for Proposed Metrics — Aligned to Functional Framework