Jan 30, 2009
Large-scale bioterrorism attacks and other public health emergencies are likely to stretch the ability of state and local health departments to provide rapid delivery of life-saving medications and other medical supplies to their populations. In 2004, the U.S. Department of Health and Human Services established the Cities Readiness Initiative (CRI) to help the nation's largest metropolitan areas develop the ability to conduct mass dispensing.
In 2007, the Centers for Disease Control and Prevention (CDC) asked RAND to provide an initial evaluation of the CRI program. Because not enough time has passed for the effects of CRI to be fully observable at all sites and the data sets required to support the evaluation are still being developed, the study focused on whether the program has succeeded in catalyzing activities critical to developing plans and building key capacities.
The study found that CRI appears to have improved communities' preparedness to conduct mass dispensing. The study found that there is merit in provisionally continuing the program, which would allow CDC and others to monitor the program's progress and reassess its impacts after two to three years.
CRI builds on a preexisting program known as the Strategic National Stockpile (SNS), which maintains caches of medications and other medical supplies that can be requested by states and communities in the event of a large-scale public health emergency. Often described as "SNS-plus," CRI provides additional funds, guidance, and technical assistance to the nation's most populous metropolitan areas.
CRI requires awardees to develop the ability to dispense antibiotics to 100 percent of the population of a metropolitan region within 48 hours of the decision to do so—a goal that is based on an outdoor anthrax bioterrorism attack scenario. The program currently involves 72 metropolitan regions and covers approximately 57 percent of the U.S. population.
Based on RAND's initial examination, CRI appears to have improved regions' readiness to dispense life-saving medications and other medical supplies on a large scale and under rapid timelines. Specifically, CRI has enabled awardees to
CRI produced these improvements primarily through two mechanisms. First, the program's focus on a single scenario (aerosolized anthrax attack) helped communities prioritize resource allocation and reach out to key partners. Participants noted that CRI's aggressive goal of dispensing medication to an entire region within 48 hours brought a concrete focus to planning efforts and helped create a sense of urgency among partners that was not there previously. Second, the specific targets established through the program's Technical Assistance Review tool (TAR) have created accountability and served as a basis for planning tasks and milestones. The TAR assesses a jurisdiction's planning in 12 core functional areas, such as command and control, public information and communication, security, inventory control, and training. Each area includes between five and 15 individual items, which are rated on a simple scale indicating the degree to which the jurisdiction has completed each item.
The study found some variability in the program's effectiveness across communities, which depends on a number of site-specific factors.
CRI may also have affected other aspects of public health emergency preparedness, but this was not evaluated systematically in the study. First, the program might have spillover effects on other areas of public health emergency preparedness not emphasized by the anthrax scenario, such as medical surveillance, medical surge, or mass evacuations. Similarly, to the extent that the CRI program's emphasis on preparation for distributing medical countermeasures changes the funding priorities of public health departments, it could potentially affect capabilities in other areas of public health.
In addition, the study was unable to assess jurisdictions' ability to implement their mass dispensing plans in emergency conditions due to the lack of evidence on which to base an assessment. Operational exercises provide a potential source of data; however, the lack of standardized performance metrics and uniform standards limits the ability to use exercises to evaluate operational capabilities at this time. The study also could not evaluate whether the gains cited justify the costs of the program.
Given the improvements that CRI appears to have brought about in staffing, equipment, planning, partnerships, and dispensing strategies, there is merit in continuing the program. However, a decision to continue the program needs to be accompanied by specific steps to build on the program's existing strengths and ensure the continuing development of systems for measurement and evaluation.
Reassess the impact and renewal of the program after two to three years. Future evaluation efforts should confirm whether CRI awardees have continued to build capacities and capabilities and consider whether to revise the program's goals (e.g., focus on a different planning function, emphasize sustaining rather than creating readiness, include other metropolitan regions in the program).
Continue development of systems for measurement and evaluation. Existing efforts to develop better tools for evaluation and measurement should be continued in order to support a robust reevaluation of the program and continuous program improvement. Emphasis might be given to developing finer-grained budget data, standard drill-based metrics, and cost-benefit analyses.
Continue improvements in technical assistance and assessment. Efforts are currently under way to improve training for consultants who provide technical assistance, and these efforts seem worth continuing. Attention should also be paid to ensuring the ability of the relevant state health department to provide assistance to local health departments in meeting CRI goals.