The Federal Emergency Management Agency commissioned development of metrics that can inform decisionmaking for awarding predisaster mitigation grants. The authors established three lines of effort (LoEs) for analysis: indirect benefits, applicant institutional capability, and community resilience. Each LoE produced a metric or framework that could support grant decisionmaking and program performance evaluation.

Developing Metrics and Scoring Procedures to Support Mitigation Grant Program Decisionmaking

Joshua Mendelsohn, Grant Johnson, Kelly Klima, Rachel Steratore, Samantha Cohen, Geoffrey Kirkwood, Lloyd Dixon, Jaime L. Hastings, Paul S. Steinberg

ResearchPublished Mar 31, 2021

Cover: Developing Metrics and Scoring Procedures to Support Mitigation Grant Program Decisionmaking
Download PDF
Additional Downloads

Includes Research Summary

The Federal Emergency Management Agency (FEMA) launched the Building Resilient Infrastructure and Communities (BRIC) program to award predisaster mitigation grants. FEMA asked the Homeland Security Operational Analysis Center (HSOAC) to develop metrics—quantitative measurements of important concepts—that can inform decisionmaking for the BRIC program. Building from discussions with program leadership, a review of stakeholder comments, and a close reading of BRIC's legal requirements, the authors established three lines of effort (LoEs) for analysis.

The indirect benefits line reviewed published measurement techniques and blended them into instructions for an input-output simulation model that better measures the full benefit to a community of mitigating an asset. The applicant institutional capability (AIC) line reviewed analogous research and interviewed subject-matter experts to develop a checklist for assessing the ability of applicants to propose or execute mitigation projects, focusing on staff retention, skills, and experience, as well as management capacity and technical capacity. The community resilience line developed an assessment framework based on BRIC's legal requirements, discussions with BRIC leadership, and standard best practices in measurement. Then, the LoE conducted a preliminary review of published resilience metrics, highlighting the potential value of action-based community resilience metrics for performance evaluation, population-based metrics for equity evaluation, and building code–based metrics as needed to improve statutory compliance. Each LoE produced a metric or framework for assessing metrics that could support BRIC grant decisionmaking and program performance evaluation. The report concludes with 11 recommendations for FEMA to consider.

Key Takeaways

The indirect benefits line found that input-output (I-O) and computable general equilibrium (CGE) are the two most widely accepted modeling strategies for estimating the economic benefits of hazard mitigation.

  • Although each type of model contains certain advantages and disadvantages, I-O models are the most practical near-term option for estimating the economic benefits of hazard mitigation.
  • The indirect benefits line provides detailed instructions for how to model economic benefits using an I-O approach.

The AIC line developed a checklist for assessing the ability of applicants to propose or execute mitigation projects.

  • The most important internal factors for high-performing applicants were an appropriately trained and skilled workforce, prior experience, and access to management and technical capabilities.
  • The specific evaluation criteria include general and key staff turnover, staff skill/expertise and prior experience with predisaster mitigation projects and grants, management/administration capabilities, and access to technical expertise to propose predisaster mitigation projects.
  • External factors outside an applicant's control—such as disaster activity and weather delays—also influence project performance.

The community resilience line developed an assessment framework for evaluating community resilience measures based on BRIC's legal requirements, discussions with leadership, and standard best practices.

  • Measures based on actions that communities can take may be more useful to BRIC than measures based on difficult-to-change census population characteristics.
  • Vulnerability measures based on difficult-to-change census population characteristics may be better suited to the measurement of equity gaps in program outcomes.
  • Building codes are heavily emphasized in the legal framework guiding disaster recovery and measure an important aspect of resilience. However, they are often missing from resilience measures.

Recommendations

  • Develop a strategy for addressing known disaster relief fund cost drivers.
  • In the near term, use a tailored I-O model to measure communitywide indirect benefits of mitigation projects.
  • Integrate the indirect benefits model into Hazus and automate as much as possible.
  • Decide whether a CGE model or an I-O model best suits the long-term needs of the BRIC program.
  • Evaluate applicant capability to propose or execute high-quality mitigation projects and develop strategies for supporting lower-capability applicants.
  • In evaluating applicant capability, focus on staff retention, staff skill and experience, management capability, and technical capability.
  • Periodically assess community resilience measures and encourage usage at FEMA of measures that performed well on the assessment.
  • In evaluating community resilience, focus on resilience, statutory compliance, scientific validation, and practicality.
  • Use action-based community resilience metrics to evaluate performance.
  • Consider population-based community resilience metrics to evaluate equity gaps.
  • Consider building code adoption and enforcement metrics to improve statutory compliance if these are not already part of the community resilience metric chosen.

Topics

Document Details

Citation

Chicago Manual of Style

Mendelsohn, Joshua, Grant Johnson, Kelly Klima, Rachel Steratore, Samantha Cohen, Geoffrey Kirkwood, Lloyd Dixon, Jaime L. Hastings, and Paul S. Steinberg, Developing Metrics and Scoring Procedures to Support Mitigation Grant Program Decisionmaking. Homeland Security Operational Analysis Center operated by the RAND Corporation, 2021. https://www.rand.org/pubs/research_reports/RRA377-1.html.
BibTeX RIS

Research conducted by

This research was sponsored by the Federal Emergency Management Agency (FEMA) and conducted within the Acquisition and Development Program of the Homeland Security Operational Analysis Center.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.