- What constitutes a suitable model for forecasting the relevant trends for COVID-19, assessing the effects of the pandemic, and informing policy responses?
Models have been widely used to inform policymakers about the implications of coronavirus disease 2019 (COVID-19) and the effects of policy responses to it. The authors of this report reviewed available models and the relevant literature to produce a framework for assessing a model's suitability for policymakers in the Commonwealth of Virginia.
Virginia's government first announced the presence of a COVID-19 case on March 7, 2020. The first interventions were put in place with only a few dozen confirmed cases having been reported, and Virginia had just over 1,000 confirmed cases when the stay-at-home order was implemented.
When assessing whether a model is suitable for informing policy, it is important to consider whether the data being used are relevant, whether the model design is appropriate, how the model has performed in the past, and the degree of transparency in the model. In this report, the authors examine assessment criteria for early-stage and late-stage models, taking into account the effects of the COVID-19 pandemic on behavior, policy, health outcomes, and the economy.
- In the early stage, before the peak in the spread, modeling generally focuses on the timing and level of the peak. Statistical models are most suitable for forecasting trends, rather than comparing policies. Systems dynamics models are well suited for comparing policies, as long as the policies in question operate through mechanisms that are captured by the model.
- Some models will cease to be useful after the early phase, or after certain policy responses to the pandemic have been phased in or out. New models might be needed as the policy environment changes.
- Late-stage models—those examining trends after the peak—should incorporate economic, social, and other concerns, in addition to the health risks, so that policymakers can have an understanding of potential trade-offs.
- Late-stage models should account for the major components of available reopening plans, including the gradual relaxation of physical-distancing measures, widespread testing and contact tracing, and improved health care capacity.
- The extent and efficacy of such measures as widespread testing and contact tracing need to be considered in any model, because these measures likely will play a key role in successful reopening.
- If and when a vaccine is developed, models should help inform the dissemination of the vaccine to minimize the risk of death from COVID-19.
Table of Contents
Assessing the Suitability of Modeling
Assessment Criteria for Early-Stage Models
Assessment Criteria for Late-Stage Models