Evaluating Simulation-Derived Scenarios for Effective Decision Support

Published in: Technological Forecasting and Social Change, v. 91, Feb. 2015, p. 64-77

Posted on RAND.org on January 01, 2014

by Andrew M. Parker, Sinduja Srinivasan, Robert J. Lempert, Sandra H. Berry

Read More

Access further information on this document at Technological Forecasting and Social Change

This article was published outside of RAND. The full text of the article can be found at the link above.

Scenario planning traditionally relies on qualitative methods to choose its scenarios. Recently, quantitative decision support tools have also begun to facilitate such choices. This study uses behavioral experiments and structured decision-maker interviews to evaluate the results of "scenario discovery," a quantitative method that defines scenarios as sets of future states of the world in which proposed policies fail to meet their goals. Statistical cluster-finding and principal component algorithms applied to large databases of computer simulation model results then help users to identify such scenarios. The two experiments examine the results of this process and demonstrate a user preference for increased accuracy and simplicity achieved through rotating the space of uncertain model input parameters, but primarily when the rotated parameters are conceptually similar. Interviews with experts suggest utility for both qualitatively- and quantitatively-derived scenarios. The former were easier to understand and had the most utility for scoping. The latter were perceived as containing more relevant information and having more utility for understanding tradeoffs and making choices among them. Overall, this study suggests the value of quantitative tools for facilitating scenario choice, while also highlighting the importance of formal evaluation in judging the utility of new methods for decision support.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

Our mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior. To help ensure our research and analysis are rigorous, objective, and nonpartisan, we subject our research publications to a robust and exacting quality-assurance process; avoid both the appearance and reality of financial and other conflicts of interest through staff training, project screening, and a policy of mandatory disclosure; and pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations, disclosure of the source of funding of published research, and policies to ensure intellectual independence. For more information, visit www.rand.org/about/principles.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.