on Facebook
on Twitter
on YouTube
Document Information
Comparing Algorithms for Scenario Discovery
While attractive in many ways, traditional scenarios have lacked an appropriate analytic foundation for inclusion in quantitative decision analyses. In previous work, the authors have proposed to remedy this situation with a systematic, analytic process they call “scenario discovery” that has already proved useful in a wide variety of applications. This study aims to evaluate alternative algorithms needed to implement this novel scenario discovery task, in which users identify concise descriptions of the combinations of input parameters to a simulation model that are strongly predictive of specified policy-relevant results. This study offers three measures of merit — coverage, density, and interpretability — and uses them to evaluate the capabilities of PRIM, a bump-hunting algorithm, and CART, a classification algorithm. The algorithms are first applied to datasets containing clusters of known and easily visualized shapes, then to datasets with unknown shapes generated by a simulation model used previously in a decision analytic application. They find both algorithms can perform the required task, but often imperfectly. The study proposes statistical tests to help evaluate the algorithms' scenarios and suggests simple modifications to the algorithms and their implementing software that might improve their ability to support decision analysis with this scenario discovery task.
Free, downloadable PDF file(s) are available below.
RAND makes an electronic version of this document available for free as a public service.
Use Adobe Acrobat Reader version 7.0 or higher for the best experience.
The research described in this report was prepared for the National Science Foundation and conducted by RAND Infrastructure, Safety, and Environment.
This product is part of the RAND working paper series. RAND working papers are intended to share researchers' latest findings, to solicit informal peer review, or to publish a technical appendix to an article published in a scientific journal. They have been approved for circulation by the sponsoring RAND research unit but typically have not been formally edited or peer reviewed. Unless otherwise indicated, working papers can be quoted and cited without permission of the author, provided the source is clearly referred to as a working paper.
Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.
The RAND Corporation is a nonprofit research organization providing objective analysis and effective solutions that address the challenges facing the public and private sectors around the world. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.
* RAND research is conducted across divisions, centers, and projects; these organizational components are represented in the "Related RAND Divisions" section above.
Top
RAND® is a registered trademark. Copyright © 1994-2010 RAND Corporation. Last Modified: December 18, 2009 