Download eBook for Free

FormatFile SizeNotes
PDF file 0.3 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

While attractive in many ways, traditional scenarios have lacked an appropriate analytic foundation for inclusion in quantitative decision analyses. In previous work, the authors have proposed to remedy this situation with a systematic, analytic process they call “scenario discovery” that has already proved useful in a wide variety of applications. This study aims to evaluate alternative algorithms needed to implement this novel scenario discovery task, in which users identify concise descriptions of the combinations of input parameters to a simulation model that are strongly predictive of specified policy-relevant results. This study offers three measures of merit — coverage, density, and interpretability — and uses them to evaluate the capabilities of PRIM, a bump-hunting algorithm, and CART, a classification algorithm. The algorithms are first applied to datasets containing clusters of known and easily visualized shapes, then to datasets with unknown shapes generated by a simulation model used previously in a decision analytic application. They find both algorithms can perform the required task, but often imperfectly. The study proposes statistical tests to help evaluate the algorithms' scenarios and suggests simple modifications to the algorithms and their implementing software that might improve their ability to support decision analysis with this scenario discovery task.

The research described in this report was prepared for the National Science Foundation and conducted by RAND Infrastructure, Safety, and Environment.

This report is part of the RAND Corporation working paper series. RAND working papers are intended to share researchers' latest findings and to solicit informal peer review. They have been approved for circulation by RAND but may not have been formally edited or peer reviewed.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.