Comparing Algorithms for Scenario Discovery
Download eBook for Free
|PDF file||0.3 MB||
Use Adobe Acrobat Reader version 10 or higher for the best experience.
While attractive in many ways, traditional scenarios have lacked an appropriate analytic foundation for inclusion in quantitative decision analyses. In previous work, the authors have proposed to remedy this situation with a systematic, analytic process they call “scenario discovery” that has already proved useful in a wide variety of applications. This study aims to evaluate alternative algorithms needed to implement this novel scenario discovery task, in which users identify concise descriptions of the combinations of input parameters to a simulation model that are strongly predictive of specified policy-relevant results. This study offers three measures of merit — coverage, density, and interpretability — and uses them to evaluate the capabilities of PRIM, a bump-hunting algorithm, and CART, a classification algorithm. The algorithms are first applied to datasets containing clusters of known and easily visualized shapes, then to datasets with unknown shapes generated by a simulation model used previously in a decision analytic application. They find both algorithms can perform the required task, but often imperfectly. The study proposes statistical tests to help evaluate the algorithms' scenarios and suggests simple modifications to the algorithms and their implementing software that might improve their ability to support decision analysis with this scenario discovery task.
The research described in this report was prepared for the National Science Foundation and conducted by RAND Infrastructure, Safety, and Environment.
This report is part of the RAND Corporation Working paper series. RAND working papers are intended to share researchers' latest findings and to solicit informal peer review. They have been approved for circulation by RAND but may not have been formally edited or peer reviewed.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.