Onward Through the Fog

Policy analysis has always involved great uncertainty. Tools have been available for handling some of that uncertainty, but policy analysis work in many fields has fallen into stereotyped problem formulations and analytical approaches. In particular, treatments of uncertainty are typically incomplete and often conceptually wrong. This report argues that these shortcomings produce pervasive systematic biases in analysis. It describes and discusses the common mode of policy analysis and identifies its two main shortcomings — omission of crucial sources of uncertainty and neglect of systems' ability to respond to the unexpected. It categorizes some varieties of uncertainty relevant to policy analysis and presents examples of ways they are commonly represented. Finally, it discusses designing and evaluating systems, and presents a collection of generic strategies for uncertain situations.