Subjective Probability Distribution Elicitation in Cost Risk Analysis

A cost estimate for a project such as the acquisition of a new aircraft or satellite system carries with it an inherent probability that the actual cost will exceed the estimate — that changes in requirements, technology, the economic environment, and a multitude of other factors that may occur over the life of the project will change the final cost. One major approach to cost risk analysis — the evaluating and quantifying of the uncertainty of a cost estimate — has been probabilistic: expressing the uncertainty in a cost estimate as a probability distribution over a range of potential costs. Cost analysts in industry and government and researchers in statistics and management have often proposed that, to get probability distributions for platforms using new and untried technologies, expert judgment should be tapped and subjective probability distributions elicited from the experts to represent cost uncertainty.

This technical report reviews procedures for eliciting subjective probability distributions in cost risk analysis, both in the cost risk field and in other disciplines in which elicitation has been a topic of research — primarily, statistics and psychology. Because of a lack of empirical work in elicitation, especially in cost risk, the author also interviewed a number of senior people in the cost risk community, who gave insight into the practices of the field.

This report should be of interest to cost analysis professionals who wish to quantify uncertainty when using expert opinions in cost risk analysis.