RAND Methodological Guidance for Conducting and Critically Appraising Delphi Panels

The Delphi method is an iterative, anonymous, structured, group-based communication process and elicitation technique designed to help policymakers make decisions under conditions of uncertainty and incomplete information. It is based on the premise that asking a hand-picked group of anonymous experts the same questions several times and sharing the other experts' answers will help objectively develop group consensus, which is used as a form of evidence. Although it was originally developed by RAND researchers as a forecasting methodology in military research, Delphi underwent many modifications and is now used by different disciplines, most notably by medicine, as a gold-standard approach for expert elicitation and stakeholder engagement. Researchers often rely on Delphi to estimate the probability of an event happening within a certain period of time, to forecast when an event is likely to occur, and to identify and prioritize key policy issues that need to be addressed.

Although the Delphi method is used by different disciplines and for different purposes, there is still a lack of methodological guidance for how to conduct rigorous Delphi studies. This manual addresses this critical gap by providing practical advice for when to use Delphi, what type of Delphi to choose, and what to consider when designing, implementing, and reporting Delphi panel results. The manual also includes the Delphi Critical Appraisal Tool (DCAT), which researchers can use both retrospectively to appraise the quality of an already conducted Delphi study and prospectively to design and implement a Delphi study.