Acceptability of an Online Modified Delphi Panel Approach for Developing Health Services Performance Measures

Results from 3 Panels on Arthritis Research

Published in: Journal of Evaluation in Clinical Practice, 2016

Posted on RAND.org on September 29, 2016

by Dmitry Khodyakov, Sean Grant, Claire E. Barber, Deborah A. Marshall, John M. Esdaile, Diane Lacaille

Read More

Access further information on this document at Journal of Evaluation in Clinical Practice

This article was published outside of RAND. The full text of the article can be found at the link above.

RATIONALE, AIMS, AND OBJECTIVES: Online modified Delphi (OMD) panel approaches can be used to engage large and diverse groups of clinical experts and stakeholders in developing health services performance measures. Such approaches are increasing in popularity among health researchers. However, information about their acceptability to participating experts and stakeholders is lacking but important to determine before recommending widespread use of online approaches. Therefore, the objective of this paper is to explore acceptability of the OMD panel approach from the participants' perspective. METHOD: We use data from participants in three OMD panels designed to develop performance measures for use in arthritis research and quality improvement efforts. At the end of each online panel, we surveyed clinical experts and stakeholders who shared their experiences with the OMD process by answering 13 close-ended questions using 7-point Likert-type scales. A mean of 5 or higher on a given question was treated as an indication of acceptability. RESULTS: Ninety-eight clinical experts and stakeholders (92% participation rate) answered survey questions about the online process. They considered the OMD panel approach to be acceptable, particularly the ease of using the online system (mean = 5.3, standard deviation = 1.3) and the understanding gained from online discussions (mean = 5.2, standard deviation = 1.0). Participants also felt that participation in the Delphi study was interesting (mean = 5.6, standard deviation =1.1). CONCLUSION: These findings illustrate likely acceptability and a potential for a more widespread use of OMD panel approaches by stakeholders in developing health services performance measures.

This report is part of the RAND Corporation external publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.