iStock
Articles featuring the ExpertLens system and methodology have been published in leading international high-impact peer-reviewed journals.
The October 2011 issue of the journal Technological Forecasting and Social Change featured "ExpertLens: A system for eliciting opinions from a large pool of non-collocated experts with diverse knowledge," an article that explains the ExpertLens system and methodology.
A RAND news release about the article provides a brief summary of ExpertLens features. The article was also featured on WashingtonPost.com and other blogs.
Our article “Collaborative learning framework for online stakeholder engagement,” published in the journal Health Expectations, describes how ExpertLens data could be modeled using a Bayesian approach. It also presents an innovative theoretical framework for analyzing and interpreting the online data collected from large and diverse groups.
A good example of how the analysis of discussion comments can augment the quantitative analyses is seen in "How can we recognize continuous quality improvement?" in the December 2013 issue of the International Journal for Quality in Health Care.
To learn about the feasibility of using ExpertLens to conduct online expert panels and experimental replicability of its findings, read "Conducting online expert panels: A feasibility and experimental replicability study" in the December 2011 issue of the journal BMC Medical Research Methodology. The article presents the results of an experiment where four online panels of different size were conducted using the identical research protocol. This study also provides valuable insights into how to perform a simple descriptive analysis of ExpertLens data.
ExpertLens data have been used in a number of methodological articles about the Delphi method. Two articles, published in the Journal of Clinical Epidemiology and Value in Health, focus on how and why participants in Delphi studies change their responses after seeing how their own responses compare to those of other participants and participating in an online discussion.
In this qualitative study of 27 low-value prescribing practices, a panel of experts was convened using an online modified-Delphi approach to develop a scientifically valid and clinically useful low-value prescribing metric.
This qualitative study provides consensus-based guidance for clinicians and highlights research and policy gaps that warrant urgent attention to meet the needs of people with cancer-related pain and opioid misuse/use disorder.
This study develops hypotheses about the impact of panel composition and topic on the propensity and meaningfulness of response changes in multistakeholder modified-Delphi panels.
This paper addresses challenges in clinical care, quality improvement, research, and public policy for prenatally opioid-exposed neonates by standardizing the clinical definition of opioid withdrawal.
We engaged experts using an online modified Delphi approach to develop consensus on the appropriateness of strategies for managing cancer pain in individuals with co-occurring advanced cancer and opioid use disorder.
The authors sought to identify the most promising strategies for improving the mental health guardianship process in Los Angeles County for adults with mental illness who are gravely disabled.
The goal of this paper is to describe a protocol for a systematic expert panel process to inform the development of a clinical definition of neonatal abstinence syndrome.
The authors describe the results of two modified-Delphi expert panels used to inform the work of a federal steering committee on the development of clinical definitions of neonatal abstinence syndrome and neonatal opioid withdrawal syndrome.
Using a modified Delphi approach, we created seven patient cases that capture important management challenges in individuals with serious illness prescribed opioid therapy.
Our objective was to explore the extent of consensus among expert stakeholders regarding what analgesic issues should be prioritized for comparative-effectiveness studies of beneficial and adverse effects of analgesic regimens in nursing home residents.
A Delphi consensus exercise to gather expert opinion on the adaptation, implementation and re-evaluation of complex population health interventions in new contexts.
This article estimates the seriousness of eleven maternal and child health outcomes that have been consistently associated with pregnancy weight gain by engaging patients and health professionals.
This article determines the seriousness of 11 maternal and child health outcomes associated with pregnancy weight gain, by engaging patients and maternal and child health professionals using an online modified Delphi panel process.
We developed heavy menstrual bleeding (HMB) care considerations using a modified-Delphi process; clinicians can use these care considerations to optimize the management of adolescents with HMB.
Clinical guidelines that meet patient-centeredness criteria are likely to be consistent with the preferences, needs, and values of families dealing with Duchenne muscular dystrophy.
This article's objective is to illustrate how to use an online modified-Delphi approach to engage patients and their representatives during different stages of clinical practice guidelines development.
Participants were satisfied with an online approach to participating in guideline development.
The RAND/PPMD Patient-Centeredness Method allows patients and their representatives to provide input in clinical practice guidelines.
A lack of standards and norms for defining and publicly registering analysis plans in advance threatens the credibility of using consensus-oriented Delphi processes to inform health research practice, and policy.
The Official Disability Guidelines (ODG) Medical Treatment Guidelines are an important resource for workers' compensation systems because they can be used to contain medical care costs while assuring appropriate care is provided.
The core elements in the Canadian Rheumatoid Arthritis Core Clinical Dataset were rated as important and feasible to collect by rheumatology care providers and people living with arthritis from across Canada.
Vignettes could be an effective way to train researchers in the responsible conduct of research in the participatory era.
Experts have identified new criteria for assessing the quality of prescribing in nursing homes.
Engaging diverse groups of stakeholders through online collaborative platforms is a promising and scalable approach to prioritizing patient-centered research topics.
An online Delphi-based process to select criteria for defining health care quality was rated as acceptable by three groups of participants, indicating it could be used to engage stakeholders in developing performance measures.
The transfer of new discoveries into both clinical practice and the wider community calls for reliance on interdisciplinary translational teams that include researchers with different areas of expertise.
This study aims to develop system-level performance measures for evaluating the care of patients with inflammatory arthritis (IA), including rheumatoid arthritis (RA), psoriatic arthritis, ankylosing spondylitis, and juvenile idiopathic arthritis.
The aim of this study was to develop key performance indicators (KPIs) to evaluate centralized intake systems for patients with osteoarthritis (OA) and rheumatoid arthritis (RA).
We use 'A National Conversation on Reducing the Burden of Suicide in the United States' as a case study of online stakeholder engagement and employ a Bayesian data modelling approach to develop a collaborative learning framework (CLF).
Patients with rheumatoid arthritis (RA) have a high risk of premature cardiovascular disease (CVD).
The National Action Alliance for Suicide Prevention Research Prioritization Task Force (RPTF) has created a prioritized national research agenda with the potential to rapidly and substantially reduce the suicide burden in the U.S. if fully funded and implemented.
Continuous quality improvement (CQI) methods are foundational approaches to improving healthcare delivery.
ExpertLens, a RAND-developed system for eliciting opinions from a large group of people, saves on the costs associated with traditional expert panels, while increasing accuracy.
It is feasible to conduct online expert panels intended to facilitate consensus finding among geographically distributed participants. The online approach may be practical for engaging large and diverse groups of stakeholders around a range of health services research topics and can help conduct multiple parallel panels to test for the reproducibility of panel conclusions.
