Technology-enabled collaborative learning and capacity-building models connect generalist providers, often located in remote areas, with specialist teams that help train these providers so that they can deliver care for conditions that generalists might not otherwise feel adequately prepared to handle. This report focuses on the quality of evidence for the effectiveness of such models and describes ways to improve measurement of effectiveness.

Read Online

Evaluation of Technology-Enabled Collaborative Learning and Capacity Building Models

Materials for a Report to Congress

by Shira H. Fischer, Adam J. Rose, Ryan K. McBain, Laura J. Faherty, Jessica L. Sousa, Monique Martineau

Full Document

Full Document

Does not include Appendix F.

FormatFile SizeNotes
PDF file 5.4 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Appendix F

FormatFile SizeNotes
PDF file 1.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Questions

  1. What programs use EELM to treat what diseases?
  2. How is the effectiveness of EELM measured?
  3. Are EELM effective?

Across the United States and internationally, multiple health care sites have embraced technology-enabled collaborative learning and capacity-building models. Such models use technology to connect generalist providers, often located in remote areas, with specialist teams that help train these providers to deliver care for patients with conditions that they might not feel adequately prepared to handle but are nevertheless within their scope of practice. The first implementation of this model, Project ECHO (Extension for Community Healthcare Outcomes), launched in 2003 in New Mexico. Project ECHO began with a focus on supporting the management of patients with hepatitis C virus (HCV) in rural regions of the state. This model has since been adapted to many different sites within the United States and other countries, and these models now address a wide range of medical conditions and other issues that providers face. This report documents what is known about ECHO and ECHO-like models (EELM). Generally speaking, the quality of evidence for the effectiveness of EELM could use improvement, but EELM mostly show positive effects in the small but growing body of research of EELM, which thus far measures more provider outcomes than patient outcomes.

Key Findings

Empirical evidence for the effects of ECHO and ECHO-like models (EELM) on patient and provider outcomes remains modest

  • Based on a literature review and a technical expert panel, the quality of evidence for the effectiveness of EELM is rated as "low" or "very low" based on generally accepted systems of measurement. However, it is important to note that many models of care delivery are supported only by low-quality evidence.
  • In the areas that researchers have measured, EELM mostly show positive effects.
  • The intention of EELM is to increase access to specialty care by educating generalist health providers, particularly those in locales with limited access such as rural and remote areas. However, there is a need for targeted funding to evaluate EELM effectively.

Recommendations

  • Implementers and evaluators can engage with policymakers, funders, and others to explore mechanisms for supporting rigorous evaluation. Such mechanisms would ideally address care delivery imperatives in the near term and enable rigorous evaluations that would expand the evidence base to support longer-term investments in EELM.
  • An expanded focus on rigorous reporting of program characteristics of EELM would help evaluators assess how the model is put into practice and what "ingredients" might lead to better outcomes and are worth replicating.
  • Building capacity to evaluate EELM is a third critical opportunity and is two-pronged. Building such capacity could help implementers design EELM to facilitate improved evaluations, and it could help researchers more effectively choose populations for study, outcomes, comparators, and study designs.
  • Implementers and evaluators can engage with policymakers, funders, and others to explore mutually beneficial mechanisms for supporting rigorous evaluation. Such mechanisms would ideally address care delivery imperatives in the near term and enable rigorous evaluations that would expand the evidence base to support longer-term investments in EELM.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    EELM in Context: History, Promise, and Challenges

  • Chapter Three

    Methods

  • Chapter Four

    Findings: Inventory

  • Chapter Five

    Findings: Evidence Review

  • Chapter Six

    Findings of Technical Expert Panel: Evaluation Options

  • Chapter Seven

    Example Evaluation Study Designs

  • Chapter Eight

    Implications and Conclusions

  • Appendix A

    Search Technique for Inventory

  • Appendix B

    Search Technique for Evidence Review

  • Appendix C

    Studies Used in Evidence Review

  • Appendix D

    Studies Considered as Examples of EELM Evaluations

  • Appendix E

    Case Studies: Technology-Based Health Care Collaborative Learning & Capacity- Building Models

  • Appendix F

    Inventory

Research conducted by

This research was funded by the Office of the Assistant Secretary for Planning and Evaluation (ASPE) and conducted by Payment, Cost, and Coverage program within RAND Health Care.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.