Natural language processing programs can "read" dictated reports and measure colonoscopy quality in an inexpensive, automated, and efficient manner.

Cover: Applying a Natural Language Processing Tool to Electronic Health Records to Assess Performance on Colonoscopy Quality Measures

Applying a Natural Language Processing Tool to Electronic Health Records to Assess Performance on Colonoscopy Quality Measures

Published in: Gastrointestinal Endoscopy, v. 75, no. 6, June 2012, p. 1233-1239e14

Posted on RAND.org on January 01, 2012

by Ateev Mehrotra, Evan S. Dellon, Robert E. Schoen, Melissa Saul, Faraz Bishehsari, Carrie M. Farmer, Henk Harkema

Research Questions

  1. Can natural language processing (NLP) be used to analyze the quality of colonoscopies?
  2. How good is the quality of colonoscopies being performed?

BACKGROUND: Gastroenterology specialty societies have advocated that providers routinely assess their performance on colonoscopy quality measures. Such routine measurement has been hampered by the costs and time required to manually review colonoscopy and pathology reports. Natural language processing (NLP) is a field of computer science in which programs are trained to extract relevant information from text reports in an automated fashion. OBJECTIVE: To demonstrate the efficiency and potential of NLP-based colonoscopy quality measurement. DESIGN: In a cross-sectional study design, we used a previously validated NLP program to analyze colonoscopy reports and associated pathology notes. The resulting data were used to generate provider performance on colonoscopy quality measures. SETTING: Nine hospitals in the University of Pittsburgh Medical Center health care system. PATIENTS: Study sample consisted of the 24,157 colonoscopy reports and associated pathology reports from 2008 to 2009. MAIN OUTCOME MEASUREMENTS: Provider performance on 7 quality measures. RESULTS: Performance on the colonoscopy quality measures was generally poor, and there was a wide range of performance. For example, across hospitals, the adequacy of preparation was noted overall in only 45.7% of procedures (range 14.6%-86.1% across 9 hospitals), cecal landmarks were documented in 62.7% of procedures (range 11.6%-90.0%), and the adenoma detection rate was 25.2% (range 14.9%-33.9%). LIMITATIONS: Our quality assessment was limited to a single health care system in western Pennsylvania. CONCLUSIONS: Our study illustrates how NLP can mine free-text data in electronic records to measure and report on the quality of care. Even within a single academic hospital system, there is considerable variation in the performance on colonoscopy quality measures, demonstrating the need for better methods to regularly and efficiently assess quality.

Key Findings

NLP is effective.

  • Natural language processing programs can “read” dictated reports and measure colonoscopy quality in an inexpensive, automated, and efficient manner.

Generally poor performance found.

  • The quality of colonoscopies measured by the NLP tool varied substantially across the nine hospitals studied, and even more widely across providers.

Recommendation

  • The quality variation observed even within a single academic hospital system reinforces the need for routine quality measurement of colonoscopies.
Access further information on this document at Gastrointestinal Endoscopy

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.