Applying a Natural Language Processing Tool to Electronic Health Records to Assess Performance on Colonoscopy Quality Measures

Published in: Gastrointestinal Endoscopy, v. 75, no. 6, June 2012, p. 1233-1239e14

Posted on on January 01, 2012

by Ateev Mehrotra, Evan S. Dellon, Robert E. Schoen, Melissa Saul, Faraz Bishehsari, Carrie M. Farmer, Henk Harkema

Read More

Access further information on this document at Gastrointestinal Endoscopy

This article was published outside of RAND. The full text of the article can be found at the link above.

Research Questions

  1. Can natural language processing (NLP) be used to analyze the quality of colonoscopies?
  2. How good is the quality of colonoscopies being performed?

BACKGROUND: Gastroenterology specialty societies have advocated that providers routinely assess their performance on colonoscopy quality measures. Such routine measurement has been hampered by the costs and time required to manually review colonoscopy and pathology reports. Natural language processing (NLP) is a field of computer science in which programs are trained to extract relevant information from text reports in an automated fashion. OBJECTIVE: To demonstrate the efficiency and potential of NLP-based colonoscopy quality measurement. DESIGN: In a cross-sectional study design, we used a previously validated NLP program to analyze colonoscopy reports and associated pathology notes. The resulting data were used to generate provider performance on colonoscopy quality measures. SETTING: Nine hospitals in the University of Pittsburgh Medical Center health care system. PATIENTS: Study sample consisted of the 24,157 colonoscopy reports and associated pathology reports from 2008 to 2009. MAIN OUTCOME MEASUREMENTS: Provider performance on 7 quality measures. RESULTS: Performance on the colonoscopy quality measures was generally poor, and there was a wide range of performance. For example, across hospitals, the adequacy of preparation was noted overall in only 45.7% of procedures (range 14.6%-86.1% across 9 hospitals), cecal landmarks were documented in 62.7% of procedures (range 11.6%-90.0%), and the adenoma detection rate was 25.2% (range 14.9%-33.9%). LIMITATIONS: Our quality assessment was limited to a single health care system in western Pennsylvania. CONCLUSIONS: Our study illustrates how NLP can mine free-text data in electronic records to measure and report on the quality of care. Even within a single academic hospital system, there is considerable variation in the performance on colonoscopy quality measures, demonstrating the need for better methods to regularly and efficiently assess quality.

Key Findings

NLP is effective.

  • Natural language processing programs can “read” dictated reports and measure colonoscopy quality in an inexpensive, automated, and efficient manner.

Generally poor performance found.

  • The quality of colonoscopies measured by the NLP tool varied substantially across the nine hospitals studied, and even more widely across providers.


  • The quality variation observed even within a single academic hospital system reinforces the need for routine quality measurement of colonoscopies.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

Our mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior. To help ensure our research and analysis are rigorous, objective, and nonpartisan, we subject our research publications to a robust and exacting quality-assurance process; avoid both the appearance and reality of financial and other conflicts of interest through staff training, project screening, and a policy of mandatory disclosure; and pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations, disclosure of the source of funding of published research, and policies to ensure intellectual independence. For more information, visit

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.