Assessment of Variation in Electronic Health Record Capabilities and Reported Clinical Quality Performance in Ambulatory Care Clinics, 2014–2017

Published in: JAMA Network Open, Volume 4, Issue 4 (2021). doi: 10.1001/jamanetworkopen.2021.7476

Posted on on May 04, 2021

by Paul G. Shekelle, Joseph D. Pane, Denis Agniel, Yunfeng Shi, Juliet Rumball-Smith, Ann C. Haas, Shira H. Fischer, Robert S. Rudin, Mark E. Totten, Julie Lai, et al.

Read More

Access further information on this document at JAMA Network Open

This article was published outside of RAND. The full text of the article can be found at the link above.


Electronic health records (EHRs) are widely promoted to improve the quality of health care, but information about the association of multifunctional EHRs with broad measures of quality in ambulatory settings is scarce.


To assess the association between EHRs with different degrees of capabilities and publicly reported ambulatory quality measures in at least 3 clinical domains of care.

Design, Setting, and Participants

This cross-sectional and longitudinal study was conducted using survey responses from 1141 ambulatory clinics in Minnesota, Washington, and Wisconsin affiliated with a health system that responded to the Healthcare Information and Management Systems Society Annual Survey and reported performance measures in 2014 to 2017. Statistical analysis was performed from July 10, 2019, through February 26, 2021.

Main Outcomes and Measures

A composite measure of EHR capability that considered 50 EHR capabilities in 7 functional domains, grouped into the following ordered categories: no functional EHR, EHR underuser, EHR, neither underuser or superuser, EHR superuser; as well as a standardized composite of ambulatory clinical performance measures that included 3 to 25 individual measures (median, 13 individual measures).


In 2014, 381 of 746 clinics (51%) were EHR superusers; this proportion increased in each subsequent year (457 of 846 clinics [54%] in 2015, 510 of 881 clinics [58%] in 2016, and 566 of 932 clinics [61%] in 2017). In each cross-sectional analysis year, EHR superusers had better clinical quality performance than other clinics (adjusted difference in score: 0.39 [95% CI, 0.12–0.65] in 2014; 0.29 [95% CI, –0.01 to 0.59] in 2015; 0.26 [95% CI, –0.05 to 0.56] in 2016; and 0.20 [95% CI, –0.04 to 0.45] in 2017). This difference in scores translates into an approximately 9% difference in a clinic's rank order in clinical quality. In longitudinal analyses, clinics that progressed to EHR superuser status had only slightly better gains in clinical quality between 2014 and 2017 compared with the gains in clinical quality of clinics that were static in terms of their EHR status (0.10 [95% CI, –0.13 to 0.32]). In an exploratory analysis, different types of EHR capability progressions had different degrees of associated improvements in ambulatory clinical quality (eg, progression from no functional EHR to a status short of superuser, 0.06 [95% CI, –0.40 to 0.52]; progression from EHR underuser to EHR superuser, 0.18 [95% CI, –0.14 to 0.50]).

Conclusions and Relevance

Between 2014 and 2017, ambulatory clinics in Minnesota, Washington, and Wisconsin with EHRs having greater capabilities had better composite measures of clinical quality than other clinics, but clinics that gained EHR capabilities during this time had smaller increases in clinical quality that were not statistically significant.

Research conducted by

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

Our mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior. To help ensure our research and analysis are rigorous, objective, and nonpartisan, we subject our research publications to a robust and exacting quality-assurance process; avoid both the appearance and reality of financial and other conflicts of interest through staff training, project screening, and a policy of mandatory disclosure; and pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations, disclosure of the source of funding of published research, and policies to ensure intellectual independence. For more information, visit

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.