In 2020, the Immediate Office of the Secretary of the Department of Health and Human Services sought to identify measures that could improve the ability of the United States and other countries to learn from international comparisons of health system performance. RAND Corporation researchers worked with 15 experts in quality measurement, clinical care, and health economics to generate and prioritize potential measure constructs.

Measures and Methodology for International Comparisons of Health Care System Performance

Final Report

Laurie T. Martin, Carl T. Berdahl, Rachel M. Burns, Emily Hoch, Evan D. Peet, Peter S. Hussey

ResearchPublished Feb 24, 2021

In 2020, the Immediate Office of the Secretary of the Department of Health and Human Services (HHS) sought to identify measures that could improve the ability of the United States and other countries to learn from international comparisons of health system performance. To inform the identification of measures for international comparison that could eventually be proposed to the Organisation for Economic Co-operation and Development (OECD), RAND Corporation researchers worked with a diverse group of 15 experts in quality measurement, clinical care, and health economics to generate and prioritize potential measure constructs that align with HHS priorities and are particularly promising for international comparisons. Eight measure constructs were identified as having the most promise for international comparison, but they will require additional development work to establish their operational definitions and specifications to ensure that any measure developed is valid and feasible for international comparisons of health system performance. This report is designed to lay a strong foundation for these future refinements by noting the degree of consensus among experts about the importance, scientific acceptability, perceived feasibility, and usability of measure constructs; summarizing the strengths and limitations of the measure constructs; and providing additional context that can be useful for informing the selection of measure constructs that might ultimately be developed into measures and proposed to OECD for consideration.

Key Takeaways

  • Of the 25 measure constructs included in the expert rating process, eight were rated as having the most promise for international comparisons: treatment and control of hypertension; access to and coverage for telehealth; quality-adjusted life expectancy; insurance coverage for mental health, behavioral health, and substance abuse services; receipt of preference-concordant end-of-life care; care continuity or consistent provider; access to mental health providers; and data transfer and interoperability.
  • Ten measure constructs were determined by experts to hold promise but will require additional refinements prior to moving toward the development of an operational definition: self-reported pain, access to primary palliative care, prices for brand-name and generic drugs, diffusion of and access to new prescription drugs, avoidable emergency department use, the percentage of patients with an opioid use disorder who were referred to or prescribed medication-assisted treatment, estimates of administrative complexity and cost, disadoption of ineffective medical services, healthy days at home, and availability of emergency medical services to prevent opioid death.
  • Seven measure constructs were not discussed because of their lower initial ratings: clinician workforce who can prescribe medication-assisted treatment or naloxone, access to opioid treatment centers, the percentage of patients with a follow-up visit within four weeks of starting an opioid for chronic pain, time to regulatory approval for new prescription drugs, travel time to provider office, health care spending in the last year of life, and spending on mental health (percentage of total health spending).

Recommendations

  • Pursue measure specification for promising measure constructs. In many cases, experts pointed to existing measures that could be leveraged.
  • Develop guidelines for assessing health care system performance. Some experts felt that if a measure construct could be influenced by factors outside of the health system, that construct was less useful for international comparisons of health care system performance given differences in populations, cultural preferences, and social conditions across countries. Other experts noted, however, that there could be ways to account for these differences in analysis to isolate the effect of health system performance on the outcome of interest.
  • Disentangle and prioritize related constructs of access, coverage, utilization, and outcomes. Several measure constructs focused on issues of access or coverage, but experts pointed out that coverage for or access to services does not necessarily equate to appropriate utilization of those services or result in improvements in health outcomes.
  • Invest in developing novel measure constructs. Experts provided a number of ideas that were deemed not specific enough to advance to the rating stage. These could be good candidates for measurement development work by HHS or other federal agencies.
  • Offer refinements to existing OECD measures. In Phase 1, experts nominated measure concepts for consideration. Many of these were refinements to or expansions of existing OECD measures, with suggestions for additional specification or recommendations to further parse out the data by key subgroups.

Topics

Document Details

Citation

Chicago Manual of Style

Martin, Laurie T., Carl T. Berdahl, Rachel M. Burns, Emily Hoch, Evan D. Peet, and Peter S. Hussey, Measures and Methodology for International Comparisons of Health Care System Performance: Final Report. Santa Monica, CA: RAND Corporation, 2021. https://www.rand.org/pubs/research_reports/RRA171-1.html.
BibTeX RIS

This research was funded by the Office of the Assistant Secretary for Planning and Evaluation and conducted by the Payment, Cost, and Coverage Program within RAND Health Care.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.