Full Document

FormatFile SizeNotes
PDF file 4.9 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Questions

  1. What measures of health system performance can be used for international comparisons?
  2. How much consensus was there among experts on the importance, scientific acceptability, perceived feasibility, and usability of measure constructs?
  3. Which of these measures could eventually be proposed to OECD for consideration?

In 2020, the Immediate Office of the Secretary of the Department of Health and Human Services (HHS) sought to identify measures that could improve the ability of the United States and other countries to learn from international comparisons of health system performance. To inform the identification of measures for international comparison that could eventually be proposed to the Organisation for Economic Co-operation and Development (OECD), RAND Corporation researchers worked with a diverse group of 15 experts in quality measurement, clinical care, and health economics to generate and prioritize potential measure constructs that align with HHS priorities and are particularly promising for international comparisons. Eight measure constructs were identified as having the most promise for international comparison, but they will require additional development work to establish their operational definitions and specifications to ensure that any measure developed is valid and feasible for international comparisons of health system performance. This report is designed to lay a strong foundation for these future refinements by noting the degree of consensus among experts about the importance, scientific acceptability, perceived feasibility, and usability of measure constructs; summarizing the strengths and limitations of the measure constructs; and providing additional context that can be useful for informing the selection of measure constructs that might ultimately be developed into measures and proposed to OECD for consideration.

Key Findings

  • Of the 25 measure constructs included in the expert rating process, eight were rated as having the most promise for international comparisons: treatment and control of hypertension; access to and coverage for telehealth; quality-adjusted life expectancy; insurance coverage for mental health, behavioral health, and substance abuse services; receipt of preference-concordant end-of-life care; care continuity or consistent provider; access to mental health providers; and data transfer and interoperability.
  • Ten measure constructs were determined by experts to hold promise but will require additional refinements prior to moving toward the development of an operational definition: self-reported pain, access to primary palliative care, prices for brand-name and generic drugs, diffusion of and access to new prescription drugs, avoidable emergency department use, the percentage of patients with an opioid use disorder who were referred to or prescribed medication-assisted treatment, estimates of administrative complexity and cost, disadoption of ineffective medical services, healthy days at home, and availability of emergency medical services to prevent opioid death.
  • Seven measure constructs were not discussed because of their lower initial ratings: clinician workforce who can prescribe medication-assisted treatment or naloxone, access to opioid treatment centers, the percentage of patients with a follow-up visit within four weeks of starting an opioid for chronic pain, time to regulatory approval for new prescription drugs, travel time to provider office, health care spending in the last year of life, and spending on mental health (percentage of total health spending).

Recommendations

  • Pursue measure specification for promising measure constructs. In many cases, experts pointed to existing measures that could be leveraged.
  • Develop guidelines for assessing health care system performance. Some experts felt that if a measure construct could be influenced by factors outside of the health system, that construct was less useful for international comparisons of health care system performance given differences in populations, cultural preferences, and social conditions across countries. Other experts noted, however, that there could be ways to account for these differences in analysis to isolate the effect of health system performance on the outcome of interest.
  • Disentangle and prioritize related constructs of access, coverage, utilization, and outcomes. Several measure constructs focused on issues of access or coverage, but experts pointed out that coverage for or access to services does not necessarily equate to appropriate utilization of those services or result in improvements in health outcomes.
  • Invest in developing novel measure constructs. Experts provided a number of ideas that were deemed not specific enough to advance to the rating stage. These could be good candidates for measurement development work by HHS or other federal agencies.
  • Offer refinements to existing OECD measures. In Phase 1, experts nominated measure concepts for consideration. Many of these were refinements to or expansions of existing OECD measures, with suggestions for additional specification or recommendations to further parse out the data by key subgroups.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Measure Constructs That Hold the Most Promise for International Comparisons

  • Chapter Three

    Measure Constructs That Received Lower or Inconsistent Ratings After Expert Discussion

  • Chapter Four

    Measure Constructs That Were Not Discussed by Experts

  • Chapter Five

    Recommended Modifications to Existing OECD Measures

  • Chapter Six

    Measure Constructs Requiring Development

  • Chapter Seven

    Recommendations and Conclusion

  • Appendix A

    Evidence Tables

Research conducted by

This research was funded by the Office of the Assistant Secretary for Planning and Evaluation and conducted by the Payment, Cost, and Coverage Program within RAND Health Care.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.