Final Report: Evaluation of Tools and Metrics to Support Employer Selection of Health Plans

by Soeren Mattke, Kristin R. Van Busum, Grant R. Martsolf

This Article

RAND Health Quarterly, 2014; 4(2):5


The Patient Protection and Affordable Care Act (ACA) places strong emphasis on quality of care as a means to improve outcomes for Americans and promote the financial sustainability of our health care system. Included in the ACA are new disclosure requirements that require health plans to provide a summary of benefits and coverage that accurately describes the benefits under the plan or coverage. These requirements are intended to support employers’ procurement of high-value health coverage for their employees. This study attempts to help employers understand the structural differences between health plans and the performance dimensions along which plans can differ, as well as to educate employers about available tools that can be used to evaluate plan options. The study also discusses the extent to which these and other tools or resources are used by employers to inform choices between health plans.

For more information, see RAND RR-327-DOL at

Full Text

Background: The Role of Employers Under the National Quality Strategy

The Patient Protection and Affordable Care Act (Affordable Care Act) places strong emphasis on quality of care as a means to improve outcomes for Americans and promote the financial sustainability of our health care system. Notably, Section 3011 of the Affordable Care Act mandates the Secretary of Health and Human Services to establish the National Quality Strategy with the aim to increase access to high-quality, affordable health care for all Americans. The National Quality Strategy is centered on the Institute of Medicine's (IOM's) so-called “Triple Aim”: better care, better health, and lower cost.

Executing such a strategy requires participation from a wide range of stakeholders, such as providers, federal agencies, health plans,* and employers. More than half of the U.S. population receives employment-based health plan coverage, and the Affordable Care Act will expand employment-based coverage. Employers are in a unique position to communicate health-related information to employees during the workday and through company communications and can therefore exert positive influence on health and health care related decisions. Employers stand to benefit from better quality of care through improved productivity and less illness-related work loss. Further, employers and employee organizations offering health coverage to employees have a fiduciary duty under the Employee Retirement Income Security Act of 1974 (ERISA) to manage plans solely in the interest of participants and beneficiaries.

The Affordable Care Act supports employers in procuring high-value coverage options by codifying new disclosure requirements that require health plans to provide a summary of benefits and coverage (SBC) that accurately describes the benefits under the plan or coverage. In addition, plan or insurers must provide a uniform glossary, which provides definitions of common terms used in health coverage. In the SBC, group health plans and health insurance issuers are required to provide disclosures to illustrate potential financial exposure of plan participants and beneficiaries based on real-world examples.**

These new disclosure requirements are critical to achieve better transparency on the coverage that different health plans provide and thus serve as a welcome new tool for employers, employee organizations (e.g., multi-employer plans), participants, and beneficiaries to make informed choices. In the long run, the expectation is that the information will foster an environment, in which health plans compete on value, with the potential to reduce overall health spending while improving quality.

Against this background, this report attempts to help employers understand the structural differences between health plans and the performance dimensions along which plans can differ, as well as to educate them about tools and resources that can be used to compare plan options. We used an environmental scan, which consisted of a review of scientific and trade literature and health insurance companies' websites; expert panels; and case studies to evaluate the current availability of quality measures and decision tools that can inform employers' choices of health plans and today's actual practice of plan selection.

Understanding the Anatomy of a Health Plan

As health plans can differ along numerous characteristics, we started by creating a simplified framework that allows employers to understand these differences and the results that health plans achieve (Figure 1). In short, the framework reflects the fact that the structural characteristics of a health plan set the context under which health care providers deliver care (or which processes they use) and how those care processes, combined with health and health care–related decisions and choices of plan members, result in outcomes. In this study, we emphasize a review of structural characteristics as those are commonly marketed as product differentiators to employers.

Conceptual Framework to Categorize Differences Between Health Plans

Figure 1. Conceptual Framework to Categorize Differences Between Health Plans

For the purposes of this study, we distinguish structural characteristics that influence provider decisions and actions from those that influence decisions and actions of plan members:

  • Provider-facing characteristics
  • Network management, i.e., the size, location, and scope of its contracted provider network
  • Payment arrangements, i.e., on what basis providers are paid and which factors influence payments.
  • Member-facing characteristics
  • Customer service, which includes administrative functions such as claim processing, providing information on coverage and benefits, and handling complaints
  • Cost-sharing policies, which include co-payments and deductibles as well as policies on how those co-payments and deductibles can vary based on where and how members obtain care
  • Care management, which reflects the support services that a plan offers to help members navigate the health care system and improve self-management.

Care processes reflect the better care component of the Triple Aim and capture the decisions and actions of providers. We based our categorization scheme for care processes on work by the Agency for Healthcare Research and Quality (AHRQ) that describes high-value care processes as “striking the right balance of services by eliminating misuse (for example, providing medications that may have dangerous interactions), avoiding overuse (for example, performing tests that a patient does not need), and avoiding underuse (for example, not screening a person for high blood pressure)” (Agency for Healthcare Research and Quality, 2008). The categorization scheme is as follows:

  • Safe practices (e.g., use of electronic prescribing)
  • Adherence to evidence-based guidelines and current standards of care (e.g., percentage of plan participants who receive mammograms)
  • Avoidance of overuse (e.g., avoidance of unnecessary and potentially harmful medical services (Institute of Medicine Committee on Quality of Health Care in America, 2001)

We based our categorization scheme for care outcome measures on the IOM report entitled Crossing the Quality Chasm, and use three domains for outcomes measurement:

  • Medical outcomes, reflecting safety, timeliness, and effectiveness, can be captured at two levels
  • Intermediate (or proxy) outcomes, such as control of hypertension
  • Health outcomes, which can be positive, such as functional status, and negative, such as surgical complications
  • Patient experience, a measure used to capture a health plan's patient-centeredness
  • Efficiency measures, which reflect resource use per relevant outcome (or value), as opposed to cost measures that do not consider medical or patient-centered outcomes.

Provider-Facing Structural Characteristics of Health Plans

Network management captures how well a plan matches providers to the needs of plan participants and their dependents, with respect to the extent of the network (i.e., number and specialty mix of physicians and hospitals) and the capacity of the network to accept new patients. The quality of network management is commonly measured by metrics for network adequacy. The Affordable Care Act has brought significant attention to the issue of network adequacy, and requires the Secretary of Health and Human Services to establish certification criteria for qualified health plans participating in a state's Health Insurance Marketplace (National Association of Insurance Commissioners, 2012).*** In response to the requirements of the Affordable Care Act, the Secretary issued a final rule stating that, in order to qualify for the Health Insurance Marketplace, health plans must

  • include essential community providers
  • maintain a network that is sufficient in number and types of providers, including providers that specialize in mental health and substance-abuse services, to assure that all services will be accessible without unreasonable delay
  • be consistent with the network adequacy provisions of section 2702(c) of the Public Health Services Act (PHSA).

Although the law and resulting rules have brought significant attention to measuring and reporting of network adequacy, standardized and widely accepted measures for network adequacy are still lacking and a broad range of operational definitions is used by different entities.

Payment arrangements are the ways in which health plans compensate providers for services delivered. As different arrangements imply different incentives for providers, it is important for health plan purchasers to understand various payment arrangements and their implications for provider behavior. Since the historically dominant fee-for-service arrangement is increasingly considered suboptimal for patients with chronic conditions (who require care coordination and ongoing disease management rather than episodic encounters), public and private payers have started to explore alternative payment models to better align care delivery with patient needs. Figure 2 displays commonly used payment arrangements, in order of increasing financial risk to providers.****

Spectrum of Provider Payment Arrangements

Figure 2. Spectrum of Provider Payment Arrangements

* This includes the scenario in which provider and payer are completely integrated (e.g., a Health Maintenance Organization) and providers are salaried. Under this circumstance, there is no distinction between provider and payer.

** Pay for performance elements can be used under any payment arrangement.

Given the increasing variety of provider payment models, it is important for employers, employee organizations (e.g., union plans), participants, and beneficiaries to have a general understanding of how providers under a given health plan option are paid. However, because there are many possible payment arrangements, employers are unlikely to have a deep understanding of each payment arrangement, and are therefore less likely to choose the “best” option. Furthermore, it is also important to keep in mind that the payment models are not unambiguously defined—the same term can have different meanings in different contexts. For example, the Centers for Medicare and Medicaid Services (CMS) has a very detailed definition and regulations for Medicare Accountable Care Organizations (ACOs), the National Committee for Quality Assurance (NCQA) has put forth its own accreditation standards for ACOs, and individual health plans are using their own definitions. Employers should review each plan's actual details to gain a general understanding of how providers are paid under a given arrangement and how that payment arrangement may influence provider decisionmaking. For example, shifting financial risk to providers may reduce the cost of care, but can also create concerns about underuse, as providers do not benefit financially from additional services.

Member-Facing Structural Characteristics of Health Plans

Customer service is the part of a health plan that is most directly member-facing, and is therefore of critical importance to overall patient experience. It includes communication with members about benefits and network, the handling of claims, the appeals process for denied claims or denied authorization of services, and other services.

Cost-sharing provisions attempt to create incentives for plan participants and beneficiaries toward desirable health and health care–related decisions. Such incentives can be part of the benefit design and differential co-payments, based on consumer decisions on how and where to seek care, can be used to steer enrollees to care options that are seen as offering improved quality and/or efficiency. Two commonly seen types of such benefit designs are Consumer Directed Health Plans and tiered benefit designs. Incentives can also take the form of direct rewards or penalties that are linked to participation in wellness programs, which can include participatory programs or health-contingent programs focused on incentivizing individuals for achieving specific health targets (e.g., reduced tobacco use, weight loss).

Care management services are increasingly offered by health plans and include support for members with chronic care needs, assistance in patient recovery from acute illness, and programs to encourage healthier lifestyles. The two most common services for patients suffering from chronic conditions are disease management, which targets patients with defined chronic diseases such as diabetes, and case management, which supports patients with high costs of care, irrespective of the underlying conditions. Patients who experience a severe acute illness can get support through nurse advice lines, acute care case management, cancer treatment management, and discharge planning. The most commonly offered products and services to promote health and prevent disease include health risk assessments (HRAs), fitness club membership discounts, biometric screenings, and smoking cessation programs.

Current State of Process Quality Measurement

Process measures capture whether providers are following safe practices, deliver all guideline-recommended care and avoid services that have no proven clinical benefit. According to our technical expert panel, early efforts to measure quality were driven by providers largely for the purpose of quality-improvement projects, as opposed to payment or driving patients' selection of providers. This has led to the development of a large number of process measures that reflect how well care is aligned with evidence and standards. Using such granular and highly technical measures is difficult for employers, but the emergence of rigorous standards for measure evaluation, particularly the National Quality Forum (NQF) process, implies that measures in national use can be assumed to be scientifically sound and relevant. The endorsement process has also led to convergence, as organizations prioritize endorsed measures rather than creating their own.

Our review suggests that many process measures have been developed and are being developed. While several domains of care remain insufficiently represented, we identified the lack of measures for overuse of care as the most important gap in process measurement. With increasing evidence that selected medical services are overused (i.e., services that are unlikely to have clinical benefit or can be potentially harmful), there is a growing interest in measures that would capture such overuse and help efforts to address it. But efforts to develop additional overuse measures are often met with resistance, as providers view them as interfering with independent clinical decisionmaking. To establish acceptable measures for overuse, medical specialists, professional societies, and the research community need to collaborate to incorporate explicit standards for appropriate and inappropriate use into guidelines, especially of high-cost and high-volume procedures. An example for such an effort is the Choosing Wisely campaign.

Current State of Outcomes Measurement

Of the three areas of relevant outcomes (medical, patient experience, and efficiency), only the area of patient experience has a widely accepted measure set, in the form of the AHRQ Consumer Assessment of Healthcare Providers and Systems (CAHPS) instruments. Some measures for intermediate and health outcomes have been developed, but substantial gaps remain. Most notably, little progress has been made to date in the measurement of efficiency of care. Available measures mostly consider cost or utilization, but do not assess resource per relevant outcome, and the state of the measurement science is too underdeveloped to suggest that scientifically sound efficiency measures will become available soon. Hence, providing side-by-side cost and quality information to employers is currently seen as the best alternative to reporting efficiency of care.

State of Practice in Employer Decisionmaking About Health Plans

While the measurement science for most domains of quality has evolved substantially, our findings suggest that employers do not factor quality measures into their decisions about health plans. The main reason appears to be that employers find it difficult to interpret the complex and detailed information that is embedded in individual quality measures. Several tools have emerged that integrate complex information on quality, and further development is ongoing in this area. Examples are standardized information requests with health plan scoring algorithms, accreditation by external organizations, and ratings systems. While such tools can theoretically help employers to make informed decisions on health plans, we did not find published evidence on actual use of these tools, and our interviews suggest that employers have not yet adopted them widely. Rather, they tend to rely on the advice of benefits consultants and, to a lesser degree, input obtained through employer coalitions.

We also learned that employers primarily consider their cost of coverage when choosing a health plan. Second only to costs, employers consider network adequacy when choosing health plans. General reputation arose as another important criterion when choosing a health plan, as it serves as heuristic for quality and matters for employee satisfaction, even though lower cost or higher quality options may be available from a less well-known health plan. The concern for wide network access and overall reputation holds important implications for how employers think about and consider the issues of quality and quality improvement. Because all health plans in employers' choice sets have large networks that include most relevant hospitals and physicians, employers perceive that there is no meaningful variation in quality across plans. Therefore, employer representatives are reluctant to consider the “quality” of a health plan as a decision criterion. Instead, employers focus on using consumer engagement strategies to steer their employees to the highest-quality providers within those broad networks.


Our findings suggest that our understanding of what differentiates health plans structurally and how we measure their performance has improved, and that tools and resources to help employers use such information are emerging. In theory, this evolution put employers into a position to select health plans based on quality.

In practice, however, employers base their decisions mostly on cost considerations, the reputation of a plan, and their employees' preference to have access to a broad provider network. While a small set of sophisticated employers may conduct a complex evaluation process that is based on multiple criteria, the typical employer, simply speaking, balances its own desire to control cost of coverage with its employees' desire for provider choice, with limited explicit regard for quality of care. This simple decision rule limits the degree to which employer decisions on health plans can influence quality of care. If most employers in a market demand access to a broad provider network, health plans will try to offer coverage products that exclude only few providers. Thus, there will be considerable overlap between the provider networks with which competing plans contract. The overlap in turn implies that the employer's choice of a health plan becomes less consequential: Most care decisions, and thus quality of care, are determined by providers, and, if plans share most providers, quality and cost will converge and so will the value offered by different plans.

This trend leaves two possible pathways through which health plan design and employer choices can lead to better value for money. The first would be closer integration between health plans and providers to improve quality and efficiency of care. Plans would contribute their ability to analyze data at the population level, benchmark providers and track patients across different providers, and collaborate with providers to lift the average quality and efficiency in a given market. The second path would be to steer plan participants and beneficiaries selectively to high-performing providers, with the expectation that market forces would push the low performers out or make them improve.

Implications for Research Agenda

Performance measures can, in principle, be applied at different levels of aggregation, from the national level to the level of individual providers. Thus, the gaps in measurement science that we identified, most notably in measuring overuse and efficiency, are as relevant for provider choice as they are for plan comparisons. Yet three methodological challenges make it more difficult to apply measures at the provider level:

  • Sample size: In contrast to health insurers, who may have millions of members, individual providers only care for relatively small numbers of patients, particularly for any given condition (e.g. diabetes), which makes it difficult to calculate quality measures reliably
  • Attribution: Quality measures must be attributable to a provider. Many evidence-based processes of care are under the control of an individual provider and thus can be attributed to her or him, but particular outcomes may be influenced by several providers and need to be properly attributed
  • Risk adjustment: For outcome measures, rates will be affected by patient-level factors. These factors must be included in a multivariate model that generates risk-adjusted measure rates.

Given the complexity of those methodological challenges, it is not surprising that provider-level measurement is still in its infancy and needs to be developed further, as will our understanding on how to engage plan participants and beneficiaries in selection of providers through financial and non-financial means.

To summarize, our analysis points to four areas in which further research and development should be pursued:

  • Efficiency measurement at different levels of the health care system (providers, provider networks, hospitals, and health plans)
  • Methods to apply measures validly and reliably at the provider level
  • Tools to consolidate complex information on provider quality and cost to inform decisions by plan participants and beneficiaries
  • Financial and non-financial strategies to increase plan participants' and beneficiaries' engagement in choosing high-quality and low-cost providers.

Progress in these four areas will strengthen our ability to improve the quality of care, while promoting the sustainability of the health care system.


Agency for Healthcare Research and Quality, Reporting Measures for the CAHPS Health Plan Survey 4.0, 2008.

Institute of Medicine Committee on Quality of Health Care in America, Crossing the Quality Chasm: A New Health System for the 21st Century, Washington, D.C.: National Academies Press, 2001.

National Association of Insurance Commissioners, “Plan Management Function: Network Adequacy White Paper,” 2012. As of September 12, 2013:

Patient Protection and Affordable Care Act; Data Collection to Support Standards Related to Essential Health Benefits; Recognition of Entities for the Accreditation of Qualified Health Plans, Final Rule, 2013a. As of September 12, 2013:

Patient Protection and Affordable Care Act; Establishment of Exchanges and Qualified Health Plans; Exchange Standards for Employers, Washington, D.C., Final Rule and Interim Final Rule, 2013b. As of September 12, 2013:

Public Health Service (PHS) Act, Section 2707(b), Washington, D.C.: Final Rule, 2013. As of September 13, 2013:


* We recognize that employers may self-insure or be fully insured. Some self-insured plans may use health insurance companies only to administer their plan as a third-party administrator but not to insure them against financial risk. To improve readability, we are using the term “plan or coverage” throughout the report to reflect both insured and self-insured health plan coverage offered by employers.

** Affordable Care Act, section 2715.

*** Affordable Care Act, section 1311(c).

**** It should be noted that the actual degree of risk sharing will depend on the contractual details of the payment arrangements.

The research described in this article was sponsored by the U.S. Department of Labor, and was produced within RAND Health, a division of the RAND Corporation.

RAND Health Quarterly is produced by the RAND Corporation. ISSN 2162-8254.