Is Physician Cost Profiling Ready for Prime Time?

Research Highlights

Key findings:

  • Between 17 and 61 percent of physicians would be assigned to a different cost category if an attribution rule other than the most common rule were used.
  • Most cost profile scores do not meet common reliability thresholds.
  • Reliability of cost profile scores is not clearly linked to the number of care episodes assigned to a physician.
  • If common cost profiling methods are used, about 22 percent of physicians would be assigned to the wrong cost category.

Passage of health care reform legislation by the 111th Congress represents major progress in reducing the number of the uninsured. Now the focus of policy discussion is likely to turn quickly from issues of coverage to the challenge of controlling health care costs.

Purchasers have been experimenting for some time with different approaches to reining in costs (consumer cost sharing, managed care, financial incentives for physicians). Most of these approaches have either met with resistance from consumers (e.g., managed care restricting choice) or have been ineffective (e.g., regulatory strategies).

The latest approach is to focus attention on the relative costs of individual physicians and to give consumers incentives to seek care from physicians with relatively lower spending patterns. It makes sense to focus on physicians because they drive much of health care spending through the decisions they make (e.g., which tests to order, what drugs to prescribe, which referrals to make, and what procedures to perform).

Purchasers have developed methods for creating overall assessments of practice patterns called cost profiles. A cost profile is a single number that places a physician on a relative scale of spending. Purchasers use cost profiles to identify physicians who are high cost or low cost (similar in concept to the number of dollar signs indicating the expense one might encounter at a particular restaurant).

Creating the profiles requires many technical decisions. Until recently, those decisions were largely a black box — different organizations made different choices. Doctors were largely not informed about how their cost profiles were created. Some doctors found that two health plans might give them two different cost profiles (e.g., one plan said they were high cost, while another found the opposite). Understanding the process of cost profiling is especially important now because under the new health care reform legislation, Medicare will be providing physician cost profiles (called relative resource reports) by 2012.

To open the black box of physician cost profiling, RAND analysts conducted a series of studies to examine the different technical decisions necessary for creating cost profiles and the utility of cost profiles for helping consumers select more “efficient” doctors. The RAND team's goal was to understand whether the decisions made in creating the profile affected whether a physician is labeled high cost or low cost.

Table 1
Pricing an Episode of Care for a Patient with Coronary Artery Disease
Service Number of services Average price per service ($) Cost ($)
Office visits with primary care physician 4 100 400
Treadmill stress test to see if coronary artery disease is stable 1 500 500
One-year supply of medication for coronary artery disease 365 1 365
Lipid profile 1 35 35
Cardiology consultation 1 175 175
Total observed costs of episode     1,475

Over the course of a year a patient with coronary artery disease visited her physician four times, had a treadmill test, obtained a year's supply of a necessary drug, and had a blood test to determine her cholesterol and triglyceride levels. She also had a consultation with a cardiologist.

The costs of the services in this example are fictional. In the analyses, the cost of each service was the average charge allowed by the relevant health plans — in the case of RAND's study, the four Massachusetts health plans.

 
Figure 1
Median Reliability of Physician Cost Profiles Varies Widely by Specialty
Median Reliability of Physician Cost Profiles Varies Widely by Specialty
 
Figure 2
Reliability Is Not Clearly Linked to the Number of Episodes Assigned to a Physician
Reliability Is Not Clearly Linked to the Number of Episodes Assigned to a Physician

SOURCE: Used with permission of the Massachusetts Medical Society

To form the empirical basis for this work, the researchers analyzed aggregated claims data on 1.1 million adults age 18–65 continuously enrolled in four Massachusetts commercial health plans in 2004 and 2005. The study sample included more than 13,700 physicians. The team used methods for constructing physician cost profiles and assigning physicians to cost categories that largely mirrored methods commonly used by health plans.

How Are Physician Cost Profiles Created?

Creating a physician cost profile involves multiple steps. These include deciding which care will be included in a cost profile (e.g., the inclusion or exclusion of preventive care), determining which costs are assigned to each unit of care, deciding which physician is responsible for care, identifying the peers to whom a physician will be compared, creating the algebra for constructing cost profiles, and deciding how to place a physician in a high-cost or low-cost category. This research highlight summarizes research on a few of these decisions.

The initial decision is whether the allocation of dollars spent on health care should be by patient — thus assigning all of the patient's costs to a single physician — or by episodes of care. Episodes of care are the clinically related services delivered to a patient with a specific condition over a certain period of time; currently, health plans commonly use commercial software to group a patient's claims into an episode of care (see Table 1 for an example).

Another key step is deciding how to assign accountability for costs. Patients often see many doctors. Which doctor should be accountable for the costs of care? One option is to assign costs to the physician who had the most visits, either from the patient or during an episode of care. Another approach would be to assign costs to the physician who accounted for largest portion of spending.

Health plans use the results of such decisions (referred to as attribution rules) to assign costs of care to a physician and categorize physicians as low, average, or high cost. But there are many permutations and combinations of the attribution rules. Does a given physician end up in the same cost category no matter which attribution rules are used? And once a cost profile is created, how reliable is it?

How Do Attribution Rules Affect Assignment of Cost Categories?

Most health plans assign physicians to one of two or three cost categories (e.g., low, average, or high cost). To gauge how the choice of attribution rules might affect the category to which a physician is assigned, RAND researchers created 12 attribution rules reflecting the current range of rules being used by insurers and calculated what fraction of physicians was assigned to a different cost category by each of 11 attribution rules, compared with the default rule. (The default rule reflected a rule currently in wide use among health plans. It uses episodes as the unit of analysis and assigns costs to the physician who generated the largest percentage of costs.)

RAND's assessment showed that the choice of attribution rules strongly affected a physician's cost category. Specifically, 17 to 61 percent of physicians would be assigned to a different cost category if a rule other than the default attribution rule were used.

How Reliable Are Physician Cost Profiles?

A related issue is the reliability of cost profiles. Once a cost profile has been created, how accurately does it reflect a physician's actual economic performance? RAND researchers used the Massachusetts data described above to answer this question.

For each physician in the study, RAND analysts constructed a summary cost profile score indicating what the costs were for the episodes of care assigned to a given physician, relative to the costs of other physicians in the same specialty who were treating patients with similar conditions and similar comorbidities. Then the analysts assessed the reliability of the profile — that is, the extent to which variation across cost profile scores reflected real variations in physicians' economic performance or simply error in how the cost profile score was constructed. Reliability is expressed on a scale from 0 to 100 percent, where 0 percent means that differences in cost profile scores result solely from measurement error, and 100 percent means that all of the differences in scores reflect actual variations in physician resource use.

RAND found that the median reliability of physician cost profiles varied widely, especially across specialties, ranging from 5 percent for vascular surgery to 97 percent for dermatology (see Figure 1). Overall, 59 percent of physicians had cost profile reliability scores below 70 percent, a commonly used reliability threshold.

Purchasers frequently require a minimum sample size before they profile individual physicians. For example, they may not profile physicians to whom fewer than 30 episodes of care have been assigned.

But the analysts found that minimum cutoffs are a relatively poor proxy for reliability. Figure 2 charts the relationship between sample size and reliability; each dot is a physician. The two circles highlight two doctors, each of whom was assigned more than 250 episodes. But one physician has a cost profile with a high reliability (almost 0.95); the other has a low reliability profile.

How Likely Is a Physician to Be Misclassified?

The RAND team used the estimates of reliability to measure the probability that a physician would be inaccurately categorized in a two-tiered classification system. In this system, physicians whose cost profiles were in the lower 25 percent of all profiles were designated as lower cost; all others were considered not lower cost.

The analysts found that many physicians were misclassified. In some cases, physicians who were actually lower cost were categorized as not lower cost; in other cases, the opposite was true (see Table 2). Overall, about 22 percent of physicians were assigned to the wrong cost category.

Table 2
Rates of Misclassification Vary Widely Across Specialities
Specialty Physicians Misclassified as Lower Cost (%) Physicians Misclassified as Not Lower Cost (%) Overall Misclassification Rate (%)
Internal medicine 50 22 25
Family practice 39 16 21
OB-GYN 36 10 17
Cardiology 40 13 20
Cardiovascular surgery 67 22 36

These substantial rates of misclassification are cause for concern. If more than 40 percent of physicians classified as lower cost are not actually lower cost, then the ability of insurance plans to control costs by channeling patients to lower-cost physicians could be severely hampered. Patients might also be disappointed if they were to change physicians expecting to receive lower-cost care but then not realize any savings.

What's Next for Physician Cost Profiling?

RAND researchers have examined multiple dimensions of physician cost profiling (see the gray box below). All of the work was based on data from non-Medicare beneficiaries in one state. Nevertheless, the work strongly suggests that current methods of physician cost profiling are not ready for prime time. However, the need to curtail rising health care costs is going to become more salient in the policy debate, not less. So current cost profiling approaches need to be improved, or new approaches need to be developed. In the interim, insurers and others need to be transparent about how they create and use physician cost profiles. square


This Highlight summarizes RAND Health research reported in the following publications:

Adams JL, Mehrotra A, and McGlynn EA, Estimating Reliability and Misclassification in Physician Profiling, Santa Monica, Calif.: RAND Corporation, TR-863-MMS, 2010.

Adams JL, Mehrotra A, Thomas JW, and McGlynn EA, Physician Cost Profiling — Reliability and Risk of Misclassification: Detailed Methodology and Sensitivity Analyses, Santa Monica, Calif.: RAND Corporation, TR-799-DOL, 2010.

Adams JL, Mehrotra A, Thomas JW, and McGlynn EA, “Physician Cost Profiling — Reliability and Risk of Misclassification,” The New England Journal of Medicine, Vol. 362, No. 11, March 18, 2010, pp. 1014–1021.

Mehrotra A, Adams JL, Thomas JW, and McGlynn EA, “The Effect of Different Attribution Rules on Individual Physician Cost Profiles,” Annals of Internal Medicine, Vol. 152, No. 10, May 18, 2010, pp. 649–654.

Mehrotra A, Adams JL, Thomas JW, and McGlynn EA, “Cost Profiles: Should the Focus Be on Individual Physicians or Physician Groups?” Health Affairs, Vol. 29, No. 8, August 2010, pp. 1532–1538.


RAND Health Research on Physician Cost Profiling

This brief describes research on reliability and attribution, but these are only two of the multiple dimensions of physician cost profiling that RAND Health researchers have analyzed. Other key publications are described below.

Reliability of Physician and Physician Group Profiles

Physician organizations have argued that instead of physicians, health plans should profile physician groups. The technical report describes how one can measure the reliability of a physician or physician group cost profile. The paper in Health Affairs examines the pluses and minuses of profiling physicians and physician groups.

Adams JL, Mehrotra A, and McGlynn EA, Estimating Reliability and Misclassification in Physician Profiling, Santa Monica, Calif.: RAND Corporation, TR-863-MMS, 2010.

Mehrotra A, Adams JL, Thomas JW, and McGlynn EA, “Cost Profiles: Should the Focus Be on Individual Physicians or Physician Groups?” Health Affairs, Vol. 29, No. 8, August 2010, pp. 1532–1538.

Statistical Testing

There is some debate on the use of statistical testing to identify physician outliers. This paper discusses the advantages and disadvantages of statistical testing in cost profiling. The authors emphasize that statistical testing is not a method for overcoming low reliability.

Adams JL, McGlynn EA, Thomas JW, and Mehrotra A, “Incorporating Statistical Uncertainty in the Use of Physician Cost Profiles,” BMC Health Services Research, [EPUB March 5, 2010], Vol. 10, No. 57, March 7, 2010.

Actual Versus Standardized Costs

This report describes differences in cost profile reliability using actual and standardized costs. Using actual costs does not substantively improve reliability.

Adams JL, Mehrotra A, Thomas JW, and McGlynn EA, Physician Cost Profiling — Reliability and Risk of Misclassification: Detailed Methodology and Sensitivity Analyses, Santa Monica, Calif.: RAND Corporation, TR-799-DOL, 2010.

Two- Versus Three-Tier Systems

These reports describe the differences in misclassification when physicians are categorized into two tiers versus three tiers. Using more tiers results in higher misclassification rates.

Adams JL, Mehrotra A, and McGlynn EA, Estimating Reliability and Misclassification in Physician Profiling, Santa Monica, Calif.: RAND Corporation, TR-863-MMS, 2010.

Adams JL, Mehrotra A, Thomas JW, and McGlynn EA, Physician Cost Profiling — Reliability and Risk of Misclassification: Detailed Methodology and Sensitivity Analyses, Santa Monica, Calif.: RAND Corporation, TR-799-DOL, 2010.

Quality Profiles

Quality is another key dimension on which physicians are profiled. These reports describe the reliability and misclassification of commonly used quality measures.

Adams JL, The Reliability of Provider Profiling: A Tutorial, Santa Monica, Calif.: RAND Corporation, TR-653-NCQA, 2009.

Adams JL, Mehrotra A, and McGlynn EA, Estimating Reliability and Misclassification in Physician Profiling, Santa Monica, Calif.: RAND Corporation, TR-863-MMS, 2010.

Scholle SH, Roski J, Adams JL, Dunn DL, Kerr EA, Dugan DP, and Jensen RE, “Benchmarking Physician Performance: Reliability of Individual and Composite Measures,” American Journal of Managed Care, Vol. 14, No. 12, December 2008, pp. 833–838.


Abstracts of all RAND Health publications and full text of many research documents can be found at RAND Health.

This research highlight was written by Mary E. Vaiana.

Copyright © 2010 RAND Corporation

The RAND Corporation is a nonprofit research organization providing objective analysis and effective solutions that address the challenges facing the public and private sectors around the world. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors.

RB-9523-1 (2010)