MACRA Palliative Care Quality Measure Development—Testing Summary Report

Background

Palliative care has grown tremendously in the past 20 years. Once thought to be limited to delivery in hospital or hospice settings, palliative care services can be delivered to patients in ambulatory (office) clinics or as a supplement to ongoing treatment, such as for cancer. At its core, palliative care is patient-centered care, and effective palliative care focuses on patient values and care preferences, symptom management, and peace at the end of life.

Palliative care stakeholders, including patients and their caregivers, providers, and health care systems, have come to see the value in systematically measuring the quality of palliative care. Yet there are few measures endorsed by the National Quality Forum to assess provider performance, and the performance measures that are in use are limited to narrowly defined populations, such as hospice patients. Perhaps surprisingly, given the focus of palliative care on the patient, there are no patient-reported measures of palliative care quality to provide insight into patients’ experiences of care.

The Centers for Medicare & Medicaid Services (CMS) set out to address gaps in palliative care quality measurement, particularly for accountability programs (such as the Quality Payment Program), through a cooperative agreement (1V1CMS331639-01-00) with the American Academy of Hospice and Palliative Medicine (AAHPM) as part of the Medicare Access and CHIP Reauthorization Act of 2015 (MACRA). The purpose of this agreement was to

• develop and test two patient-reported performance measures for patients receiving ambulatory palliative care
• ensure that the proposed measures incorporate patient voice and preferences and are broadly applicable to patients with serious illness and their families receiving palliative care services in a range of ambulatory primary and specialty care settings
• convene a technical expert panel that incorporates patient, caregiver, and family input directly into the performance measure development, specification, testing, and implementation processes
• submit the proposed palliative care measures to CMS's 2021 Measures Under Consideration list for the Quality Payment Program and to the National Quality Forum for review and endorsement so that providers can measure and improve the quality of care in ambulatory palliative care.

AAHPM partnered with RAND Health Care and the National Coalition for Hospice and Palliative Care to develop two patient-reported performance measures of palliative care experience in the domains of symptoms and communication. The symptom measure is intended to assess the extent to which patients who used ambulatory palliative care received the help they wanted for their pain, and the communication measure is intended to measure the extent to which they felt heard and understood by their palliative care provider and team. In this article, we describe the final measure specifications, methods, testing results, and considerations for implementation for the proposed Receiving Desired Help for Pain measure. A companion study covers the same information for the proposed Feeling Heard and Understood measure (Ahluwalia et al., 2021).

Methods and Results

The proposed Receiving Desired Help for Pain performance measure was tested rigorously in a multiphase process. We began by creating a patient experience survey instrument, then tested the wording and structure of data elements with patients who received palliative care in an ambulatory setting. We conducted an alpha test first to test and finalize the survey-based data elements and pilot the survey within five palliative care programs. Subsequently, we tested the performance measure at the national level in a beta field test, which occurred over ten rounds through 44 programs. The national beta test was paused from March to September 2020 because of interruptions in program operations during the coronavirus disease 2019 (COVID-19) public health emergency.

National Beta Test Patient Population and Sampling

Feedback from our technical expert clinical user and patient panel (TECUPP) and project advisory group reinforced the importance of including in the performance measure's target population all patients ages 18 and older who had at least one ambulatory palliative care visit. Working with the Center to Advance Palliative Care and the National Coalition for Hospice and Palliative Care, we identified 395 ambulatory palliative care programs in the United States from which to develop a sampling frame.

We contacted 238 palliative care programs, 70 of which were deemed ineligible to participate in the test for such reasons as not providing ambulatory care or not having practitioners eligible for the Merit-based Incentive Payment System (MIPS). Of the remaining programs, 44 participated by sending at least one sample file during field testing, 39 participated before and after the pause in data collection, three did not continue participation in September 2020, and two programs joined at that time. Ten were hospice sites, 24 were hospitals, and ten were ambulatory or other administrative sites.

Among these programs, we aimed to field 6,000 to 7,500 surveys and receive 2,400 to 3,000 completed surveys from patients receiving ambulatory palliative care. Patient sampling was conducted each month from November 2019 to February 2021, with a pause from March 2020 to September 2020 because of the COVID-19 pandemic. Fielding was conducted in ten rounds of 56 days each, beginning with patient prenotification and ending after 56 days. Patients were first contacted by mail or email to notify them (or their caregivers, if they were too ill to participate) that the survey would arrive soon, then they were mailed or emailed the survey. Patients were given three weeks to respond to the survey and up to eight attempts to respond to the survey through a computer-assisted telephone interview. Patients who did not respond to the survey within 56 days were considered nonrespondents.

Over the course of the national beta field test, the survey instrument changed slightly. The instrument contained 43 data elements at the start of the test. In March 2020, during the sixth round of fielding, five data elements were added regarding the COVID-19 pandemic. When we resumed fielding in September 2020, we revised the instrument to include a total of eight data elements regarding the COVID-19 pandemic.

We fielded the survey to 7,595 sampled patients across ten rounds of data collection. Of the surveys fielded, 3,356 were not returned, 1,435 were excluded from any analyses because of ineligibility for the larger study, and 2,804 were returned and included in analyses. This resulted in a 37 percent raw response rate and a 46 percent response rate when excluding ineligible patients. Responses were distributed evenly across programs. With respect to mode, the majority of returned surveys were completed by mail (as opposed to by phone or web). About 19 percent of surveys were excluded because of ineligibility, such as the patient being deceased or not being involved in completing the survey. Most respondents were female (56 percent), White (88 percent), non-Hispanic/Latino (95 percent), and highly educated. The mean age of respondents was 63 years. One-third of programs each contributed 31 to 50 completed surveys.

Critical Data Element Testing

Although many performance measures assess standardized clinical outcomes and processes of care (e.g., pain reduced to a comfortable level within 48 hours [the NQF #0209 measure]), assessing the subjective experience of pain symptoms—and receiving help for those symptoms—requires a different approach. To create a patient-reported measure of experience among ambulatory palliative care patients about receiving help for pain symptoms, we evaluated a data element during our national beta testing that asks, “In the last 6 months, did you get as much help as you wanted for your pain from this provider and team?” We also evaluated this data element's reliability (both internal consistency and test-retest) and convergent validity, as follows:

• Among the 2,804 completed surveys in our full sample, 1,926 respondents (about 67 percent) indicated that they both had pain and wanted help for their pain.
• Reliability of the data element was high, with a 0.90 test-retest correlation between Time 1 and Time 2 scores and an 88 percent agreement in responses among the two times. This supports data element reliability and supports use in the construction of the performance measure.
• Receiving Desired Help for Pain results were positively associated with high Consumer Assessment of Healthcare Providers and Systems (CAHPS) communication scores, supporting convergent validity of the data elements.

Performance Measure Testing

We evaluated the Receiving Desired Help for Pain performance measure first by evaluating the distribution of question responses, then by a set of potential risk adjusters based on program information, program data, and U.S. Census data; then by three potential denominator exclusions, one approach to scoring, statistically significant differences among programs, reliability at the program level, and measure score validity. We found the following:

• About 80 percent of respondents indicated they definitely had pain, wanted help for that pain, and received help for that pain. Although responses skewed toward the high end of the scale, variability across programs was also high.
• None of the potential risk adjustment variables were found to have significant associations with programs, which means they could confound the relationship between quality of care and measure score.
• Analysis of variance by respondent type (i.e., patient-only, proxy-assisted, and proxy-only) found no significant difference in responses from any of the respondent types. In keeping with the spirit of the performance measure, we decided to exclude proxy-only responses but retained proxy-assisted responses.

Our examination of the distribution of scores showed differences from the mean across programs.

• Bayesian generalized mixed-effects models showed a reasonable level of reliability based on between-program variability and within-program variability, given our sample sizes. Recalculations according to program size showed some sensitivity to small programs.
• The Receiving Desired Help for Pain performance measure was significantly and positively associated with the CAHPS communication measure, supporting the convergent validity of the Receiving Desired Help for Pain performance measure.

Feasibility Assessment

To assess the feasibility of implementing Receiving Desired Help for Pain in practice, we interviewed stakeholders to understand (1) costs and burden to programs, (2) programs’ perspectives on implementation, and (3) potential variability in patients’ experiences of care by racial and ethnic minority groups. Our findings are as follows:

• Using a survey vendor to administer the survey would minimize burden to programs. Most programs have prior experience using such vendors.
• Quotes from five CMS-approved survey vendors to administer the mixed-mode patient surveys varied from approximately $2,500 to $12,500 per year.
• Although program interviewees agreed on the value of asking patients to report on their experiences of receiving help for their pain, many raised concerns regarding the Receiving Desired Help for Pain measure, including unrealistic patient expectations (particularly regarding substance use) and survey bias for patients who are unhappy with their pain management.
• For quality assessment and improvement efforts, program interviewees discussed setting realistic goals, managing expectations through educational efforts, and setting boundaries regarding opioid use.
• Most programs had little to no experience with providing services over telehealth before the pandemic. Benefits of telehealth included enhanced personal connections (including with family members) and medication reconciliation; challenges included loss of human touch with patients.
• Programs indicated that their patient populations tend not to be very diverse, though patient perceptions of palliative care may not encourage racial and ethnic minorities to seek such care.
• Patients and family members who identify as Black or African American or as Asian who were interviewed for their perceptions of palliative care indicated (1) a general lack of awareness of palliative care, (2) generally feeling that their palliative care providers did their best to address their pain in most cases, and (3) higher satisfaction with palliative care providers than providers in the general health care system.

Implementation Considerations and Conclusions

As part of our examination of the testing and analytic procedures used during the national beta field test, we identified several considerations for programs planning to implement the Receiving Desired Help for Pain performance measure into clinical practice, quality improvement efforts, and quality payment programs.

First, it is important to recognize that certain populations and types of visits had to be excluded from the national beta field test, but these populations and visits will be important for programs to consider as they start to use this performance measure. Pediatric populations, for instance, were not included because these patients and their families require distinct considerations for ensuring their voice is reflected in the performance measure. Telehealth visits were not included because of their low incidence before the pandemic, although the public health emergency shifted many in-person visits to a telehealth platform. Because it is likely that telehealth visits will continue in greater numbers after the pandemic ends, this performance measure should incorporate telehealth in future testing. Indeed, programs with smaller numbers of visits tended to exhibit lower reliability, which should be considered when establishing requirements for program participation.

In terms of data collection, it is important to note that scores were lower for surveys completed by telephone. Because patients (or caregivers) were only contacted as a last resort if the survey had not been completed by mail or phone, we think this effect may be due to phone respondents being disinclined to participate because of negative experiences with palliative care. Thus, future tests could explore ways to optimize data collection. The survey instrument should be used and tested in other languages as well, particularly Spanish. Although we developed a Spanish-language translation of the survey instrument used in testing, we were unable to test the reliability and validity of the performance measure using Spanish-language data collection.

While testing this performance measure, we engaged clinicians, health systems, and patients and caregivers, and we offered guidance on how to use this performance measure in practice to drive systematic performance measurement and improvement in priority areas for palliative care. Stakeholder feedback suggests that this performance measure can be successfully implemented and can provide valuable information to guide palliative care improvement in patient experience.

We also feel that future research could focus on two areas:

• an evaluation of the implementation of this performance measure into clinical practice and regular use
• an exploration and analysis of disparities in ambulatory palliative care access, utilization, and experience, including (but not limited to) racial, gender, ethnic, cultural, and language disparities.

Receiving Desired Help for Pain: Final Measure Specifications

Performance Measure Description

The percentage of patients ages 18 years and older who had an ambulatory palliative care visit and report getting the help they wanted for their pain from their palliative care provider and team within six months of the ambulatory palliative care visit.

Performance Measure Type

This is a patient-reported outcome performance measure.

Denominator

All patients ages 18 years and older who had an ambulatory palliative care visit.

Denominator Criteria

All patients ages 18 years and older on date of encounter.

AND

Ambulatory palliative care visit, defined as

• ICD-10 Z51.5 (Encounter for Palliative Care) OR
• Provider Hospice and Palliative Care Specialty Code 17; AND
• CPT 99201-99205 (New Office Visit); OR CPT 99211-99215 (Established Office Visit); or Place of Service (POS) Code 11 – Office.

WITH

An eligible provider type: physicians (including doctors of medicine, osteopathy, dental surgery, dental medicine, podiatric medicine, and optometry), osteopathic practitioners, chiropractors, physician assistants, nurse practitioners, clinical nurse specialists, certified registered nurse anesthetists, physical therapists, occupational therapists, clinical psychologists, qualified speech-language pathologists, qualified audiologists, and registered dietitians or nutrition professionals (this list is based on 2019 MIPS eligible clinician types).

Denominator Exclusions

• Patients who respond “No” to the question “In the last 6 months, have you ever had pain?” or the question “In the last 6 months, did you want help from this provider and team for this pain?”
• Patients who do not complete and return the patient experience survey within six months of the eligible ambulatory palliative care visit.
• Patients who respond on the patient experience survey that they did not receive care by the listed ambulatory palliative care provider in the past six months (i.e., disavow the provider or program).
• Patients who were deceased when the survey reached them.
• Patients for whom a proxy completed the entire survey on their behalf for any reason (i.e., with no patient involvement), including patients who were deceased at the time the survey reached them.

Numerator

The number of patients ages 18 years and older who reported getting the help they wanted for their pain from their palliative care provider and team within six months of their ambulatory palliative care visit. The Receiving Desired Help for Pain measure is composed of a single data element: “In the last 6 months, did you get as much help as you wanted for your pain from this provider and team?” Individuals can respond using three discrete values: 0 = No; 1 = Yes, somewhat; and 2 = Yes, definitely. The performance measure is calculated using the data element response, passing the performance measure if an individual responds “Yes, definitely” to receiving the help they wanted for their pain from their palliative care provider and team and failing otherwise (i.e., if an individual responds “Yes, somewhat” or “No”).