Download Free Electronic Document

FormatFile SizeNotes
PDF file 1.3 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Brief
Doctor and nurse talk with husband patient about examining result from doctor, family health care, palliative care, with wife holding hand of patient at hospital.

Photo by Panumas/AdobeStock

Office-based palliative care use has expanded dramatically in the past 20 years. Palliative care programs will soon be able to measure their patients' experience of care and use those data to assess the quality of care and launch improvement efforts.

Office-based palliative care helps patients with serious illnesses get the patient-centered services they want, early in the course of their disease. Variability in the delivery of and approaches to this care, however, has complicated its inclusion in quality payment and accountability programs.

The Centers for Medicare & Medicaid Services (CMS) recognized that actionable mechanisms to systematically measure and support quality improvement for palliative care services outside hospital or hospice settings would be helpful to both patients and providers. Patients could use quality metrics to choose high-quality providers, and providers could use quality metrics to assess their quality standings relative to other providers and to launch improvement efforts.

In 2018, the American Academy of Hospice and Palliative Medicine (AAHPM), the National Coalition for Hospice and Palliative Care (NCHPC), and the RAND Corporation embarked on a multiyear project funded by CMS to engage palliative care stakeholders in the development of two patient-centered quality measures of symptom management and communication. This research brief describes how these measures were developed and tested and provides considerations for providers that plan to use them.

Stakeholder Engagement Factored Prominently in Measure Choice

People with serious, chronic illnesses have distinct physical, psychological, and spiritual health care needs (Frosch et al., 2012; Institute of Medicine, 2015; Norton et al., 2003). Knowing whether the patient feels heard and understood can help palliative care programs create and sustain an environment that meets these patients' needs (Gramling et al., 2016). In addition, patients with serious illness are commonly referred to palliative care to help with pain management, improve quality of life and functional status, and reduce acute care utilization.

The research team reviewed the literature on palliative and serious illness care assessments to find potential measures that would be particularly applicable to palliative care patients. They also conducted focus groups and interviews and participated in consensus panels to elicit the opinions of providers, patients, caregivers, patient advocates, and policymakers on the relevant aspects of patient symptoms and communication for the measures. These stakeholders agreed on assessing whether patients feel heard and understood and whether they are receiving desired help for their pain across a variety of conditions and circumstances.

Researchers Rigorously Developed and Tested Survey Items for the Measures

Creating survey items that can measure patient experience of care involved several steps. First, researchers evaluated existing measures with similar goals and tailored their wording and rating options to patients who received palliative care in office-based settings (see figure). Next, they deployed a pilot test with a small group of patients to ensure that the wording of the survey items was interpreted as intended (cognitive testing) and to set parameters for the next round of testing (alpha testing). Finally, researchers undertook robust, nationally representative testing (beta testing) to examine (1) patterns of answers by patient characteristics, (2) the feasibility of implementation in office-based palliative care programs, and (3) multiple metrics of the measures' performance across diverse programs, including their reliability and validity. Two research reports provide the full details on each measure's development and testing, as well as the survey instrument (Ahluwalia et al., 2021a; Ahluwalia et al., 2021b).

Because of various constraints, the measures were not tested with pediatric populations, with telehealth visits, or in languages other than English. These will all be important considerations to pursue in future tests so that the measures can be used extensively with confidence.

The Survey Items Accurately Measured Patient Experience

researchers sent the survey to 7,595 adult patients from 44 palliative care programs, and 2,804 surveys were completed by mail, phone, or web

Beta testing took place over a 15-month period that included the onset of the coronavirus disease 2019 (COVID-19) pandemic, with a pause in testing while many in-person palliative care programs were closed. Ultimately, researchers sent the survey to 7,595 adult patients from 44 palliative care programs, and 2,804 surveys were completed by mail, phone, or web. Most respondents were White (88 percent) and female (56 percent), with high levels of education and an average age of 63. The patient characteristics largely reflected the overall sample, with very small differences between respondents and nonrespondents.

Researchers evaluated all survey responses to assess how well the measures detect differences in patients' experiences of care across palliative care programs. Overall, scores skewed toward the high end of the scale but were distributed across the various response options. On average, 71 percent of respondents' answers used the highest score for the Feeling Heard and Understood items, indicating that a strong majority affirmed feeling heard and understood by their providers. Scores were also highly variable across programs, indicating that the items can illustrate program-level differences. Results for Receiving Desired Help for Pain were similar, with 80 percent of respondents using the highest score for the items and with high variability in responses across programs. Overall, the study's results showed that all items exhibited high reliability and validity; thus, they can consistently measure patient experience as intended.

Palliative Care Programs Will Soon Be Able to Use the Measures

As of this writing, the National Quality Forum (NQF) and other entities are set to review both measures in the coming months. Although reviews are underway, programs can begin to use the measures for local quality improvement efforts. Before surveying their patient population, programs should map out survey details, such as cost and sample sizes. Survey administration vendors and options will affect overall cost, and researchers estimated the minimum sample sizes to be 37 patients for Feeling Heard and Understood and 33 patients for Receiving Desired Help for Pain.

Formal endorsement is not expected before the end of 2021. In the meantime, AAHPM and other stakeholders are working to shepherd these measures into practice, and the palliative care community can look forward to using a data-driven approach to improving care.


  • Ahluwalia, Sangeeta C., Brian G. Vegetabile, Maria Orlando Edelen, Claude Messan Setodji, Anthony Rodriguez, Adam Scherling, Jessica Phillips, Carrie M. Farmer, Jordan M. Harrison, Julia Bandini, Wenjing Huang, Lucy Schulson, Anne Walling, Sarah Dalton, Monique Martineau, Owen Hall, Danielle Schlang, Melissa A. Bradley, and Katherine Ast, MACRA Palliative Care Quality Measure Development— Testing Summary Report: Measure Name: Feeling Heard and Understood, Santa Monica, Calif.: RAND Corporation, RR-A400-1, 2021a. As of September 3, 2021:
  • ———, MACRA Palliative Care Quality Measure Development—Testing Summary Report: Measure Name: Receiving Desired Help for Pain, Santa Monica, Calif.: RAND Corporation, RR-A400-2, 2021b. As of September 3, 2021:
  • Frosch, Dominick L., Suepattra G. May, Katharine A. S. Rendle, Caroline Tietbohl, and Glyn Elwyn, "Authoritarian Physicians and Patients' Fear of Being Labeled 'Difficult' Among Key Obstacles to Shared Decision Making," Health Affairs, Vol. 31, No. 5, May 2012, pp. 1030–1038.
  • Gramling, Robert, Susan Stanek, Susan Ladwig, Elizabeth Gajary-Coots, Jenica Cimino, Wendy Anderson, and Sally A. Norton, "Feeling Heard and Understood: A Patient-Reported Quality Measure for the Inpatient Palliative Care Setting," Journal of Pain and Symptom Management, Vol. 51, No. 2, February 2016, pp. 150–154.
  • Institute of Medicine, Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life, Washington, D.C.: National Academies Press, 2015.
  • Norton, Sally A., Virginia P. Tilden, Susan W. Tolle, Christine A. Nelson, and Susan Talamantes Eggman, "Life Support Withdrawal: Communication and Conflict," American Journal of Critical Care, Vol. 12, No. 6, November 2003, pp. 548–555.

Research conducted by

This report is part of the RAND research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.