Patient-Reported Quality Measures for Palliative Care: The Time is now



      While progress has been made in the ability to measure the quality of hospice and specialty palliative care, there are notable gaps. A recent analysis conducted by Center for Medicare and Medicaid Services (CMS) revealed a paucity of patient-reported measures, particularly in palliative care domains such as symptom management and communication.


      The research team, consisting of quality measure and survey developers, psychometricians, and palliative care clinicians, used established state-of-the art methods for developing and testing patient-reported measures.


      We applied a patient-centered, patient-engaged approach throughout the development and testing process. This sequential process included 1) an information gathering phase; 2) a pre-testing phase; 3) a testing phase; and 4) an endorsement phase.


      To fill quality measure gaps identified during the information gathering phase, we selected two draft measures (“Feeling Heard and Understood” and “Receiving Desired Help for Pain”) for testing with patients receiving palliative care in clinic-based settings. In the pre-testing phase, we used an iterative process of cognitive interviews to refine draft items and corresponding response options for the proposed measures. The alpha pilot test supported establishment of protocols for the national beta field test. Measures met conventional criteria for reliability, had strong face and construct validity, and there was diversity in program level scores. The measures received National Quality Forum (NQF) endorsement.


      These measures highlight the key role of patient voices in palliative care and fill a much-needed gap for patient-reported experience measures in our field.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Pain and Symptom Management
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • National Consensus Project for Quality Palliative Care
        Clinical Practice Guidelines for Quality Palliative Care. 4th edition. National Coalition for Hospice and Palliative Care, Richmond, VA2018
        • Ahluwalia SC
        • Chen C
        • Raaen L
        • et al.
        A systematic review in support of the National Consensus Project Clinical Practice guidelines for quality palliative care, Fourth edition.
        J Pain Symptom Manage. 2018; 56: 831-870
      1. National Quality Forum. Palliative and End-of-Life Care. Last accessed 12/4/2022.

        • Dy SM
        • Kiley KB
        • Ast K
        • et al.
        Measuring what matters: top-ranked quality indicators for Hospice and Palliative Care from the American Academy of Hospice and Palliative Medicine and Hospice and Palliative Nurses Association.
        J Pain Symptom Manage. 2015; 49: 773-781
        • Zheng NT
        • Li Q
        • Hanson LC
        • et al.
        Nationwide quality of hospice care: findings from the Centers For Medicare & Medicaid Services hospice quality reporting program.
        J Pain Symptom Manage. 2018; 55 (427–32.e1)
        • Anhang Price R
        • Stucky B
        • Parast L
        • et al.
        Development of valid and reliable measures of patient and family experiences of hospice care for public reporting.
        J Palliat Med. 2018; 21: 924-932
        • Schenck AP
        • Rokoske FS
        • Durham DD
        • Cagle JG
        • Hanson LC.
        The PEACE project: identification of quality measures for hospice and palliative care.
        J Palliat Med. 2010; 13: 1451-1459
        • Lorenz KA
        • Rosenfeld K
        • Wenger N.
        Quality indicators for palliative and end-of-life care in vulnerable elders.
        J Am Geriatr Soc. 2007; 55: S318-S326
        • Walling AM
        • Asch SM
        • Lorenz KA
        • et al.
        The quality of care provided to hospitalized patients at the end of life.
        Arch Intern Med. 2010; 170: 1057-1063
        • Dy SM
        • Lorenz KA
        • O'Neill SM
        • et al.
        Cancer quality-ASSIST supportive oncology quality indicator set: feasibility, reliability, and validity testing.
        Cancer. 2010; 116: 3267-3275
        • Walling AM
        • Tisnado D
        • Asch SM
        • et al.
        The quality of supportive cancer care in the veterans affairs health system and targets for improvement.
        JAMA Intern Med. 2013; 173: 2071-2079
      2. U.S. Department of Veterans Affairs. Center for health equity research and promotion: the bereaved family survey-inpatient Available from:

        • Anhang Price R
        • Tolpadi A
        • Schlang D
        • et al.
        Characteristics of Hospices providing high-quality care.
        J Palliat Med. 2020; 23: 1639-1643
        • Price RA
        • Parast L
        • Haas A
        • Teno JM
        • Elliott MN.
        Black and Hispanic patients receive Hospice care similar to that of White patients when in the same Hospices.
        Health Aff (Millwood). 2017; 36: 1283-1290
      3. Centers for Medicare & Medicaid Services. CMS quality measure development plan environmental scan and gap analysis report (MACRA, Section 102). Group HSA, editor. Baltimore, MD.

      4. Chen EK, Ahluwalia SC, Shetty K, et al. Development of palliative care quality measures for outpatients in a clinic-based setting: A report on information gathering activities. 2020.

      5. Ahluwalia SC, Vegetabile BG, Edelen MO, et al. MACRA palliative care quality measure development—testing summary report: measure name: receiving desired help for pain. 2021.

        • Ahluwalia SC
        • Vegetabile BG
        • Edelen MO
        • et al.
        MACRA palliative care quality measure development–testing summary report: Measure name: Feeling heard and understood.
        RAND Corporation, Santa Monica, CA2021
        • Rodriguez A
        • Edelen MO
        • Scherling A
        • et al.
        MACRA palliative care quality measure development: Summary report of alpha testing.
        RAND Corporation, Santa Monica, CA2020
        • Ahluwalia SC
        • Vegetabile BG
        • Edelen MO
        • et al.
        New patient-centered quality measures for office-based palliative care.
        RAND Corporation, Santa Monica, CA2021
        • Rollison J
        • Bandini JI
        • Gilbert M
        • Phillips J
        • Ahluwalia SC.
        Incorporating the patient and caregiver voice in palliative care quality measure development.
        J Pain Symptom Manage. 2022; 63: 293-300
        • Bandini JI
        • Scherling A
        • Farmer C
        • et al.
        Experiences with telehealth for outpatient palliative care: findings from a mixed-methods study of patients and providers across the United States.
        J Palliat Med. 2022; 25: 1079-1087
        • Edelen MO
        • Rodriguez A
        • Huang W
        • Gramling R
        • Ahluwalia SC.
        A novel scale to assess palliative care patients' experience of feeling heard and understood.
        J Pain Symptom Manage. 2022; 63 (689–697.e1)
        • National Quality Forum
        Building a roadmap from patient-reported outcome measures to patient-reported outcome performance measures.
      6. Schultz E, Ast K, Kirch R. Centering measurement on patients and family caregivers while developing two novel quality measures. The Medical Care Blog. April 15, 2022., Last accessed 12/4/2022.

      7. Schultz E, Ast K, Sullivan G. Measuring what matters to patients ant their family caregivers: Measure development isn’t just for measure developers. The Medical Care Blog. March 31, 2022. Last accessed 12/4/2022.

      8. American Academy of Hospice and Palliative Medicine. Summary of technical expert clinical user patient panel (TECUPP) meeting and measure specification panel (MSP) meeting April 10 - April 11, 2019. 2019.

      9. American Academy of Hospice and Palliative Medicine. Summary of technical expert clinical user patient panel (TECUPP) virtual meeting October 21, 2019. 2019.

      10. American Academy of Hospice and Palliative Medicine. Summary of technical expert clinical user patient panel (TECUPP) virtual meeting June 15, 2020. 2020.

        • Ingersoll LT
        • Saeed F
        • Ladwig S
        • et al.
        Feeling heard and understood in the hospital environment: benchmarking communication quality among patients with advanced cancer before and after palliative care consultation.
        J Pain Symptom Manage. 2018; 56: 239-244
        • Gramling R
        • Stanek S
        • Ladwig S
        • et al.
        Feeling heard and understood: a patient-reported quality measure for the inpatient palliative care setting.
        J Pain Symptom Manage. 2016; 51: 150-154
        • Savage SR
        • Kirsh KL
        • Passik SD.
        Challenges in using opioids to treat pain in persons with substance use disorders.
        Addict Sci Clin Pract. 2008; 4: 4-25
        • Feldman DB
        • Periyakoil VS.
        Posttraumatic stress disorder at the end of life.
        J Palliat Med. 2006; 9: 213-218
        • National Quality Forum
        Ambulatory Palliative Care Patients’ Experience of feeling heard and understood.
        2021 (Available from:)
        • National Quality Forum
        Ambulatory Palliative Care Patients’ Experience of receiving desired help for pain.
        2021 (Available from:)
        • Higashi T
        • Shekelle PG
        • Adams JL
        • et al.
        Quality of care is associated with survival in vulnerable older patients.
        Ann Intern Med. 2005; 143: 274-281
        • Basch E
        • Deal AM
        • Dueck AC
        • et al.
        Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment.
        JAMA. 2017; 318197198
      11. Center to Advance Palliative Care. CAPC payment primer- what to know about payment for palliative care delivery. Available from:

      12. California Health Care Foundation. Lessons learned from payer-provider partnerships for community-based palliative care. 2018.

        • Teno JM
        • Price RA
        • Makaroun LK.
        Challenges of measuring quality of community-based programs for seriously Ill individuals and their families.
        Health Aff (Millwood). 2017; 36: 1227-1233
        • Check DK
        • Kaufman BG
        • Kamal AH
        • Casarett DJ.
        Top ten tips palliative care clinicians should know about integrating population health principles into practice.
        J Palliat Med. 2020; 23: 568-572
        • Ritchie CS
        • Leff B.
        Population health and tailored medical care in the home: the roles of home-based primary care and home-based palliative care.
        J Pain Symptom Manage. 2018; 55: 1041-1046
        • Casarett D
        • Teno J.
        Why population health and palliative care need each other.
        JAMA. 2016; 316: 27-28
        • Kelley AS
        • Hanson LC
        • Ast K
        • et al.
        The serious illness population: ascertainment via electronic health record or claims data.
        J Pain Symptom Manage. 2021; 62: e148-e155
        • Walling AM
        • Sudore RL
        • Bell D
        • et al.
        Population-based pragmatic trial of advance care planning in primary care in the University of California Health System.
        J Palliat Med. 2019; 22: 72-81
        • Tandon P
        • Walling A
        • Patton H
        • Taddei T.
        AGA clinical practice update on palliative care management in cirrhosis: expert review.
        Clin Gastroenterol Hepatol. 2021; 19: 646-656.e3
        • Ferrell BR
        • Temel JS
        • Temin S
        • Smith TJ.
        Integration of palliative care into standard oncology care: ASCO Clinical Practice Guideline Update Summary.
        J Oncol Pract. 2017; 13: 119-121
        • Lanken PN
        • Terry PB
        • Delisser HM
        • et al.
        An official American Thoracic Society clinical policy statement: palliative care for patients with respiratory diseases and critical illnesses.
        Am J Respir Crit Care Med. 2008; 177: 912-927
        • Galla JH.
        Clinical practice guideline on shared decision-making in the appropriate initiation of and withdrawal from dialysis. The Renal Physicians Association and the American Society of Nephrology.
        J Am Soc Nephrol. 2000; 11: 1340-1342
        • Braun LT
        • Grady KL
        • Kutner JS
        • et al.
        Palliative care and cardiovascular disease and stroke: a policy statement from the American Heart Association/American Stroke Association.
        Circulation. 2016; 134: e198-e225
        • Kamal AH
        • Kirkland KB
        • Meier DE
        • et al.
        A person-centered, registry-based learning health system for palliative care: a path to coproducing better outcomes, experience, value, and science.
        J Palliat Med. 2018; 21: S61-S67