Research Article|Articles in Press

Quality Indicators for Adolescents and Young Adults with Advanced Cancer: A Modified Delphi Process with Patients, Family Members, and Clinicians



      : Quality measures have been devised for end-of-life care of older adults with cancer, but are lacking for adolescents and young adults (AYAs).


      : We previously conducted interviews with AYAs, family caregivers, and clinicians to identify priority domains for high quality care of AYAs with advanced cancer. The goal of this study was to use a modified Delphi process to form consensus around the highest priority quality indicators.


      : A modified Delphi process was conducted with 10 AYAs with recurrent or metastatic cancer, 11 family caregivers, and 29 multidisciplinary clinicians, using small group web conferences. Participants were asked to rate the importance of each of 41 potential quality indicators, rank the 10 most important, and engage in discussion to reconcile differences.


      : Of 41 initial indicators, 34 were rated as highly important (rating 7, 8, or 9 on a 9-point scale) by >70% of participants. The panel was unable to reach consensus around the 10 most important indicators. Instead, participants recommended retaining a larger set of indicators to reflect potential for different priorities across the population, resulting in a final set of 32 indicators. Recommended indicators broadly encompassed attention to physical symptoms; quality of life; psychosocial and spiritual care; communication and decision-making; relationships with clinicians; care and treatment; and independence.


      : A patient- and family-centered process for quality indicator development led to strong endorsement of multiple potential indicators by Delphi participants. Further validation and refinement will be performed using a survey of bereaved family members.

      Key words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Pain and Symptom Management
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Bleyer A.
        The adolescent and young adult gap in cancer care and outcome.
        Curr Probl Pediatr Adolesc Health Care. 2005; 35: 182-217
      1. Cancer Epidemiology in Older Adolescents and Young Adults 15 to 29 Years of Age, Including SEER Incidence and Survival: 1975-2000.
        National Cancer Institute, NIH, Bethesda, MD2006
        • Siegel RL
        • Miller KD
        • Fuchs HE
        • Jemal A.
        Cancer statistics, 2022.
        CA: A Cancer Journal for Clinicians. 2022; 72: 7-33
        • Mack JW
        • Chen K
        • Boscoe FP
        • et al.
        High Intensity of End-of-Life Care Among Adolescent and Young Adult Cancer Patients in the New York State Medicaid Program.
        Med Care. 2015; 53: 1018-1026
        • Mack JW
        • Chen LH
        • Cannavale K
        • Sattayapiwat O
        • Cooper RM
        • Chao CR.
        End-of-Life Care Intensity Among Adolescent and Young Adult Patients With Cancer in Kaiser Permanente Southern California.
        JAMA Oncol. 2015; 1: 592-600
        • Wright AA
        • Keating NL
        • Balboni TA
        • Matulonis UA
        • Block SD
        • Prigerson HG.
        Place of death: correlations with quality of life of patients with cancer and predictors of bereaved caregivers' mental health.
        J Clin Oncol. 2010; 28: 4457-4464
        • Wright AA
        • Zhang B
        • Ray A
        • et al.
        Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment.
        JAMA. 2008; 300: 1665-1673
        • Pritchard S
        • Cuvelier G
        • Harlos M
        • Barr R.
        Palliative care in adolescents and young adults with cancer.
        Cancer. 2011; 117: 2323-2328
        • Wein S
        • Pery S
        • Zer A.
        Role of palliative care in adolescent and young adult oncology.
        J Clin Oncol. 2010; 28: 4819-4824
        • Hinds P
        • Drew D
        • Oakes LL
        • Fouladi M
        • Spunt SL
        • Church C
        • Furman WL.
        End-of-life care preferences of pediatric patients with cancer.
        J Clin Oncol. 2005; 23: 9055-9057
      2. Identifying and addressing the needs of adolescents and young adults with cancer: Workshop summary. Washington, DC 2013.

        • Nass SJ
        • Beaupin LK
        • Demark-Wahnefried W
        • et al.
        Identifying and addressing the needs of adolescents and young adults with cancer: summary of an Institute of Medicine workshop.
        Oncologist. 2015; 20: 186-195
      3. Delivering High-Quality Cancer Care : Charting a New Course for a System in Crisis.
        Institute of Medicine, Washington, DC2013
        • Dy SM
        • Lorenz KA
        • O'Neill SM
        • et al.
        Cancer Quality-ASSIST supportive oncology quality indicator set: feasibility, reliability, and validity testing.
        Cancer. 2010; 116: 3267-3275
        • Dy SM
        • Kiley KB
        • Ast K
        • et al.
        Measuring what matters: top-ranked quality indicators for hospice and palliative care from the american academy of hospice and palliative medicine and hospice and palliative nurses association.
        J Pain Symptom Manage. 2015; 49: 773-781
      4. Palliative and end-of-life care performance measures. Accessed June 1, 2015.

        • Hanson LC
        • Scheunemann LP
        • Zimmerman S
        • Rokoske FS
        • Schenck AP.
        The PEACE project review of clinical instruments for hospice and palliative care.
        J Palliat Med. 2010; 13: 1253-1260
        • Campion FX
        • Larson LR
        • Kadlubek PJ
        • Earle CC
        • Neuss MN.
        Advancing performance measurement in oncology: quality oncology practice initiative participation and quality outcomes.
        J Oncol Pract. 2011; 7 (31s-35s)
        • Lorenz KA
        • Rosenfeld K
        • Wenger N.
        Quality indicators for palliative and end-of-life care in vulnerable elders.
        J Am Geriatr Soc. 2007; 55 (S318-326)
      5. National Comprehensive Cancer Network. Practice Guidelines in Oncology. Palliative care. Accessed.

        • Earle CC
        • Landrum MB
        • Souza JM
        • Neville BA
        • Weeks JC
        • Ayanian JZ.
        Aggressiveness of cancer care near the end of life: is it a quality-of-care issue?.
        J Clin Oncol. 2008; 26: 3860-3866
      6. Goodman D, Morden N, Chang C, Fisher E, Wennberg J. Trends in Cancer Care Near the End of Life: A Dartmouth Atlas of Health Care Brief. September 4, 2013. 2013.

        • Peppercorn JM
        • Smith TJ
        • Helft PR
        • et al.
        American society of clinical oncology statement: toward individualized care for patients with advanced cancer.
        J Clin Oncol. 2011; 29: 755-760
        • Wright A
        • Mack JW
        • Balboni T
        • Massaro A
        • Matulonis U
        • Block S
        • Prigerson H.
        Influence of Patients' Preferences and Treatment Site on Cancer Patients' End-of-Life Care.
        under review. 2009;
        • Mack JW
        • Fisher L
        • Kushi L
        • et al.
        Patient, Family, and Clinician Perspectives on End-of-Life Care Quality Domains and Candidate Indicators for Adolescents and Young Adults With Cancer.
        JAMA Netw Open. 2021; 4 (e2121888)
        • Fitch K
        • Bernstein SJ
        • Aguilar MD
        • et al.
        The RAND/UCLA Appropriateness Method User's Manual.
        RAND Corporation, Santa Monica, CA2001
        • Hasson F
        • Keeney S
        • McKenna H.
        Research guidelines for the Delphi survey technique.
        J Adv Nurs. 2000; 32: 1008-1015
        • Sparks JB
        • Klamerus ML
        • Caverly TJ
        • et al.
        Planning and Reporting Effective Web-Based RAND/UCLA Appropriateness Method Panels: Literature Review and Preliminary Recommendations.
        J Med Internet Res. 2022; 24 (e33898)
        • Broder MS
        • Gibbs SN
        • Yermilov I.
        An Adaptation of the RAND/UCLA Modified Delphi Panel Method in the Time of COVID-19.
        J Healthc Leadersh. 2022; 14: 63-70
        • Nasa P
        • Jain R
        • Juneja D.
        Delphi methodology in healthcare research: How to decide its appropriateness.
        World J Methodol. 2021; 11: 116-129
        • Steinhauser KE
        • Christakis NA
        • Clipp EC
        • McNeilly M
        • McIntyre L
        • Tulsky JA.
        Factors considered important at the end of life by patients, family, physicians, and other care providers.
        JAMA. 2000; 284: 2476-2482
        • Wiener L
        • Bedoya S
        • Battles H
        • et al.
        Voicing their choices: Advance care planning with adolescents and young adults with cancer and other serious conditions.
        Palliative and Supportive Care. 2022; 20: 462-470
        • Bedoya SZ
        • Fry A
        • Gordon ML
        • et al.
        Adolescent and Young Adult Initiated Discussions of Advance Care Planning: Family Member, Friend and Health Care Provider Perspectives.
        Frontiers in Psychology. 2022; 13
        • Johnston EE
        • Martinez I
        • Wolfe J
        • Asch SM.
        Quality measures for end-of-life care for children with cancer: A modified Delphi approach.
        Cancer. 2021; 127: 2571-2578
        • Ananth P
        • Mun S
        • Reffat N
        • et al.
        Refining Patient-Centered Measures of End-of-Life Care Quality for Children With Cancer.
        JCO Oncol Pract. 2022; 18 (e372-e382)