Advertisement

Measuring Advance Care Planning: Optimizing the Advance Care Planning Engagement Survey

Open ArchivePublished:December 30, 2016DOI:https://doi.org/10.1016/j.jpainsymman.2016.10.367

      Abstract

      Context

      A validated 82-item Advance Care Planning (ACP) Engagement Survey measures a broad range of behaviors. However, concise surveys are needed.

      Objectives

      The objective of this study was to validate shorter versions of the survey.

      Methods

      The survey included 57 process (e.g., readiness) and 25 action items (e.g., discussions). For item reduction, we systematically eliminated questions based on face validity, item nonresponse, redundancy, ceiling effects, and factor analysis. We assessed internal consistency (Cronbach's alpha) and construct validity with cross-sectional correlations and the ability of the progressively shorter survey versions to detect change one week after exposure to an ACP intervention (Pearson correlation coefficients).

      Results

      Five hundred one participants (four Canadian and three US sites) were included in item reduction (mean age 69 years [±10], 41% nonwhite). Because of high correlations between readiness and action items, all action items were removed. Because of high correlations and ceiling effects, two process items were removed. Successive factor analysis then created 55-, 34-, 15-, nine-, and four-item versions; 664 participants (from three US ACP clinical trials) were included in validity analysis (age 65 years [±8], 72% nonwhite, 34% Spanish speaking). Cronbach's alphas were high for all versions (four items 0.84–55 items 0.97). Compared with the original survey, cross-sectional correlations were high (four items 0.85; 55 items 0.97) as were delta correlations (four items 0.68; 55 items 0.93).

      Conclusion

      Shorter versions of the ACP Engagement Survey are valid, internally consistent, and able to detect change across a broad range of ACP behaviors for English and Spanish speakers. Shorter ACP surveys can efficiently measure broad ACP behaviors in research and clinical settings.

      Key Words

      Introduction

      Advance care planning (ACP) is a process that supports adults at any age or stage of health in understanding and sharing their personal values, life goals, and preferences regarding future medical care.
      IOM
      Report: Dying in America: Improving quality and honoring individual preferences near the end of life.
      Increasing attention has been paid to ACP over the past several years by health care systems, insurers, and researchers because ACP can improve patient and family satisfaction, increase alignment of medical care with patients' values, and reduce costs.
      • Klingler C.
      • In der Schmitten J.
      • Marckmann G.
      Does facilitated advance care planning reduce the costs of care near the end of life? Systematic review and ethical considerations.
      • Silveira M.J.
      • Kim S.Y.
      • Langa K.M.
      Advance directives and outcomes of surrogate decision making before death.
      • Hammes B.J.
      • Rooney B.L.
      • Gundrum J.D.
      A comparative, retrospective, observational study of the prevalence, availability, and specificity of advance care plans in a county that implemented an advance care planning microsystem.
      With the proliferation of new ACP initiatives, tools, and reimbursement models for ACP, measuring the success of an ACP program has become increasingly important. Traditionally, successful ACP has been measured only by the completion of an advance directive. However, several studies have shown that the ACP process is complex and involves many different ACP behaviors, such as identifying one's values, choosing a surrogate decision maker, and discussing values with surrogates and clinicians and completing advance directives.
      • Sudore R.L.
      • Fried T.R.
      Redefining the “planning” in advance care planning: preparing for end-of-life decision making.
      • Sudore R.L.
      • Schickedanz A.D.
      • Landefeld C.S.
      • et al.
      Engagement in multiple steps of the advance care planning process: a descriptive study of diverse older adults.
      • Fried T.R.
      • Redding C.A.
      • Robbins M.L.
      • Paiva A.
      • O'Leary J.R.
      • Iannone L.
      Stages of change for the component behaviors of advance care planning.
      • Fried T.R.
      • Redding C.A.
      • Robbins M.L.
      • Paiva A.
      • O'Leary J.R.
      • Iannone L.
      Promoting advance care planning as health behavior change: development of scales to assess decisional balance, medical and religious beliefs, and processes of change.
      To measure the complex behavior process of ACP, we created and validated the ACP Engagement Survey.
      • Sudore R.L.
      • Stewart A.L.
      • Knight S.J.
      • et al.
      Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.
      Although this survey has robust psychometric properties,
      • Sudore R.L.
      • Stewart A.L.
      • Knight S.J.
      • et al.
      Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.
      the original version is too long (82 items, mean of 49 minutes to complete)
      • Howard M.
      • Bonham A.J.
      • Heyland D.K.
      • et al.
      Measuring engagement in advance care planning: a cross-sectional multicentre feasibility study.
      for widespread use in research and clinical settings. The purpose of this study was to create and validate progressively shorter versions of the survey that would be able to detect change in response to an ACP intervention.

      Methods

      The development and validation of the ACP Engagement Survey have been described in detail elsewhere.
      • Sudore R.L.
      • Stewart A.L.
      • Knight S.J.
      • et al.
      Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.
      • Sudore R.L.
      • Knight S.J.
      • McMahan R.D.
      • et al.
      A novel website to prepare diverse older adults for decision making and advance care planning: a pilot study.
      In short, the survey was guided by input from experts in ACP and based on Social Cognitive and Behavior Change Theories.
      • Sudore R.L.
      • Stewart A.L.
      • Knight S.J.
      • et al.
      Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.
      The survey focuses on four behavior change constructs (i.e., knowledge, contemplation, self-efficacy, and readiness) within four ACP domains (i.e., surrogate decision makers, values and quality of life, leeway in surrogate decision making, and asking doctors questions). After pilot testing with patients and ongoing input from ACP experts, 33 questions were added to the original survey concerning desired medical treatment and the involvement of friends and family in ACP, as has been previously described.
      • Howard M.
      • Bonham A.J.
      • Heyland D.K.
      • et al.
      Measuring engagement in advance care planning: a cross-sectional multicentre feasibility study.
      This resulted in an 82-item version of the survey with 57 behavior change “Process Measures” (measured with five-point Likert response options) and 25 ACP “Action Measures” (measured with “yes” or “no” response options) (Table 1). The survey has good internal consistency (Process Measures Cronbach's alpha 0.94) and test-retest reliability (Process Measures intraclass correlation 0.70; Action Measures 0.87).
      • Sudore R.L.
      • Stewart A.L.
      • Knight S.J.
      • et al.
      Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.
      Table 1Original ACP Engagement Survey and Questions Retained in Progressively Shorter Versions After Item Reduction
      Question #
      Items reduced before factor analysis.
      SubscaleTypeOriginal 82-Item QuestionnaireVersions
      55 Items34 Items15 ItemsNine ItemsFour Items
      Domain: medical decision maker
       1Knowledge
      Response options for Process-Knowledge items (five-point Likert responses): “Not at all,” “A little,” “Somewhat,” “Fairly,” “Extremely.”
      ProcessHow well informed are you about who can be a medical decision maker?X
       2KnowledgeProcessHow well informed are you about what makes someone a good medical decision maker?XX
       3KnowledgeProcessHow well informed are you about the types of decisions that a medical decision maker may have to make for you in the future?XX
       4Contemplation
      Response options for Process-Contemplation items (five-point Likert responses): “Never,” “Once,” “A few times,” “Several times,” “A lot.”
      ProcessHow much have you thought about who your medical decision maker should be?XX
       5ContemplationProcessHow much have you thought about asking someone to be your medical decision maker?X
       6ContemplationProcessHow much have you thought about talking with your doctors about who you want your medical decision maker to be?X
       7ContemplationProcessHow much have you thought about talking with your other family and friends about who you want your medical decision maker to be?X
       8Self-efficacy
      Response options for Process-Self-efficacy items (five-point Likert responses): “Not at all,” “A little,” “Somewhat,” “Fairly,” “Extremely.”
      ProcessHow confident are you that today you could ask someone to be your medical decision maker?XXXX
       9Self-efficacyProcessHow confident are you that today you could talk with your doctor about who you want your medical decision maker to be?XX
       10Self-efficacyProcessHow confident are you that today you could talk with your other family and friends about who you want your medical decision maker to be?XX
       11
      Items reduced before factor analysis.
      DecisionActionHave you already decided who you want your medical decision maker to be?
       12
      Items reduced before factor analysis.
      Readiness
      Response options for Process-Readiness items (five-point Likert responses): “I have never thought about it,” “I have thought about it, but I am not ready to do it,” “I am thinking about doing it in the next 6 months,” “I am definitely planning to do it in the next 30 days,” “I have already done it.”
      ProcessHow ready are you to decide who you want your medical decision maker to be?
       13
      Items reduced before factor analysis.
      Action
      Response options for Action items: “Yes,” “no,” or “I am not sure.” A “no” or “I am not sure” response are both coded as “no.”
      ActionHave you already formally asked someone to be your medical decision maker?
       14ReadinessProcessHow ready are you to formally ask someone to be your medical decision maker?XXXX
       15
      Items reduced before factor analysis.
      ActionActionHave you talked with your doctor about who you want your medical decision maker to be?
       16ReadinessProcessHow ready are you to talk with your doctor about who you want your medical decision maker to be?XXXX
       17
      Items reduced before factor analysis.
      ActionActionHave you already talked to your other family and friends about who you want your medical decision maker to be?
       18ReadinessProcessHow ready are you to talk to your other family and friends about who you want your medical decision maker to be?XX
       19
      Items reduced before factor analysis.
      ActionActionHave you signed official papers naming a person or group of people to make medical decisions for you?
       20ReadinessProcessHow ready are you to sign official papers naming a person or group of people to make medical decisions for you?XXXXX
      Domain: Quality of life—health situations
       21ContemplationProcessHow much have you thought about whether or not certain health situations would make your life not worth living?X
       22ContemplationProcessHow much have you thought about talking with your medical decision maker about whether or not certain health situations would make your life not worth living?X
       23ContemplationProcessHow much have you thought about talking with your doctor about whether or not certain health situations would make your life not worth living?X
       24ContemplationProcessHow much have you thought about talking with your other family and friends about whether or not certain health situations would make your life not worth living?X
       25Self-efficacyProcessHow confident are you that today you could talk with your medical decision maker about whether or not certain health situations would make your life not worth living?XX
       26Self- EfficacyProcessHow confident are you that today you could talk with your doctor about whether or not certain health situations would make your life not worth living?XX
       27Self-efficacyProcessHow confident are you that today you could talk with your other family and friends about whether or not certain health situations would make your life not worth living?XX
       28
      Items reduced before factor analysis.
      DecisionActionHave you already decided whether or not certain health situations would make your life not worth living?
       29ReadinessProcessHow ready are you to decide whether or not certain health situations would make your life not worth living?XX
       30
      Items reduced before factor analysis.
      ActionActionHave you talked with your decision maker about whether or not certain health situations would make your life not worth living?
       31ReadinessProcessHow ready are you to talk to your decision maker about whether or not certain health situations would make your life not worth living?XX
       32
      Items reduced before factor analysis.
      ActionActionHave you talked with your doctor about whether or not certain health situations would make your life not worth living?
       33ReadinessProcessHow ready are you to talk to your doctor about whether or not certain health situations would make your life not worth living?XX
       34
      Items reduced before factor analysis.
      ActionActionHave you talked with your other family and friends about whether or not certain health situations would make your life not worth living?
       35ReadinessProcessHow ready are you to talk to your other family and friends about whether or not certain health situations would make your life not worth living?XX
       36
      Items reduced before factor analysis.
      ActionActionHave you signed official papers to put your wishes in writing about whether or not certain health situations would make your life not worth living? These forms are sometimes called an advance directive or living will.
       37ReadinessProcessHow ready are you to sign official papers putting your wishes in writing about whether or not certain health situations would make your life not worth living?X
      Domain: quality of life—medical care at the end of life
       38ContemplationProcessHow much have you thought about the care you would want if you were very sick or near the end of life?X
       39ContemplationProcessHow much have you thought about talking with your medical decision maker about the care you would want if you were very sick or near the end of life?X
       40ContemplationProcessHow much have you thought about talking with your doctors about the care you would want if you were very sick or near the end of life?X
       41ContemplationProcessHow much have you thought about talking with your other family and friends about the care you would want if you were very sick or near the end of life?XX
       42Self- EfficacyProcessHow confident are you that today you could talk with your medical decision maker about the care you would want if you were very sick or near the end of life?XXXX
       43Self-efficacyProcessHow confident are you that today you could talk with your doctor about the care you would want if you were very sick or near the end of life?XXXX
       44Self-efficacyProcessHow confident are you that today you could talk with your other family and friends about the care you would want if you were very sick or near the end of life?XX
       45
      Items reduced before factor analysis.
      DecisionActionHave you already decided on the medical care you would want if you were very sick or near the end of life?
       46ReadinessProcessHow ready are you to decide on the medical care you would want if you were very sick or near the end of life?XX
       47
      Items reduced before factor analysis.
      ActionActionHave you talked with your decision maker about what kind of medical care you would want if you were very sick or near the end of life?
       48ReadinessProcessHow ready are you to talk to your decision maker about the kind of medical care you would want if you were very sick or near the end of life?XXXXX
       49
      Items reduced before factor analysis.
      ActionActionHave you ever talked with your doctor about what kind of medical care you want if you were very sick or near the end of life?
       50ReadinessProcessHow ready are you to talk to your doctor about the kind of medical care you would want if you were very sick or near the end of life?XXXXX
       51
      Items reduced before factor analysis.
      ActionActionHave you ever talked with your other family and friends about what kind of medical care you want if you were very sick or near the end of life?
       52ReadinessProcessHow ready are you to talk to your other family and friends about the kind of medical care you would want if you were very sick or near the end of life?XX
       53
      Items reduced before factor analysis.
      ActionActionHave you signed official papers to put your wishes in writing about the kind of medical care you would want if you were very sick or near the end of life? These forms are sometimes called an advance directive or living will.
       54ReadinessProcessHow ready are you to sign official papers putting your wishes in writing about the kind of medical care you would want if you were very sick or near the end of life?XXXXX
      Domain: flexibility for surrogate decision making
       55KnowledgeProcessHow well informed are you about what it means to give a medical decision maker flexibility to make future decisions?X
       56KnowledgeProcessHow well informed are you about the different amounts of flexibility a person can give their medical decision maker?X
       57ContemplationProcessHow much have you thought about the amount of flexibility you would want to give your medical decision maker?X
       58ContemplationProcessHow much have you thought about talking with your medical decision maker about how much flexibility you want to give them?XX
       59ContemplationProcessHow much have you thought about talking with your doctor about how much flexibility you want to give your decision maker?X
       60ContemplationProcessHow much have you thought about talking with other friends and family about how much flexibility you want to give your decision maker?X
       61Self-efficacyProcessHow confident are you that today you could talk with your decision maker about how much flexibility you want to give them?XX
       62Self-efficacyProcessHow confident are you that today you could talk with your doctor about how much flexibility you want to give your medical decision maker?XXX
       63Self-efficacyProcessHow confident are you that today you could talk with your other family and friends about how much flexibility you want to give your medical decision maker?XX
       64
      Items reduced before factor analysis.
      DecisionActionHave you decided how much flexibility you would want to give a medical decision maker if they have to make decisions on your behalf (meaning for you)?
       65
      Items reduced before factor analysis.
      ReadinessProcessHow ready are you to decide how much flexibility you would want to give a medical decision maker if they have to make decisions on your behalf?
       66
      Items reduced before factor analysis.
      ActionActionHave you talked with your medical decision maker about how much flexibility you want to give them?
       67ReadinessProcessHow ready are you to talk to your decision maker about how much flexibility you want to give them?XXX
       68
      Items reduced before factor analysis.
      ActionActionHave you talked with your doctor about how much flexibility you want to give your medical decision maker?
       69ReadinessProcessHow ready are you to talk to your doctor about how much flexibility you want to give your decision maker?XXX
       70
      Items reduced before factor analysis.
      ActionActionHave you already talked to your other family and friends about how much flexibility you want to give your medical decision maker?
       71ReadinessProcessHow ready are you to talk to your other family and friends about how much flexibility you want to give your medical decision maker?X
       72
      Items reduced before factor analysis.
      ActionActionHave you signed official papers to put your wishes in writing about how much flexibility to give your decision maker?
       73ReadinessProcessHow ready are you to sign official papers putting your wishes in writing about how much flexibility to give your decision maker?XX
      Domain: asking questions of medical providers
       74KnowledgeProcessHow well informed are you about the types of questions you can ask your doctor that will help you make a good medical decision?X
       75ContemplationProcessHow much have you thought about questions you will ask your doctor to help make good medical decisions?X
       76Self-efficacyProcessHow confident are you that today you could ask the right questions of your doctor to help make good medical decisions?XXX
       77
      Items reduced before factor analysis.
      ActionActionHave you ever asked your doctor about the risks of treatment?
       78
      Items reduced before factor analysis.
      ActionActionHave you ever asked your doctor about the benefits of treatments?
       79
      Items reduced before factor analysis.
      ActionActionHave you ever asked your doctor about your other options to the treatments the doctors were suggesting?
       80
      Items reduced before factor analysis.
      ActionActionHave you ever asked your doctor about what your quality of life would be like after starting a treatment?
       81
      Items reduced before factor analysis.
      ActionActionHave you ever asked your doctor to repeat information if you did not understand it the first time?
       82ReadinessProcessHow ready are you to ask your doctor questions to help you make a good medical decision?XXX
      a Items reduced before factor analysis.
      b Response options for Process-Knowledge items (five-point Likert responses): “Not at all,” “A little,” “Somewhat,” “Fairly,” “Extremely.”
      c Response options for Process-Contemplation items (five-point Likert responses): “Never,” “Once,” “A few times,” “Several times,” “A lot.”
      d Response options for Process-Self-efficacy items (five-point Likert responses): “Not at all,” “A little,” “Somewhat,” “Fairly,” “Extremely.”
      e Response options for Process-Readiness items (five-point Likert responses): “I have never thought about it,” “I have thought about it, but I am not ready to do it,” “I am thinking about doing it in the next 6 months,” “I am definitely planning to do it in the next 30 days,” “I have already done it.”
      f Response options for Action items: “Yes,” “no,” or “I am not sure.” A “no” or “I am not sure” response are both coded as “no.”
      The present study had two phases. The goal of Phase 1 “Item Reduction” was to systematically reduce items and create progressively shorter versions of the survey (Fig. 1). The goal of Phase 2 “Internal Consistency and Construct Validity” was to test the internal consistency and the construct validity of the shorter versions and ensure they could detect change in response to an ACP intervention. Informed consent was obtained from all participants, and the study was approved by the respective Institutional Review Boards in the United States and Canada.
      Figure thumbnail gr1
      Fig. 1Methods for creating and validating shorter versions of the Advance Care Planning Engagement Survey.

      Participants

      For Phase 1, “Item Reduction,” we included participants from both the United States (US) and Canada. In the US, from February 2013 to September 2014, we included baseline data of English-speaking participants enrolled in three ongoing randomized trials of ACP interventions at the San Francisco Veterans Affairs (VA) Medical Center and San Francisco General Hospital (SFGH). The methods of these trials have been published.
      • Sudore R.
      • Le G.M.
      • McMahan R.
      • Feuz M.
      • Katen M.
      • Barnes D.E.
      The advance care planning PREPARE study among older Veterans with serious and chronic illness: study protocol for a randomized controlled trial.
      In brief, participants were included if they were 60 years or older (VA) or 55 years or older (SFGH), had two or more chronic medical illnesses, and had seen a primary care physician two or more times in the past year. Patients were excluded if they had cognitive impairment or dementia, blindness, deafness, or psychosis determined from administrative data or screening.
      In Canada, from April to September 2014, we included participants who reported being able to speak and read English and whose physicians reported they did not have cognitive impairment. A convenience sample of patients from varying age groups was recruited in the following clinical settings as part of ongoing additional studies and clinical activities: 1) primary care in Hamilton, Ontario and Edmonton, Alberta, including patients of 50 years or older, 2) inpatient hospital settings in Hamilton and Kingston, Ontario, including patients of 80 years or older or 55 years or older with clinical markers of advanced chronic disease, 3) outpatient cancer centers in Edmonton, Alberta; Hamilton, Ontario; and Kelowna and Vancouver, British Columbia, including patients of 19 years or older, and 4) outpatient dialysis care centers in Edmonton, Alberta, including patients 19 years or older.
      In Phase 2, “Internal Consistency and Construct Validity,” we included participants from the US who were enrolled between February 2013 and March 2016 in three ongoing US ACP trials. Some of Phase 1 US participants were also included in Phase 2. However, because the validity analyses are descriptive and not predictive, we decided to include all trial participants enrolled at the time of the Phase 2 analysis. This included both English- and Spanish-speaking participants at the time of Phase 2 analysis. In the Phase 2, we included both baseline and one-week follow-up trial data to be able to assess the survey's ability to detect change in response to an ACP intervention.

      Phase 1, Item Reduction

      We conducted incremental analyses to create successively shorter survey versions (Fig. 1). We considered that longer versions would have superior psychometric properties and may be more appropriate for research, whereas shorter versions may have lower validity but may be more appropriate in the clinical setting.
      First, we assessed survey items for face validity. Feedback was obtained from all authors and five research assistants who were involved in data collection and had obtained feedback from patients about the survey. Over several meetings, we reviewed survey items and discussed the ACP behavior change constructs and domains thought to be most important to retain. During these meetings, the team also flagged questions that appeared particularly difficult for study participants to understand.
      Second, we determined the nonresponse or missing rate for each question as a marker of a difficult-to-answer question. Questions with nonitem response rates greater than 10% were flagged for possible deletion before factor analysis.
      • Barnett V.
      • Lewis T.
      Outliers in statistical data.
      Third, we assessed for redundant items. Specifically, there appeared to be redundancy between the Action questions (e.g., “Have you already formally asked someone to be your medical decision maker,” with response options of “yes” or “no”) and the fifth response option in the five-point Likert scale for the corresponding Readiness questions (e.g., “How ready are you to formally ask someone to be your medical decision maker,” with “5” being “I already did it”). We compared responses to these potentially redundant items and decided to delete Action Measures with agreement greater than 80%. By removing the Action instead of the Readiness measures, we would retain the detail of the five-point Readiness measures and the ability to dichotomize responses as yes/no if desired. To further address redundancy, we then assessed correlations between the remaining survey items within and across ACP domains. Items that had correlation coefficients greater than 0.80 were removed before factor analysis.
      • Furr R.M.
      • Bacharach V.R.
      Psychometrics: An introduction.
      Fourth, we looked for ceiling effects by assessing the proportion of patients who reported a “5” on the five-point Likert scale for any Process Measure. We used 70% with a response option of “5” to flag an item for removal.
      • Furr R.M.
      • Bacharach V.R.
      Psychometrics: An introduction.
      Fifth, we used exploratory factor analysis using varimax rotation to identify factors in which the items naturally grouped. Factors with eigenvalues of more than one were retained.
      • Furr R.M.
      • Bacharach V.R.
      Psychometrics: An introduction.
      These factors were reviewed, and then we selected items from each factor that either had the highest factor loading or met our face validity criteria for the most important ACP constructs determined a priori. Items are standardly removed from factor analysis if they have loading factors of less than 0.3. Factor analysis was conducted iteratively to create progressively shorter versions of the survey and ensure the appropriate items loaded on remaining factors.
      In the sixth and final reduction analysis step, all items were systematically removed, one at a time, in separate analyses for each progressively shorter versions of the survey, to determine if removal of single items improved the Cronbach's alpha. We considered deleting items whose removal from the shortened surveys improved the Cronbach's alpha greater than 0.05.
      • Furr R.M.
      • Bacharach V.R.
      Psychometrics: An introduction.

      Methods: Phase 2, Internal Consistency and Construct Validity

      We first determined internal consistency (i.e., Cronbach's alpha) for all versions of the survey that resulted from Phase 1 analyses. Then, we measured construct validity in three ways. We assessed the correlations of the overall average five-point scores of the original, 82-item version of the survey with the progressively shorter versions using Pearson correlation coefficients (Fig. 1). The overall average five-point score was created using the five-point Likert scores for the Process Measures and by assigning a value of 5 to Action Measure response options of “yes” and a zero to a response option of “no.”
      Next, we assessed whether each version of the survey was associated with patients' self-reports of prior planning, defined as having drafted a will, having made funeral plans, or completed an advance directive, using chi-square tests. We hypothesized that both the original and shorter versions of the survey would be associated with these pre-planning activities. Previous planning questions were asked before and separate from the survey.
      Finally, we felt it was highly important that the shorter versions of the survey be able to detect change in response to an ACP intervention. The aforementioned ACP trials, from which Phase 2 recipient data were obtained, use the original 82-item version of the survey. As previously described, each study arm receives an intervention (i.e., controls receive an advance directive and the intervention arm receives a directive plus an ACP Web site).
      • Sudore R.
      • Le G.M.
      • McMahan R.
      • Feuz M.
      • Katen M.
      • Barnes D.E.
      The advance care planning PREPARE study among older Veterans with serious and chronic illness: study protocol for a randomized controlled trial.
      Therefore, we expect an increase in survey scores regardless of study arm. To maintain blinding, we deleted the randomization identification and combined both arms into one cohort. We excluded individuals who had missing data for any survey item. We then assessed the change (delta) in total average five-point survey scores from baseline to one week after ACP intervention exposure and assessed the correlations of the deltas for the original, 82-item version with progressively shorter survey versions.

      Statistical Analysis

      For descriptive statistics, we used percentages, means, and standard deviations (SD). To determine item nonresponse and ceiling effects, we used percentages. For correlations, we used Pearson correlation coefficients with coefficients of 0.80 or more considered acceptable. For internal consistency, we used Cronbach's alpha with a coefficient of 0.80 or more considered acceptable. For factor analysis, we used varimax rotation with the PROC FACTOR procedures in SAS® and created scree plots with statements of “method = principal rotate = varimax scree.” For associations of the surveys with prior planning activities, we used chi-squared tests. To determine the ability of the surveys to detect change, we used Pearson correlation coefficients to assess the correlations of the deltas in average five-point survey scores. For Phase 2 only, we stratified our validation analyses on English and Spanish language.

      Results

      Participants

      For Phase 1, 501 English-speaking patients from the US (n = 352) and Canada (n = 150) were included, and for Phase 2, 664 English and Spanish-speaking patients from the US were included (Table 2). Two hundred and four English-speaking participants from the US were included in both phases.
      Table 2Patient Characteristics by Study Phase
      Phase 1: Item Reduction, n = 501, n (%)Phase 2: Validation, n = 664, n (%)
      Origin: Canadian150 (30%)0 (0%)
      Age: Mean (SD)69.2 (10.1)64.9 (7.7)
       ≥ 65 years328 (65%)307 (46%)
      Women170 (34%)303 (46%)
      Race/ethnicity
       White294 (59%)189 (28%)
       Black or African American111 (22%)139 (21%)
       Latino or Hispanic36 (7%)271 (41%)
       Asian or Pacific Islander28 (6%)41 (6%)
       Multi-ethnic or other30 (6%)23 (3%)
      Language: Spanish0 (0%)224 (34%)
      Education: Less than or equal to high school162 (32%)326 (49%)
      Health status: fair to poor169 (34%)265 (40%)
      Married/long-term relationship227 (45%)259 (39%)

      Phase 1, Item Reduction

      Face validity: The overall behavior change constructs prioritized by the team included more activating constructs, such as self-efficacy and readiness rather than knowledge and contemplation. The team also felt that readiness questions concerning being “ready to decide” were difficult for participants to understand, and two questions were flagged for poor understanding, being ready to decide about a decision maker (Table 1, Question 12) and flexibility for the decision maker (Question 65). In addition, several members felt that the ACP domains of flexibility for decision makers and asking doctors questions were less important than other standard aspects of ACP, and several members felt that flexibility was a new and difficult concept for many participants. Furthermore, the team unanimously felt that the Action Measures were redundant to the readiness questions and supported their deletion if corroborated by the analysis (as mentioned subsequently). Finally, the team felt that items about specific questions asked of a provider (i.e., risks, benefits, etc., Table 1, Questions 77–81) were not as helpful as asking about Readiness to ask questions in general (Table 1, Question 82). Although several items were flagged for possible deletion, none were deleted during the face validity stage (Fig. 1).
      No items were deleted due to nonresponse because no item had more than 10% missing responses (mean nonresponse 3.1% [±1.2%], data not shown). Redundancy between Action items (yes/no) and corresponding Readiness items (five-point Likert scale with “5” indicating “I have already done it”) was high (mean 96.1%, ±SD 5.9%), and only one of 25 items had discrepancy more than 20% (Table 1, Question 80). Therefore, we decided to remove all 25 Action items, leaving 57 Process Measure items measured on a five-point Likert scale.
      For the remaining 57 items, no Pearson correlation coefficients met the 0.80 threshold (data not shown). A correlation of 0.66 was found between items concerning readiness to choose flexibility for a decision maker and readiness to talk to the surrogate about flexibility (Table 1, questions 65 and 67). We removed Question 65 based on face validity and the correlation, leaving 56 items (Table 1, Fig. 1).
      For the remaining 56 items, only one item concerning readiness to decide on a surrogate reached our threshold for a ceiling effect and was deleted (Table 1, Question 12, data not shown).
      In factor analysis, the remaining 55 items loaded on 10 factors, which explained 68.8% of the variance (Appendices 1–5). Within these 10 factors, we chose 34 items that spanned all 10 factors, had factor loading values greater than 0.3 (all >0.45), and were prioritized based on behavior change constructs of readiness and self-efficacy in the domains of surrogates and quality of life per our face validity assessment (Appendix 1). In repeated factor analysis, the 34-item survey loaded on seven factors, which explained 66.7% of the variance. Using the earlier mentioned criteria, we then chose 15 items that spanned six factors. The 15-items loaded on three factors, which explained 61.7% of the variance. Using the earlier mentioned criteria, we then chose nine items within the three factors. The nine items loaded onto two factors that explained 59.9% of the variance. Finally, using the earlier mentioned criteria, we chose four items from two factors. These four items loaded onto one domain that explained 58.5% of the variance. All factor loading values in all analyses were greater than 0.3.
      In the final item reduction step, no step-by-step deletion of any individual item from any of the progressively shorter versions of the survey resulted in an increase in the Cronbach's alpha more than 0.0002 (data not shown). Therefore, no survey versions were further item reduced.

      Phase 2, Internal Consistency and Construct Validation

      The Cronbach's alpha for the different versions of the survey overall, and by English and Spanish speakers, ranged from 0.85 to 0.97 (all P < 0.001, Table 3). Using the original, 82-item survey as the reference, the Pearson correlation coefficients of the overall mean five-point scores with shorter surveys versions were high (0.85–0.97, all P < 0.001, Table 4). Correlation coefficients also remained high for both English and Spanish speakers.
      Table 3Overall Internal Consistency of the Original and Progressively Shorter Survey Versions in English and Spanish
      SurveyCronbach's Alpha
      Overall (n = 664)English (n = 431)Spanish (n = 233)
      82 Items0.970.970.96
      55 Items0.970.970.96
      34 Items0.960.950.95
      15 Items0.920.920.91
      Nine items0.890.890.87
      Four items0.840.860.74
      Table 4Correlation of Average Scores of Progressively Shorter Survey Versions to the Original Survey in English and Spanish
      SurveyOverall (n = 664)English (n = 431)Spanish (n = 233)
      Average Five-point Score,
      Based on an average five-point score. Process items scored on a five-point Likert scale and action items scored as “yes” = 5 and “no” = 0.
      Mean (SD)
      Correlation
      Pearson correlation coefficient.
      P-ValueAverage Five-point Score,
      Based on an average five-point score. Process items scored on a five-point Likert scale and action items scored as “yes” = 5 and “no” = 0.
      Mean (SD)
      Correlation
      Pearson correlation coefficient.
      P-ValueAverage Five-point Score,
      Based on an average five-point score. Process items scored on a five-point Likert scale and action items scored as “yes” = 5 and “no” = 0.
      Mean (SD)
      Correlation
      Pearson correlation coefficient.
      P-Value
      82 Items2.54 (.94)2.78 (.95)2.10 (.76)
      55 Items2.84 (.90)0.97<0.0013.08 (.87)0.97<0.0012.39 (.77)0.97<0.001
      34 Items3.07 (.98)0.94<0.0013.35 (.92)0.94<0.0012.54 (.85)0.94<0.001
      15 Items3.16 (1.02)0.91<0.0013.43 (.96)0.90<0.0012.66 (.94)0.90<0.001
      Nine items3.11 (1.09)0.89<0.0013.39 (1.05)0.89<0.0012.61 (.97)0.87<0.001
      Four items2.70 (1.22)0.85<0.0012.96 (1.24)0.85<0.0012.20 (1.02)0.80<0.001
      SD = standard deviation.
      a Based on an average five-point score. Process items scored on a five-point Likert scale and action items scored as “yes” = 5 and “no” = 0.
      b Pearson correlation coefficient.
      The average five-point scores on all versions of the survey were higher for people who engaged in prior planning (all P < 0.001, Table 5). In addition, all versions were able to detect change in response to an ACP intervention. Correlations of the change scores for the original survey with progressively shorter versions were high overall and for both English and Spanish speakers (range 0.66–0.94, all P < 0.001) (Table 6).
      Table 5Association of Survey Scores With Pre-Planning Activities
      N = 559. Five Phase 2 participants were missing data for one or more pre-planning activities.
      SurveyAverage Five-point Scores, Mean (SD)
      Based on an average five-point score. Process items scored on a five-point Likert scale and action items scored as “yes” = 5 and “no” = 0.
      Made Out a WillNo WillP-Value
      n = 151n = 508
      82 Items3.3 (1.0)2.3 (.8)<0.001
      55 Items3.5 (.9)2.6 (.8)<0.001
      34 Items3.8 (.9)2.9 (.9)<0.001
      15 Items3.9 (.9)2.9 (.9)<0.001
      Nine items3.9 (1.0)2.9 (1.0)<0.001
      Four items3.6 (1.2)2.4 (1.1)<0.001
      Funeral PlansNo Plans
      n = 178n = 479
      82 Items2.8 (1.0)2.5 (.9)<0.001
      55 Items3.1 (.9)2.8 (.9)<0.001
      34 Items3.3 (1.0)3.0 (1.0)<0.001
      15 Items3.4 (1.0)3.1 (1.0)<0.001
      Nine items3.4 (1.1)3.0 (1.1)<0.001
      Four items3.0 (1.2)2.6 (1.2)<0.001
      ADNo AD
      n = 148n = 505
      82 Items3.5 (.8)2.3 (.8)<0.001
      55 Items3.6 (.8)2.6 (.8)<0.001
      34 Items3.9 (.8)2.8 (.9)<0.001
      15 Items4.0 (.8)2.9 (1.0)<0.001
      Nine items4.0 (.0)2.9 (1.0)<0.001
      Four items3.8 (1.1)2.4 (1.1)<0.001
      SD = standard deviation.
      a N = 559. Five Phase 2 participants were missing data for one or more pre-planning activities.
      b Based on an average five-point score. Process items scored on a five-point Likert scale and action items scored as “yes” = 5 and “no” = 0.
      Table 6Correlation Between Change Scores in Response to an ACP Intervention Between Progressively Shorter Survey Versions and the Original Survey
      This is a blinded sample that combined data from three ongoing randomized trials and that includes both the control groups (i.e., given an advance directive only) and the intervention groups (i.e., given the PREPARE Web site plus an advance directive).
      SurveyOverall (n = 664)English (n = 431)Spanish (n = 233)
      Mean Change,
      Mean change (deltas) obtained by calculating the difference between the average five-point scores at baseline to one-week after reviewing an ACP intervention. Based on an average five-point score. Process items scored on a five-point Likert scale and action items scored as “yes” = 5 and “no” = 0.
      Mean (SD)
      CorrelationP-ValueMean Change,
      Mean change (deltas) obtained by calculating the difference between the average five-point scores at baseline to one-week after reviewing an ACP intervention. Based on an average five-point score. Process items scored on a five-point Likert scale and action items scored as “yes” = 5 and “no” = 0.
      Mean (SD)
      CorrelationP-ValueMean Change,
      Mean change (deltas) obtained by calculating the difference between the average five-point scores at baseline to one-week after reviewing an ACP intervention. Based on an average five-point score. Process items scored on a five-point Likert scale and action items scored as “yes” = 5 and “no” = 0.
      Mean (SD)
      CorrelationP-Value
      82 Items0.4 (.7)0.4 (.6)0.4 (.7)
      55 Items0.3 (.6)0.93<0.0010.3 (.6)0.93<0.0010.3 (.7)0.94<0.001
      34 Items0.3 (.7)0.89<0.0010.3 (.6)0.88<0.0010.3 (.7)0.90<0.001
      15 Items0.3 (.7)0.82<0.0010.3 (.7)0.82<0.0010.3 (.8)0.82<0.001
      Nine items0.3 (.8)0.74<0.0010.3 (.7)0.75<0.0010.4 (.9)0.75<0.001
      Four items0.4 (1.0)0.68<0.0010.4 (.9)0.66<0.0010.5 (1.0)0.70<0.001
      SD = standard deviation.
      a This is a blinded sample that combined data from three ongoing randomized trials and that includes both the control groups (i.e., given an advance directive only) and the intervention groups (i.e., given the PREPARE Web site plus an advance directive).
      b Mean change (deltas) obtained by calculating the difference between the average five-point scores at baseline to one-week after reviewing an ACP intervention. Based on an average five-point score. Process items scored on a five-point Likert scale and action items scored as “yes” = 5 and “no” = 0.

      Discussion

      Using survey data from diverse study participants from the US and Canada, we used rigorous and systematic item reduction methods to create progressively shorter versions of the ACP Engagement Survey from the original 82-item version to a 55-, 34-, 15-, nine-, and four-item version. Then, using data from multicenter randomized trials among English- and Spanish-speaking patients in the US, we demonstrated that these progressively shorter versions of the survey retained high internal consistency and had high construct validity, including the ability to detect change in response to an ACP intervention.
      The original development and validation of the 82-item ACP Engagement Survey allowed measurement of multiple ACP behaviors, in addition to advance directive completion. Although the original survey included a larger number of ACP behaviors and quantified behavior change constructs (i.e., knowledge, contemplation, self-efficacy, and readiness) for each individual ACP behavior, it was long with a mean administration time of 49 minutes.
      • Howard M.
      • Bonham A.J.
      • Heyland D.K.
      • et al.
      Measuring engagement in advance care planning: a cross-sectional multicentre feasibility study.
      The present study has helped to produce psychometrically sound shorter versions of the survey while still measuring a broad range of ACP behavior change constructs and domains. As the internal consistency and construct validity were slightly higher for longer versions of the survey, researchers may wish to use the longer versions (55, 34, and 15 items) to determine efficacy of a program or ACP tool or to detect nuanced differences in specific ACP behaviors. However, the shorter versions (i.e., nine and four items) remained psychometrically sound and may be appropriate to use for quality improvement initiatives in the clinical settings or research studies with limited resources. Decisions concerning which length of the survey to use will depend on the balance between the needed sample size to detect change in response to an ACP intervention, participant survey burden, and study resources.
      The strengths of this study include the strong theoretical basis underpinning the original 82-item survey, which was informed by Social Cognitive and Behavior Change Theories. Additional strengths were the inclusion of a multidisciplinary team of ACP content experts and racially/ethnically diverse patients from multiple medical centers from two countries, the rigorous and systematic psychometric validation in two phases, and the ability of the shorter versions of the survey to detect change in response to an ACP intervention. Furthermore, we included English and Spanish speakers in Phase 2 construct validity assessments and validity remained robust in both languages.
      This study has several limitations. Phase 1, item reduction, only included English speakers; however, construct validity remained high for Spanish speakers in Phase 2. In Phase 2, validation of the survey took place in only one area of the US, which may limit generalizability. Generalizability of our findings beyond North America is also unknown. Furthermore, in addition to rigorous psychometric analysis, our team of ACP experts used face validity to prioritize behavior change constructs and ACP domains. Other research groups may prioritize different ACP constructs and domains. Finally, we still do not know the scoring thresholds of the survey associated with patient-centered outcomes. Based on Social Cognitive and Behavior Change Theories, any increase in the average five-point score may be associated with clinically meaningful improvement in patient-centered outcomes, such as receipt of care that is consistent with patients' goals. However, further longitudinal studies will be needed to determine the exact thresholds for a full range of ACP behaviors that lead to improved ACP outcomes.
      In conclusion, several shorter versions of the ACP Engagement Survey are valid, internally consistent, and able to detect change across a broad range of ACP behaviors and ACP domains. The surveys are also ready for use among English and Spanish speakers. Having several psychometrically sound shortened versions of the survey provides flexibility to researchers and quality improvement experts when choosing surveys to measure successful ACP programs and tools in the research and clinical setting.

      Disclosures and Acknowledgments

      This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Dr. Sudore, however, is supported in part by the following grants: NIH R01AG045043, PCORI-1306–01500, VA HSR&D 11–110-2, the American Cancer Society (ACS) #19659, and NIH U24NR014637. No funding bodies had a role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, and approval of the manuscript; or decision to submit the manuscript for publication. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs. The other authors declare no conflicts of interest.

      Appendix.

      Appendix 1Factor Analysis With the 55-Item Version of the Survey, n = 501
      DomainsSubscaleQuestion #Survey QuestionFactor LoadingRetained for 34-Item Version
      Factor 1
       QOLSelf-efficacy42How confident are you that today you could talk with your medical decision maker about the care you would want if you were very sick or near the end of life?0.7505X
       FLEXSelf-efficacy62How confident are you that today you could talk with your doctor about how much flexibility you want to give your medical decision maker?0.72995X
       QOLSelf-efficacy43How confident are you that today you could talk with your doctor about the care you would want if you were very sick or near the end of life?0.72002X
       FLEXSelf-efficacy61How confident are you that today you could talk with your decision maker about how much flexibility you want to give them?0.70473X
       QOLSelf-efficacy26How confident are you that today you could talk with your doctor about whether or not certain health situations would make your life not worth living?0.66252X
       QOLSelf-efficacy25How confident are you that today you could talk with your medical decision maker about whether or not certain health situations would make your life not worth living?0.64755X
       DMSelf-efficacy8How confident are you that today you could ask someone to be your medical decision maker?0.63294X
       DMSelf-efficacy9How confident are you that today you could talk with your doctor about who you want your medical decision maker to be?0.63277X
       DMSelf-efficacy10How confident are you that today you could talk with your other family and friends about who you want your medical decision maker to be?0.53885X
      Factor 2
       QOLContemplation38How much have you thought about the care you would want if you were very sick or near the end of life?0.73172
       QOLContemplation21How much have you thought about whether or not certain health situations would make your life not worth living?0.71662
       QOLContemplation39How much have you thought about talking with your medical decision maker about the care you would want if you were very sick or near the end of life?0.71441
       QOLContemplation22How much have you thought about talking with your medical decision maker about whether or not certain health situations would make your life not worth living?0.69707
       QOLContemplation24How much have you thought about talking with your other family and friends about whether or not certain health situations would make your life not worth living?0.57988
       QOLReadiness29How ready are you to decide whether or not certain health situations would make your life not worth living?0.57048X
       QOLReadiness46How ready are you to decide on the medical care you would want if you were very sick or near the end of life?0.48871X
      Factor 3
       QOLReadiness52How ready are you to talk to your other family and friends about the kind of medical care you would want if you were very sick or near the end of life?0.69998X
       QOLReadiness35How ready are you to talk to your other family and friends about whether or not certain health situations would make your life not worth living?0.6605X
       QOLSelf-efficacy44How confident are you that today you could talk with your other family and friends about the care you would want if you were very sick or near the end of life?0.65418X
       FLEXSelf-efficacy63How confident are you that today you could talk with your other family and friends about how much flexibility you want to give your medical decision maker?0.62537X
       FLEXReadiness71How ready are you to talk to your other family and friends about how much flexibility you want to give your medical decision maker?0.5855
       QOLSelf-efficacy27How confident are you that today you could talk with your other family and friends about whether or not certain health situations would make your life not worth living?0.58294X
       QOLContemplation41How much have you thought about talking with your other family and friends about the care you would want if you were very sick or near the end of life?0.57117
       DMReadiness18How ready are you to talk to your other family and friends about who you want your medical decision maker to be?0.55163X
      Factor 4
       FLEXContemplation58How much have you thought about talking with your medical decision maker about how much flexibility you want to give them?0.77612
       FLEXContemplation57How much have you thought about the amount of flexibility you would want to give your medical decision maker?0.73945
       FLEXContemplation59How much have you thought about talking with your doctor about how much flexibility you want to give your decision maker?0.69154
       FLEXContemplation60How much have you thought about talking with other friends and family about how much flexibility you want to give your decision maker?0.67834
      Factor 5
       QOLReadiness33How ready are you to talk to your doctor about whether or not certain health situations would make your life not worth living?0.78326X
       QOLReadiness50How ready are you to talk to your doctor about the kind of medical care you would want if you were very sick or near the end of life?0.76664X
       DMReadiness16How ready are you to talk with your doctor about who you want your medical decision maker to be?0.65307X
       FLEXReadiness69How ready are you to talk to your doctor about how much flexibility you want to give your decision maker?0.65113X
       QOLContemplation23How much have you thought about talking with your doctor about whether or not certain health situations would make your life not worth living?0.56606
       QOLContemplation40How much have you thought about talking with your doctors about the care you would want if you were very sick or near the end of life?0.56388
      Factor 6
       DMKnowledge2How well informed are you about what makes someone a good medical decision maker?0.85653X
       DMKnowledge3How well informed are you about the types of decisions that a medical decision maker may have to make for you in the future?0.78301X
       DMKnowledge1How well informed are you about who can be a medical decision maker?0.7825
       FLEXKnowledge55How well informed are you about what it means to give a medical decision maker flexibility to make future decisions?0.60476
       FLEXKnowledge56How well informed are you about the different amounts of flexibility a person can give their medical decision maker?0.55696
      Factor 7
       DMContemplation5How much have you thought about asking someone to be your medical decision maker?0.74107
       DMContemplation4How much have you thought about who your medical decision maker should be?0.69099X
       DMContemplation7How much have you thought about talking with your other family and friends about who you want your medical decision maker to be?0.67899
       DMContemplation6How much have you thought about talking with your doctors about who you want your medical decision maker to be?0.59162
      Factor 8
       QOLReadiness54How ready are you to sign official papers putting your wishes in writing about the kind of medical care you would want if you were very sick or near the end of life?0.78719X
       QOLReadiness37How ready are you to sign official papers putting your wishes in writing about whether or not certain health situations would make your life not worth living?0.7453
       FLEXReadiness73How ready are you to sign official papers putting your wishes in writing about how much flexibility to give your decision maker?0.70768X
       DMReadiness20How ready are you to sign official papers naming a person or group of people to make medical decisions for you?0.45441X
      Factor 9
       QOLReadiness48How ready are you to talk to your decision maker about the kind of medical care you would want if you were very sick or near the end of life?0.60591X
       QOLReadiness31How ready are you to talk to your decision maker about whether or not certain health situations would make your life not worth living?0.59538X
       DMReadiness14How ready are you to formally ask someone to be your medical decision maker?0.54097X
       FLEXReadiness67How ready are you to talk to your decision maker about how much flexibility you want to give them?0.47456X
      FACTOR 10
       QUESTContemplation75How much have you thought about questions you will ask your doctor to help make good medical decisions?0.70726
       QUESTKnowledge74How well informed are you about the types of questions you can ask your doctor that will help you make a good medical decision?0.67435
       QUESTSelf-efficacy76How confident are you that today you could ask the right questions of your doctor to help make good medical decisions?0.65978X
       QUESTReadiness82How ready are you to ask your doctor questions to help you make a good medical decision?0.64189X
      DM = medical decision maker; QOL = quality of life; FLEX = flexibility for the surrogate decision maker; QUEST = asking questions of medical providers.
      Appendix 2Factor Analysis With the 34-Item Version of the Survey, n = 501
      DomainSubscaleQuestion #Survey QuestionFactor LoadingRetained for 15-Item Version
      Factor 1
       QOLSelf-efficacy43How confident are you that today you could talk with your doctor about the care you would want if you were very sick or near the end of life?0.75753X
       FLEXSelf-efficacy62How confident are you that today you could talk with your doctor about how much flexibility you want to give your medical decision maker?0.7489X
       QOLSelf-efficacy26How confident are you that today you could talk with your doctor about whether or not certain health situations would make your life not worth living?0.71715
       QOLSelf-efficacy42How confident are you that today you could talk with your medical decision maker about the care you would want if you were very sick or near the end of life?0.67081X
       FLEXSelf-efficacy61How confident are you that today you could talk with your decision maker about how much flexibility you want to give them?0.6225X
       QOLSelf-efficacy25How confident are you that today you could talk with your medical decision maker about whether or not certain health situations would make your life not worth living?0.60905
       DMSelf-efficacy9How confident are you that today you could talk with your doctor about who you want your medical decision maker to be?0.54063
      Factor 2
       QOLReadiness29How ready are you to decide whether or not certain health situations would make your life not worth living?0.80307
       QOLReadiness31How ready are you to talk to your decision maker about whether or not certain health situations would make your life not worth living?0.74796
       QOLReadiness46How ready are you to decide on the medical care you would want if you were very sick or near the end of life?0.63459
       QOLReadiness48How ready are you to talk to your decision maker about the kind of medical care you would want if you were very sick or near the end of life?0.61073X
       DMReadiness14How ready are you to formally ask someone to be your medical decision maker?0.45894X
      Factor 3
       FLEXReadiness69How ready are you to talk to your doctor about how much flexibility you want to give your decision maker?0.76301X
       QOLReadiness33How ready are you to talk to your doctor about whether or not certain health situations would make your life not worth living?0.75602
       QOLReadiness50How ready are you to talk to your doctor about the kind of medical care you would want if you were very sick or near the end of life?0.75397X
       DMReadiness16How ready are you to talk with your doctor about who you want your medical decision maker to be?0.7221X
      Factor 4
       QOLSelf-efficacy44How confident are you that today you could talk with your other family and friends about the care you would want if you were very sick or near the end of life?0.77791
       FLEXSelf-efficacy63How confident are you that today you could talk with your other family and friends about how much flexibility you want to give your medical decision maker?0.75515
       QOLSelf-efficacy27How confident are you that today you could talk with your other family and friends about whether or not certain health situations would make your life not worth living?0.66325
       QOLReadiness52How ready are you to talk to your other family and friends about the kind of medical care you would want if you were very sick or near the end of life?0.64977
       QOLReadiness35How ready are you to talk to your other family and friends about whether or not certain health situations would make your life not worth living?0.6048
       DMReadiness18How ready are you to talk to your other family and friends about who you want your medical decision maker to be?0.49298
      Factor 5
       FLEXReadiness73How ready are you to sign official papers putting your wishes in writing about how much flexibility to give your decision maker?0.75295
       QOLReadiness54How ready are you to sign official papers putting your wishes in writing about the kind of medical care you would want if you were very sick or near the end of life?0.66702X
       DMReadiness20How ready are you to sign official papers naming a person or group of people to make medical decisions for you?0.52597X
       FLEXReadiness67How ready are you to talk to your decision maker about how much flexibility you want to give them?0.48607X
       FLEXContemplation58How much have you thought about talking with your medical decision maker about how much flexibility you want to give them?0.4728
      Factor 6
       DMSelf-efficacy8How confident are you that today you could ask someone to be your medical decision maker?0.66856X
       DMSelf-efficacy10How confident are you that today you could talk with your other family and friends about who you want your medical decision maker to be?0.59714
       DMContemplation4How much have you thought about who your medical decision maker should be?0.44845
      Factor 7
       DMKnowledge3How well informed are you about the types of decisions that a medical decision maker may have to make for you in the future?0.79318
       DMKnowledge2How well informed are you about what makes someone a good medical decision maker?0.77068
       QUESTSelf-efficacy76How confident are you that today you could ask the right questions of your doctor to help make good medical decisions?0.6021X
       QUESTReadiness82How ready are you to ask your doctor questions to help you make a good medical decision?0.39666X
      DM = medical decision maker; QOL = quality of life; FLEX = flexibility for the surrogate decision maker; QUEST = asking questions of medical providers.
      Appendix 3Factor Analysis With the 15-Item Version of the Survey, n = 501
      DomainSubscaleQuestion #Survey QuestionFactor LoadingRetained for Nine-Item Version
      Factor 1
       DMReadiness14How ready are you to formally ask someone to be your medical decision maker?0.78782X
       DMReadiness20How ready are you to sign official papers naming a person or group of people to make medical decisions for you?0.77456X
       QOLReadiness48How ready are you to talk to your decision maker about the kind of medical care you would want if you were very sick or near the end of life?0.69771X
       QOLReadiness54How ready are you to sign official papers putting your wishes in writing about the kind of medical care you would want if you were very sick or near the end of life?0.58319X
       DMSelf-efficacy8How confident are you that today you could ask someone to be your medical decision maker?0.58285X
      Factor 2
       FLEXSelf-efficacy62How confident are you that today you could talk with your doctor about how much flexibility you want to give your medical decision maker?0.81753
       QOLSelf-efficacy43How confident are you that today you could talk with your doctor about the care you would want if you were very sick or near the end of life?0.7937X
       QOLSelf-efficacy42How confident are you that today you could talk with your medical decision maker about the care you would want if you were very sick or near the end of life?0.65406X
       FLEXSelf-efficacy61How confident are you that today you could talk with your decision maker about how much flexibility you want to give them?0.65401
       QUESTSelf-efficacy76How confident are you that today you could ask the right questions of your doctor to help make good medical decisions?0.63063
      Factor 3
       FLEXReadiness69How ready are you to talk to your doctor about how much flexibility you want to give your decision maker?0.81969
       QOLReadiness50How ready are you to talk to your doctor about the kind of medical care you would want if you were very sick or near the end of life?0.75862X
       DMReadiness16How ready are you to talk with your doctor about who you want your medical decision maker to be?0.71665X
       FLEXReadiness67How ready are you to talk to your decision maker about how much flexibility you want to give them?0.57136
       QUESTReadiness82How ready are you to ask your doctor questions to help you make a good medical decision?0.43226
      DM = medical decision maker; QOL = quality of life; FLEX = flexibility for the surrogate decision maker; QUEST = asking questions of medical providers.
      Appendix 4Factor Analysis With the Nine-Item Version of the Survey, n = 501
      DomainSubscaleQuestion #Survey QuestionFactor LoadingRetained for Four-Item Version
      Factor 1
       QOLReadiness50How ready are you to talk to your doctor about the kind of medical care you would want if you were very sick or near the end of life?0.78746X
       DMReadiness16How ready are you to talk with your doctor about who you want your medical decision maker to be?0.72991
       QOLReadiness48How ready are you to talk to your decision maker about the kind of medical care you would want if you were very sick or near the end of life?0.67932X
       QOLReadiness54How ready are you to sign official papers putting your wishes in writing about the kind of medical care you would want if you were very sick or near the end of life?0.66582X
       DMReadiness20How ready are you to sign official papers naming a person or group of people to make medical decisions for you?0.58578X
       DMReadiness14How ready are you to formally ask someone to be your medical decision maker?0.58201
      Factor 2
       QOLSelf-efficacy42How confident are you that today you could talk with your medical decision maker about the care you would want if you were very sick or near the end of life?0.86799
       DMSelf-efficacy8How confident are you that today you could ask someone to be your medical decision maker?0.81362
       QOLSelf-efficacy43How confident are you that today you could talk with your doctor about the care you would want if you were very sick or near the end of life?0.67436
      DM = medical decision maker; QOL = quality of life.
      Appendix 5Factor Analysis With the Four-Item Version of the Survey, n = 501
      DomainSubscaleQuestion #Survey QuestionFactor Loading
      Factor 1
       QOLReadiness48How ready are you to talk to your decision maker about the kind of medical care you would want if you were very sick or near the end of life?0.79526
       QOLReadiness50How ready are you to talk to your doctor about the kind of medical care you would want if you were very sick or near the end of life?0.68681
       QOLReadiness54How ready are you to sign official papers putting your wishes in writing about the kind of medical care you would want if you were very sick or near the end of life?0.79998
       DMReadiness20How ready are you to sign official papers naming a person or group of people to make medical decisions for you?0.78022
      DM = medical decision maker; QOL = quality of life.

      References

        • IOM
        Report: Dying in America: Improving quality and honoring individual preferences near the end of life.
        2014 (Available at:) (Accessed July 11, 2016)
        • Klingler C.
        • In der Schmitten J.
        • Marckmann G.
        Does facilitated advance care planning reduce the costs of care near the end of life? Systematic review and ethical considerations.
        Palliat Med. 2016; 30: 423-433
        • Silveira M.J.
        • Kim S.Y.
        • Langa K.M.
        Advance directives and outcomes of surrogate decision making before death.
        N Engl J Med. 2010; 362: 1211-1218
        • Hammes B.J.
        • Rooney B.L.
        • Gundrum J.D.
        A comparative, retrospective, observational study of the prevalence, availability, and specificity of advance care plans in a county that implemented an advance care planning microsystem.
        J Am Geriatr Soc. 2010; 58: 1249-1255
        • Sudore R.L.
        • Fried T.R.
        Redefining the “planning” in advance care planning: preparing for end-of-life decision making.
        Ann Intern Med. 2010; 153: 256-261
        • Sudore R.L.
        • Schickedanz A.D.
        • Landefeld C.S.
        • et al.
        Engagement in multiple steps of the advance care planning process: a descriptive study of diverse older adults.
        J Am Geriatr Soc. 2008; 56: 1006-1013
        • Fried T.R.
        • Redding C.A.
        • Robbins M.L.
        • Paiva A.
        • O'Leary J.R.
        • Iannone L.
        Stages of change for the component behaviors of advance care planning.
        J Am Geriatr Soc. 2010; 58: 2329-2336
        • Fried T.R.
        • Redding C.A.
        • Robbins M.L.
        • Paiva A.
        • O'Leary J.R.
        • Iannone L.
        Promoting advance care planning as health behavior change: development of scales to assess decisional balance, medical and religious beliefs, and processes of change.
        Patient Educ Couns. 2012; 86: 25-32
        • Sudore R.L.
        • Stewart A.L.
        • Knight S.J.
        • et al.
        Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.
        PLoS One. 2013; 8: e72465
        • Howard M.
        • Bonham A.J.
        • Heyland D.K.
        • et al.
        Measuring engagement in advance care planning: a cross-sectional multicentre feasibility study.
        BMJ Open. 2016; 6: e010375
        • Sudore R.L.
        • Knight S.J.
        • McMahan R.D.
        • et al.
        A novel website to prepare diverse older adults for decision making and advance care planning: a pilot study.
        J Pain Symptom Manage. 2014; 47: 674-686
        • Sudore R.
        • Le G.M.
        • McMahan R.
        • Feuz M.
        • Katen M.
        • Barnes D.E.
        The advance care planning PREPARE study among older Veterans with serious and chronic illness: study protocol for a randomized controlled trial.
        Trials. 2015; 16: 570
        • Barnett V.
        • Lewis T.
        Outliers in statistical data.
        John Wiley & Sons, New York1984
        • Furr R.M.
        • Bacharach V.R.
        Psychometrics: An introduction.
        Sage Publications, Thousand Oaks, California2014