Advertisement

Adaptation and Preliminary Validation of the Advance Care Planning Engagement Survey for Surrogate Decision Makers

Open ArchivePublished:January 23, 2019DOI:https://doi.org/10.1016/j.jpainsymman.2019.01.008

      Abstract

      Context

      Although measures that assess patient engagement in the advance care planning (ACP) process exist, there are no validated tools to assess surrogate decision makers' (SDMs') role in ACP.

      Objectives

      The objective of this study was to adapt and begin to validate a patient-oriented questionnaire for use with SDMs of patients with chronic illness.

      Methods

      Questions from the 55-item patient-oriented ACP engagement survey were adapted for SDMs and assessed for face validity. The resultant 47-item questionnaire was administered to 65 SDMs of patients with chronic illness. Responses were assessed and items were flagged for removal based on item redundancy, nonresponses, and ceiling effects. A preliminary exploratory factor analysis was performed, internal consistency was assessed, and domains were constructed based on findings.

      Results

      The 47-item questionnaire was administered to 65 participants (mean age 51.8; 81% female; 96% Caucasian). Seventeen items were removed owing to redundancy (r > 0.80), and 13 items lacking face validity were removed. In a preliminary exploratory factor analysis of the resultant 17-item questionnaire, a three-factor solution was deemed most statistically and conceptually sound. Items were organized into domains: 1) serving as an SDM (seven items); 2) contemplation (four items); 3) readiness (six items). Internal consistency for each domain was high (Cronbach alpha 0.90–0.91).

      Conclusion

      The 17-item ACP engagement survey for SDMs (ACP-17-SDM) is a conceptually sound and reliable questionnaire adaptation of the original ACP engagement survey. This questionnaire may be used by researchers in parallel with the patient-oriented ACP engagement survey to more fully understand the impact of ACP interventions on SDMs. Larger studies are needed to more closely examine construct validity.

      Key Words

      Introduction

      Advance care planning (ACP) was recently defined by an international consensus panel as a process intended to “help ensure people receive medical care that is consistent with their values, goals, and preferences during serious and chronic illness.”
      • Sudore R.L.
      • Lum H.D.
      • You J.J.
      • et al.
      Defining advance care planning for adults: a consensus definition from a multidisciplinary Delphi panel.
      The panel also noted that ACP should include selection of surrogate decision makers (SDMs) capable of making medical decisions should the patient become incapacitated.
      • Sudore R.L.
      • Lum H.D.
      • You J.J.
      • et al.
      Defining advance care planning for adults: a consensus definition from a multidisciplinary Delphi panel.
      The role of the SDM may also vary based on legal and cultural factors. When ACP is performed in advance of major medical events, patients experience less distress, experience improved satisfaction with care, and are more likely to receive care consistent with their wishes.
      • Detering K.M.
      • Hancock A.D.
      • Reade M.C.
      • Silvester W.
      The impact of advance care planning on end of life care in elderly patients: randomised controlled trial.
      • Kirchhoff K.T.
      • Hammes B.J.
      • Kehl K.A.
      • Briggs L.A.
      • Brown R.L.
      Effect of a disease-specific advance care planning intervention on end-of-life care.
      • Silveira M.J.
      • Kim S.Y.H.
      • Langa K.M.
      Advance directives and outcomes of surrogate decision making before death.
      • Teno J.M.
      • Gozalo P.L.
      • Bynum J.P.W.
      • et al.
      Change in end-of-life care for Medicare beneficiaries: site of death, place of care, and health care transitions in 2000, 2005, and 2009.
      • Casarett D.
      • Karlawish J.
      • Morales K.
      • Crowley R.
      • Mirsch T.
      • Asch D.A.
      Improving the use of hospice services in nursing homes: a randomized controlled trial.
      • Wright A.A.
      • Keating N.L.
      • Ayanian J.Z.
      • et al.
      Family perspectives on aggressive cancer care near the end of life.
      • Morrison R.S.
      • Chichin E.
      • Carter J.
      • Burack O.
      • Lantz M.
      • Meier D.E.
      The effect of a social work intervention to enhance advance care planning documentation in the nursing home.
      Similarly, SDMs of patients who have done ACP are less likely to experience distress from medical decision making and have improved satisfaction with care.
      • Detering K.M.
      • Hancock A.D.
      • Reade M.C.
      • Silvester W.
      The impact of advance care planning on end of life care in elderly patients: randomised controlled trial.
      • Kramer B.J.
      • Boelk A.Z.
      • Auer C.
      Family conflict at the end of life: lessons learned in a model program for vulnerable older adults.
      • Kramer B.J.
      • Kavanaugh M.
      • Trentham-Dietz A.
      • Walsh M.
      • Yonker J.A.
      Predictors of family conflict at the end of life: the experience of spouses and adult children of persons with lung cancer.
      Despite these known benefits, however, <30% of Americans and Canadians have completed the ACP process.
      • Rao J.K.
      • Anderson L.A.
      • Lin F.C.
      • Laux J.P.
      Completion of advance directives among U.S. consumers.
      • Resnick H.E.
      • Hickman S.
      • Foster G.L.
      Documentation of advance directives among home health and hospice patients: United States, 2007.
      • Greenberg M.R.
      • Weiner M.
      • Greenberg G.B.
      Risk-reducing legal documents: controlling personal health and financial resources.
      • Howard M.
      • Bernard C.
      • Klein D.
      • et al.
      Older patient engagement in advance care planning in Canadian primary care practices: Results a multisite Survey.
      Finding new ways to engage patients and their families in these important conversations is critical if we hope to increase the number of individuals who have documented advance care plans so that clinicians and families may carry out their wishes. It is equally important to consider the role of SDMs in the ACP process because studies have found that up to 70% of individuals will need some or all their end-of-life decisions to be made by a surrogate.
      • Silveira M.J.
      • Kim S.Y.H.
      • Langa K.M.
      Advance directives and outcomes of surrogate decision making before death.
      As ACP interventions have been developed and tested, however, assessment has focused primarily on patient engagement with ACP, by measuring either advance directive completion rates,
      • Bravo G.
      • Dubois M.-F.
      • Wagneur B.
      Assessing the effectiveness of interventions to promote advance directives among older adults: a systematic review and multi-level analysis.
      • Tamayo-Velazquez M.I.
      • Simon-Lorda P.
      • Villegas-Portero R.
      • et al.
      Interventions to promote the use of advance directives: an overview of systematic reviews.
      or more recently, engagement in the comprehensive ACP process as measured by the ACP engagement survey.
      • Sudore R.L.
      • Heyland D.K.
      • Barnes D.E.
      • Howard M.
      • Fassbender K.
      • Robinson C.A.
      • et al.
      Measuring advance care planning: optimizing the advance care planning engagement survey.
      • Sudore R.L.
      • Stewart A.L.
      • Knight S.J.
      • et al.
      Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.
      Absent, however, is a measure of surrogate engagement in the ACP process.
      To measure patient engagement in ACP, Sudore et al. developed and validated a questionnaire grounded in social cognitive theory and the transtheoretical model of behavior change that has been utilized as an important outcome measure in several studies and interventional trials.
      • Sudore R.L.
      • Heyland D.K.
      • Barnes D.E.
      • Howard M.
      • Fassbender K.
      • Robinson C.A.
      • et al.
      Measuring advance care planning: optimizing the advance care planning engagement survey.
      • Sudore R.L.
      • Stewart A.L.
      • Knight S.J.
      • et al.
      Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.
      • Sudore R.
      • Le G.M.
      • McMahon R.
      • Feuz M.
      • Katen M.
      • Barnes D.E.
      The advance care planning PREPARE study among older Veterans with serious and chronic illness: study protocol for a randomized controlled trial.
      • Radhakrishnan K.
      • Van Scoy L.J.
      • Jillapalli R.
      • Saxena S.
      • Kim M.T.
      Community-based game intervention to improve South Asian Indian Americans' engagement with advanced care planning.
      • Van Scoy L.
      • Reading J.
      • Hopkins M.
      • et al.
      Community Game Day: using an end-of-life conversation game to encourage advance care planning.
      • Zapata C.
      • Wistar E.
      • Horton C.
      • Lum H.
      • Sudore R.
      Using a video-based advance care planning (ACP) website to facilitate group visits for diverse older adults in primary care is feasible and improves ACP engagement (TH307D).
      Although the psychometrics of that questionnaire, called the ACP engagement survey, have been well studied, it has not been adapted for use with the patients' SDMs. The purpose of this study was to adapt the existing 55-item patient-oriented ACP engagement survey for use with SDMs and begin to assess the validity of the adapted survey.

      Methods

      This prospective study was conducted in five phases (Fig. 1).
      Figure thumbnail gr1
      Fig. 1Methods for creating and validating the ACP engagement survey–surrogate decision maker version (17 items). ACP = advance care planning.

      Phase 1: Adapting the Original ACP Engagement Survey for Use With SDMs

      The patient version of the 55-item ACP engagement survey (ACP-55)
      • Sudore R.L.
      • Heyland D.K.
      • Barnes D.E.
      • Howard M.
      • Fassbender K.
      • Robinson C.A.
      • et al.
      Measuring advance care planning: optimizing the advance care planning engagement survey.
      assesses four domains: 1) SDMs; 2) values and quality of life; 3) leeway in surrogate decision making; and 4) asking doctors questions.
      • Sudore R.L.
      • Stewart A.L.
      • Knight S.J.
      • et al.
      Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.
      For each domain, items assess four behavior change constructs arising from the transtheoretical model of behavior change and social cognitive theory: 1) knowledge; 2) contemplation; 3) self-efficacy; and 4) readiness.
      • Sudore R.L.
      • Stewart A.L.
      • Knight S.J.
      • et al.
      Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.
      The ACP-55 has been previously shown to have high reliability (Cronbach alpha 0.97) and construct validity based on both statistically significant associations between survey scores and completion of advance directives as well as correlations between change scores in response to an ACP intervention (r = 0.89, P < 0.001).
      • Sudore R.L.
      • Heyland D.K.
      • Barnes D.E.
      • Howard M.
      • Fassbender K.
      • Robinson C.A.
      • et al.
      Measuring advance care planning: optimizing the advance care planning engagement survey.
      The survey has been used as an outcome in several studies examining the impact of ACP interventions.
      • Radhakrishnan K.
      • Van Scoy L.J.
      • Jillapalli R.
      • Saxena S.
      • Kim M.T.
      Community-based game intervention to improve South Asian Indian Americans' engagement with advanced care planning.
      • Van Scoy L.
      • Reading J.
      • Hopkins M.
      • et al.
      Community Game Day: using an end-of-life conversation game to encourage advance care planning.

      Howard M RC, McKenzie M, Fyles G, et al. Effectiveness of an interactive website to engage patients in advance care planning in outpatient settings: a multicenter, prospective, before-after study. Ann Fam Med. [Under Review].

      The ACP-55 question stems were reworded to apply to SDMs and then examined by an eight-person panel of ACP researchers, clinicians, and laypersons for face validity. Resultantly, eight items were removed and six items were flagged for reevaluation after data collection. The result was a 47-item survey (ACP-SDM-47).

      Phase 2: Administration of the ACP-SDM-47

      After Institutional Review Board approval, the Penn State Survey Research Center recruited a convenience sample of participants identified from medical record databases using the following inclusion criterion: 1) >18 years old; 2) can read and write English; 3) had a family member who has a chronic illness (defined using Iezonni's ICD-9 Criterion of chronic illness)
      • Iezzoni L.I.
      Multiple chronic conditions and disabilities: implications for health services research and data demands.
      ; and 4) were willing to complete and return the survey. The Iezonni chronic illness codes are grouped into eight categories that include the following: cancer with poor prognosis; chronic pulmonary disease; coronary artery disease; congestive heart failure; peripheral vascular disease; severe, chronic liver disease; diabetes with end organ damage; and renal failure. Participants were also recruited from the community around Penn State University Park via flyers, social media postings, and community e-mail lists. Interested participants contacted the survey center and screened by phone to confirm 1) their family member had a qualifying chronic illness (via self-report) and 2) that they did not have mental impairment as measured by a score of >21 on the Montreal Cognitive Assessment (phone version).
      • Wong A.
      • Nyenhuis D.
      • Black S.E.
      • et al.
      Montreal cognitive assessment 5-minute protocol is a brief, valid, reliable, and feasible cognitive screen for telephone administration.
      Eligible participants were sent a paper copy of the ACP-SDM-47, a prepaid return envelope, and a $10.00 stipend. Since this study focused on the feasibility, usability, and acceptability of the adapted questionnaire and because we used a convenience sampling strategy, a power analysis was not conducted.

      Phase 3, Initial Item Reduction

      First, frequency counts for each of the 47 items were examined for those with >10% nonresponses or missing data with the intention of item removal. Second, responses were examined for ceiling effect, defined as any item where the selection of the item at the top of the scale occurred in >80% of instances. Third, an item correlation matrix was constructed to assess for item redundancy, defined as those items with an r > 0.8. Redundant items were flagged for removal. Finally, the study team reevaluated the face validity of all 47 items, informed by the psychometric results.

      Phase 4, Preliminary Exploratory Factor Analysis

      After removing items identified through the aforementioned item reduction process, a preliminary exploratory factor analysis of this small sample was conducted to see if the remaining items grouped into domains. To avoid losing participants with some missing items, before implementing the factor analysis, we used the expectation-maximization algorithm to impute the most likely values for missing items given the values of the participants' nonmissing items and assuming the correlation structure observed between items among all participants.
      We used the common factor model with PROMAX oblique factor rotation to allow for correlation between the underlying factors. The first step in the exploratory factor analysis was to decide how many factors should be retained. When making this decision, we considered the amount of variance explained by the two-, three-, four-, and five-factor solutions as well as the interpretability and parsimony of the resulting rotated factor solutions.
      • Fabrigar L.R.
      • Wegener D.T.
      Exploratory factor analysis.
      Once the number of factors was decided upon and the rotated factor solution was obtained, we examined the factor loading matrix and identified the factors on which each item loaded most strongly. We grouped variables according to which item they loaded most heavily on (>0.5) and labeled the resultant grouping (“domain”) to reflect the contents of its items. Face validity of the item groupings and the strength of the loadings and cross-loadings across domains were considered to determine whether any items should be placed in a domain other than the one that they loaded most heavily on. This was done through consensus among authors, only for situations where the factor loadings of an item were very similar between two domains, and if the item was felt to be more conceptually aligned to the domain with the slightly lower loading.

      Phase 5, Evaluation of Internal Consistency

      Internal reliability of the domains was assessed by Cronbach's alpha. We reassessed Cronbach's alpha after removing one item at a time from each domain to see if any item was reducing the internal consistency of the domain it was placed in.
      All analyses were conducted in SAS (version 9.4; SAS Institute Inc., Cary, NC).

      Results

      Phase 1, Adaptation of the ACP-55 and Review for Face Validity

      During Phase 1, we adapted the original patient version of the 55-item ACP engagement survey by altering the question stems to apply to SDMs and key informants (Appendix 1). For example, the patient-oriented ACP-55 item “How confident are you that today you could ask someone to be your medical decision maker?” was changed to “As of today, how confident are you that you could serve as your loved one's medical decision maker” for the ACP-47-SDM. We opted not to change the wording of the question stems from the original survey because these stems were generated from focus groups including both patients and surrogates and the original survey was vetted extensively by patients.
      • Sudore R.L.
      • Stewart A.L.
      • Knight S.J.
      • et al.
      Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.

      Howard M RC, McKenzie M, Fyles G, et al. Effectiveness of an interactive website to engage patients in advance care planning in outpatient settings: a multicenter, prospective, before-after study. Ann Fam Med. [Under Review].

      Because our goal was to align the survey as closely as possible to the well-validated ACP-55, stems were kept as close to the original as possible.
      After reviewing the stem adaptations, eight items were removed owing to a lack of face validity. For example, one item for the patient-oriented survey asked “how much have you thought about whether or not certain health situations would make your life not worth living?” The adapted version of this question became “how much have you thought about whether or not your loved one has considered certain health situations that would make his/her life not worth living?” This was deemed to lack face validity because the item now examined whether the SDM thought about whether the patient thought about the issues, a somewhat abstract concept that is noncritical to serving in the role of an SDM.

      Phase 2, Survey Administration

      Next, the ACP-47-SDM was administered to 65 participants with a mean age of 51.8 years (SD 13.8) and 81% female. No participants were excluded owing to cognitive dysfunction. Racial data were collected for 49 participants who were 96% Caucasian and 4% African American and were missing for 21 participants due to administrative error.

      Phase 3, Item Reduction

      Table 1 shows the reasons for exclusion for each of the removed items. No items were deleted as a result of missing data, nonresponses, or ceiling effect. We removed 17 items due to redundancy (correlations with r > 0.8; Appendix 2). Two items that were highly correlated (r = 0.744) and therefore deleted owing to their redundancy were as follows: “how much have you thought about the possibility of being asked to make medical decisions for your loved one?” and “how much have you thought about your role as your loved one's medical decision maker?” The remaining items were reviewed for face validity and 13 additional items were deleted. Reasons for deletion based on face validity included items involving two separate ideas within a single question (Item #5), for example, thinking about playing the role of SDM and also thinking about being part of a discussion with others. Seven items were deleted because the group felt that the question inquired about thoughts/behaviors that were not the role of the SDM to initiate (e.g., discussing one's role as an SDM with the patient's doctor; Items # 6, 8, 9, 11, 12, 23) and are also behaviors that the SDM cannot “act” upon unless the patient themselves invited the action. Thus, including these items could negatively impact the SDM's engagement score with regards to behaviors outside their own control. Hence, these items were deleted. Some items had multiple reasons for deletion as noted in Table 1. The resulting survey consisted of 17 remaining items.
      Table 1Reasons for Item Exclusions From the 47-Item ACP Engagement Survey–Surrogate Decision Maker Version (Postsurvey Administration)
      ItemQuestionReason for Exclusion
      1How well informed are you about who can be a medical decision maker?Not excluded
      2How well informed are you about what makes someone a good medical decision maker?Not excluded
      3How well informed are you about the types of decisions that you may have to make for your loved one in the future?Not excluded
      4How much have you thought about your role as your loved one's medical decision maker?Not excluded
      5How much have you thought about being part of a discussion with your loved one's doctors about your role as a medical decision maker for your loved one?Lacked face validity; double-barreled question
      6How much have you thought about being part of a discussion with your loved one's other family and friends about your role as your loved one's medical decision maker?Lacked face validity
      7As of today, how confident are you that you could serve as your loved one's medical decision maker?Not excluded
      8As of today, how confident are you that you could be part of a discussion with your loved one's doctors about your role as your loved one's medical decision maker?Redundant with other items, lacked face validity
      9As of today, how confident are you that you could be part of a discussion with your loved one's other family and friends about your role as your loved one's medical decision maker?Redundant with other items, lacked face validity
      10How ready are you to formally discuss with your loved one your role as their medical decision maker?Not excluded
      11How ready are you to be part of a discussion with your loved one's doctor about your role as your loved one's medical decision maker?Lacked face validity
      12How ready are you to be part of a discussion with your loved one's other family and friends about your role as your loved one's medical decision maker?Redundant with other items, lacked face validity
      13How ready are you to be named a medical decision maker in official papers that are signed by your loved one?Lacked face validity
      14How much have you thought about talking with your loved one about whether or not there are certain health situations that would make your loved one's life not worth living?Not excluded
      15How much have you thought about being part of a discussion with your loved one's doctors about whether or not there are certain health situations that would make your loved one's life not worth living?Not excluded
      16How much have you thought about being part of a discussion with your loved ones other family and friends about whether or not there are certain health situations that would make your loved one's life not worth living?Redundant with other items
      17As of today, how confident are you that you could talk with your loved one about whether or not certain health situations would make their life not worth living?Redundant with other items
      18As of today, how confident are you that you could be part of a discussion with your loved one's doctors about whether or not there are certain health situations that would make your loved one's life not worth living?Redundant with other items
      19As of today, how confident are you that you could talk with your loved one's other family and friends about whether or not there are certain health situations that would make your loved one's life not worth living?Redundant with other items
      20How ready are you to talk with your loved one about whether or not there are certain health situations that would make their life not worth living?Not excluded
      21How ready are you to be part of a discussion with your loved one's doctor about whether or not there are certain health situations that would make your loved one's life not worth living?Not excluded
      22How ready are you to be part of a discussion with your loved one's other family and friends about whether or not there are certain health situations that would make your loved one's life not worth living?Redundant with other items
      23How ready are you help your loved one sign official papers putting his/her wishes in writing about whether or not there are certain health situations that would make your loved one's life not worth living?Lacked face validity
      24How much have you thought about talking with your loved one about the care he/she would want if they were very sick or near the end of life?Not excluded
      25How much have you thought about being part of a discussion with your loved one's doctors about the care your loved one would want if he/she was very sick or near the end of life?Not excluded
      26How much have you thought about being part of a discussion with your loved one's other family and friends about the care your loved one would want if he/she were very sick or near the end of life?Redundant with other items
      27As of today, how confident are you that you could talk with your loved one about the care he/she would want if he/she were very sick or near the end of life?Not excluded
      28As of today, how confident are you that you could be part of a discussion with your loved one's doctors about the care your loved one would want if he/she were very sick or near the end of life?Not excluded
      29As of today, how confident are you that you could be part of a discussion with your loved one's other family and friends about the care your loved one would want if he/she were very sick or near the end of life?Redundant with other items
      30How ready are you to talk to your loved one about the kind of medical care he/she would want if they were very sick or near the end of life?Not excluded
      31How ready are you to be part of a discussion with your loved one's doctor about the kind of medical care your loved one would want if he/she were very sick or near the end of life?Not excluded
      32How ready are you to be part of a discussion with your loved one's other family and friends about the kind of medical care he/she would want if he/she were very sick or near the end of life?Redundant with other items
      33How ready are you to help your loved one sign official papers putting his/her wishes in writing about the kind of medical care he/she would want if he/she very sick or near the end of life?Redundant with other items; lacked face validity
      34How well informed are you about the different amounts of flexibility a person can give their medical decision maker?Lacked face validity
      35How much have you thought about the amount of flexibility you would have as your loved one's medical decision maker?Redundant with other items; lacked face validity
      36How much have you thought about talking with your loved one about the amount of flexibility he/she would want to give you as a medical decision maker?Lacked face validity
      37How much have you thought about being part of a discussion with your loved one's doctor about the amount of flexibility you would have as the medical decision maker?Redundant with other items; lacked face validity
      38How much have you thought about being part of a discussion with your loved one's other family and friends about the amount of flexibility you would have as the medical decision maker?Lacked face validity
      39As of today, how confident are you that you could talk with your loved one about how much flexibility he/she would want to give you as a medical decision maker?Lacked face validity
      40As of today, how confident are you that you could be part of a discussion with your loved one's doctor about how much flexibility your loved one would want to give you as a medical decision maker?Lacked face validity
      41As of today, how confident are you that you could be part of a discussion with your loved one's other family and friends about how much flexibility your loved one would want to give you as a medical decision maker?Lacked face validity
      42How ready are you to talk to your loved one about how much flexibility he/she would want to give you as a medical decision maker?Lacked face validity
      43How ready are you to be part of a discussion with your loved one's doctor about how much flexibility your loved one would want to give you as a medical decision maker?Redundant with other items; lacked face validity
      44How ready are you to be part of a discussion with your loved one's other family and friends about how much flexibility your loved one would want to give you as a medical decision maker?Redundant with other items; lacked face validity
      45How ready are you to help your loved one sign official papers about how much flexibility he/she would want to give you as a medical decision maker?Redundant with other items; lacked face validity
      46How confident are you that today you could ask the right questions of your loved one's doctors to help make good medical decisions for your loved one if your loved one was unable to speak for themselves?Deleted due to typo in question stem
      47How ready are you to ask your loved one's doctor questions to help you make a good medical decision for your loved one if your loved one was unable to speak for themselves?Not excluded
      ACP = advance care planning.

      Phase 4, Preliminary Exploratory Factor Analysis

      Eight (12%) of the 65 participants had missing data for at least one of the 17 items. One participant missed seven items, and seven respondents missed between one and four items. All missing item values were imputed prior to factor analysis so all 65 participants could be included (see methods).
      A one-factor solution explained 78% of the variance in the 17 items, two-factor solution explained 86%, three-factor solution explained 91%, four-factor solution explained 95%, and five-factor solution explained 99%. We reviewed the two-, three-, four-, and five-factor solutions and selected the three-factor solution for its simplicity and interpretability. The three factors were correlated with each other with the three pairwise Pearson's correlation coefficients ranging from 0.59 to 0.63.
      We reviewed the items that loaded onto the three-factor solution and grouped items according to their factor loadings (Table 2). The seven items grouped in Factor 1 were reviewed for face validity by assessing the common features of these items with regards to their theoretical construct (e.g., self-perceived confidence). The identified similarities were that all items assessed either knowledge or self-efficacy about serving as an SDM, and thus the factor was labeled “serving as SDMs.” No changes were made to Factor 1 based on face validity or clinical sensibility. The loadings of Factor 1 ranged from 0.48 to 0.89.
      Table 2Exploratory Analysis of the 17-Item Questionnaire Items
      ItemTheoretical ConstructFactor 1 Serving as SDM DomainFactor 2 Contemplation DomainFactor 3 Readiness DomainCommunality
      The communality is the proportion of the variance of the item accounted for by the three-factor solution. Ideally, the communality would be near 1 and would be caused by a near 1 loading on single factor, but communalities greater than 0.4 or 0.5 are typically considered acceptable.
      1. How well informed are you about who can be a medical decision maker?Knowledge0.89−0.03−0.020.74
      2. How well informed are you about what makes someone a good medical decision maker?Knowledge0.89−0.120.060.75
      3. How well informed are you about the types of decisions that you may have to make for your loved one in the future?Knowledge0.700.090.030.62
      4. How much have you thought about your role as your loved one's medical decision maker?Contemplation0.520.190.120.53
      5. As of today, how confident are you that you could serve as your loved one's medical decision maker?Self-efficacy0.700.060.030.57
      6. As of today, how confident are you that you could talk with your loved one about the care he/she would want if he/she were very sick or near the end of life?Self-efficacy0.480.230.140.55
      7. As of today, how confident are you that you could be part of a discussion with your loved one's doctor about the care your loved one would want if he/she were very sick or near the end of life?Self-efficacy0.510.310.110.67
      8. How much have you thought about talking with your loved one about whether or not there are certain health situations that would make your loved one's life not worth living?Contemplation0.290.240.380.62
      9. How much have you thought about being part of a discussion with your loved one's doctors about whether or not there are certain health situations that would make your loved one's life not worth living?Contemplation0.060.490.300.57
      10. How much have you thought about talking with your loved one about the care he/she would want if they were very sick or near the end of life?Contemplation0.140.620.280.86
      11. How much have you thought about being part of a discussion with your loved one's doctors about the care he/she would want if they were very sick or near the end of life?Contemplation0.051.04−0.121.00
      12. How ready are you to formally discuss with your loved one your role as their medical decision maker?Readiness0.38−0.060.500.56
      13. How ready are you to talk to your loved one about the kind of medical care he/she would want if they were very sick or near the end of life?Readiness−0.040.090.930.92
      14. How ready are you to talk to your loved one about whether or not there are certain health situations that would make your loved one's life not worth living?Readiness0.160.070.720.76
      15. How ready are you to be part of a discussion with your loved one's doctor about whether or not there are certain health situations that would make your loved one's life not worth living?Readiness0.200.220.490.63
      16. How ready are you to be part of a discussion with your loved one's doctor about the kind of medical care your loved one would want if he/she were very sick or near the end of life?Readiness0.000.420.460.63
      17. How ready are you to ask your loved one's doctor questions to help you make a good medical decision for your loved one if your loved one was unable to speak for themselves?Readiness−0.110.470.280.37
      SDM = surrogate decision maker.
      a The communality is the proportion of the variance of the item accounted for by the three-factor solution. Ideally, the communality would be near 1 and would be caused by a near 1 loading on single factor, but communalities greater than 0.4 or 0.5 are typically considered acceptable.
      Factor 2 had five items that were reviewed using similar procedures. It was noted that three of the five items involved contemplation about various aspects of surrogate decision making. One item (Item #8; how much have you thought about talking with your loved one about whether or not there are certain health situations that would make your loved one's life not worth living?) that was related to contemplation loaded weakly into the third factor (0.38) with a loading of 0.24 in Factor 2. We decided to move this item into Factor 2 based on face validity because the other items in Factor 2 all assessed contemplation. It was also noted that one item (Item #17; how ready are you to ask your loved one's doctor questions to help you make a good medical decision for your loved one if your loved one was unable to speak for themselves?) loaded into Factor 2 with a value of 0.47 but had higher face validity in Factor 3 and was therefore moved to Factor 3. Factor 2 was named “Contemplation.” The resulting Factor 2 had factor loadings ranging from 0.24 to 1.04.
      After moving Item #17 to Factor 3, that factor contained six items with a loading range of 0.28 to 0.93. All items in Factor 3 were related to readiness to serve in the role of SDM, and the factor was thus named “Readiness.”
      The resultant survey consists of 17 items with three factors (“domains”) (see Appendix 3 for final survey): 1) serving as SDM (seven items), 2) contemplation (four items), and 3) readiness (six items).

      Phase 5, Evaluation of Internal Consistency

      The Cronbach's alpha for Factors 1, 2, and 3 were 0.91, 0.91, and 0.90, respectively, which is generally considered an indication of excellent internal consistency. In no cases did the Cronbach's alpha significantly increase when an item was removed.

      Discussion

      It is increasingly recognized that ACP is a complex process involving multiple stakeholders in addition to patients themselves, such as family, friends, and clinicians. Although much attention has been paid to the study of how ACP interventions engage patients in the ACP process, less work has focused on engaging the SDMs in ACP, despite their equally important role.
      • Jimenez G.
      • Tan W.S.
      • Virk A.K.
      • Low C.K.
      • Car J.
      • Yan Ho A.H.
      Overview of systematic reviews of advance care planning: summary of evidence and global lessons.
      This may be due, in part, to a lack of validated measures that assess how interventions impact SDMs.
      This study resulted in a questionnaire that measures the extent to which SDMs have engaged in the ACP process. To do so, we adapted a well-validated patient-centered survey that is grounded in behavior change theories and related constructs relevant to ACP (knowledge, contemplation, self-efficacy, and readiness). The result was a 47-item adapted questionnaire that was then shortened into a short, 17-item questionnaire. This questionnaire could be useful for researchers seeking to measure engagement in the ACP process by SDMs and also may provide utility by allowing comparisons between engagement of the patients (using the original survey) with engagement of their SDMs. That said, relevant ACP behaviors for a patient may be slightly different than for a surrogate. For this reason, we did not anticipate that all items of the original 55-item survey would map onto the adapted SDM survey. Furthermore, shortened versions of the original patient-oriented survey (4, 9, 15, and 34 items) have been published and are being validated.
      • Sudore R.L.
      • Heyland D.K.
      • Barnes D.E.
      • Howard M.
      • Fassbender K.
      • Robinson C.A.
      • et al.
      Measuring advance care planning: optimizing the advance care planning engagement survey.
      Although our preliminary exploratory factor analysis is limited by small sample size, it provides useful information it resulted in identification of three domains within the final 17-item questionnaire (The ACP-17-SDM; Appendix 3). Domain 1, “Serving as SDM,” includes seven items that assess knowledge (three items), contemplation (one item), self-efficacy (three items). Domain 2, “Contemplation,” includes four items that assess contemplation (how much surrogates have thought about end-of-life issues).
      The third domain, “Readiness,” includes six items that assess readiness and measure participants' stage of readiness to engage in various ACP behaviors. Responses to the readiness question are based on the transtheoretical model's five stages of change, allowing the respondent's “stage of change” for each of the six ACP behaviors to be assessed in addition to the total domain score. Given the limited precision of the factor loadings due to the small sample size, we recommend each factor be reported as a domain score computed as the unweighted average of the items in that domain. Larger studies are needed to verify the validity of these domains.
      Still, the domain scores may be helpful for interventionalists examining mechanisms by which an intervention engages participants in ACP. For example, an intervention intended to start conversations would be expected to increase participants' contemplation scores with lesser effect on “readiness” scores (which assesses subsequent actions). Ideal ACP interventions will increase participants' overall engagement in the ACP process and effect all of these domains equally, yet the complexity of the ACP process likely makes a single intervention insufficient, and consideration should be given to how ACP interventions effect each of these domains.
      A limitation of this study is the small, homogenous, convenience sample, which could limit the stability and generalizability of the results. Furthermore, using convenience sampling may result in selection bias related to literacy or other factors. Second, we did not collect data on the patients' medical conditions. Third, we have not yet assessed the tool's responsiveness to intervention.
      • Sudore R.L.
      • Heyland D.K.
      • Barnes D.E.
      • Howard M.
      • Fassbender K.
      • Robinson C.A.
      • et al.
      Measuring advance care planning: optimizing the advance care planning engagement survey.
      Fourth, some items had fairly low loadings on the factors they were assigned. Even so, this study introduces a brief, 17-item questionnaire that assesses SDM engagement in ACP and is derived from a well-validated patient-oriented survey. Additional studies with larger sample size and varied demographics, literacy, and cultures are needed to further validate and possibly shorten the questionnaire to mirror work in the patient-oriented version of survey
      • Sudore R.L.
      • Heyland D.K.
      • Barnes D.E.
      • Howard M.
      • Fassbender K.
      • Robinson C.A.
      • et al.
      Measuring advance care planning: optimizing the advance care planning engagement survey.
      and to assess its' responsiveness to intervention.

      Disclosures and Acknowledgments

      The authors would like to acknowledge the Pennsylvania State University Survey Research Center for assistance with participant recruitment and data collection, Jean Reading for assistance with protocol/IRB development and data entry, and the research team of Drs. Michael Green and Benjamin Levi for reviewing early versions of the survey adaptation.
      This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. No authors have any conflicts of interest to disclose.
      Ethical approval: The study was approved by the Penn State Hershey Institutional Review Board; all participants provided consent via implied consent via return of the questionnaire.

      Appendix

      Appendix 1Adaptation of the 55-Item Patient-Oriented ACP Engagement Survey
      ItemOriginal Question (Patient Oriented)Adapted Question (Surrogate Decision Maker Oriented)Group Decision After Phase 1 Review
      1How well informed are you about who can be a medical decision maker?UnchangedContinued testing
      2How well informed are you about what makes someone a good medical decision maker?UnchangedContinued testing
      3How well informed are you about the types of decisions that a medical decision maker may have to make for you in the future?How well informed are you about the types of decisions that you may have to make for your loved one in the future?Continued testing
      4How much have you thought about who your medical decision maker should be?How much have you thought about your role as your loved one's medical decision-maker?Continued testing
      5How much have you thought about asking someone to be your medical decision maker?How much have you thought about the possibility of being asked to make medical decisions for your loved one?Excluded due to a lack of face validity
      6How much have you thought about talking with your doctors about who you want your medical decision maker to be?How much have you thought about being part of a discussion with your loved one's doctors about your role as a medical decision-maker for your loved one?Continued testing
      7How much have you thought about talking with your other family and friends about who you want your medical decision maker to be?How much have you thought about being part of a discussion with your loved one's other family and friends about your role as your loved one's medical decision-maker?Continued testing
      8How confident are you that today you could ask someone to be your medical decision maker?As of today, how confident are you that you could serve as your loved one's medical decision maker?Continued testing
      9How confident are you that today you could talk with your doctors about who you want your medical decision maker to be?As of today, how confident are you that you could be part of a discussion with your loved one's doctors about your role as your loved one's medical decision maker?Continued testing
      10How confident are you that today you could talk with your other family and friends about who you want your medical decision maker to be?As of today, how confident are you that you could be part of a discussion with your loved one's other family and friends about your role as your loved one's medical decision maker?Continued testing
      11How ready are you to formally ask someone to be your medical decision maker?How ready are you to formally discuss with your loved one your role as their medical decision maker?Continued testing
      12How ready are you to talk with your doctor about who you want your medical decision maker to be?How ready are you to be part of a discussion with your loved one's doctor about your role as your loved one's medical decision maker?Continued testing
      13How ready are you to talk to your other family and friends about who you want your medical decision maker to be?How ready are you to be part of a discussion with your loved one's other family and friends about your role as your loved one's medical decision maker?Continued testing
      14How ready are you to sign official papers naming a person or group of people to make medical decisions for you?How ready are you to be named a medical decision maker in official papers that are signed by your loved one?Continued testing
      15How much have you thought about whether or not certain health situations would make your life not worth living?How much have you thought about whether or not your loved one has considered certain health situations that would make his/her life not worth living?Excluded due to a lack of face validity
      16How much have you thought about talking with your decision maker about whether or not certain health situations would make your life not worth living?How much have you thought about talking with your loved one about whether or not there are certain health situations that would make your loved one's life not worth living?Continued testing
      17How much have you thought about talking with your doctors about whether or not certain health situations would make your life not worth living?How much have you thought about being part of a discussion with your loved one's doctors about whether or not there are certain health situations that would make your loved one's life not worth living?Continued testing
      18How much have you thought about talking with your other family and friends about whether or not certain health situations would make your loved one's life not worth living?How much have you thought about being part of a discussion with your loved ones other family and friends about whether or not there are certain health situations that would make your loved one's life not worth living?Continued testing
      19How confident are you that today you could talk with your decision maker about whether or not certain health situations would make your life not worth living?As of today, how confident are you that you could talk with your loved one about whether or not certain health situations would make their life not worth living?Continued testing
      20How confident are you that today you could talk with your doctors about whether or not certain health situations would make your life not worth living?As of today, how confident are you that you could be part of a discussion with your loved one's doctors about whether or not there are certain health situations that would make your loved one's life not worth living?Continued testing
      21How confident are you that today you could talk with your other family and friends about whether or not certain health situations would make your life not worth living?As of today, how confident are you that you could talk with your loved one's other family and friends about whether or not there are certain health situations that would make your loved one's life not worth living?Continued testing
      22How ready are you to decide whether or not certain health situations would make your life not worth living?How ready are you to help your loved one decide whether or not certain health situations would make their life not worth living?Excluded due to a lack of face validity
      23How ready are you to talk to your decision maker about whether or not certain health situations would make your life not worth living?How ready are you to talk with your loved one about whether or not there are certain health situations that would make their life not worth living?Continued testing
      24How ready are you to talk to your doctor about whether or not certain health situations would make your life not worth living?How ready are you to be part of a discussion with your loved one's doctor about whether or not there are certain health situations that would make your loved one's life not worth living?Continued testing
      25How ready are you to talk to your other family and friends about whether or not certain health situations would make your life not worth living?How ready are you to be part of a discussion with your loved one's other family and friends about whether or not there are certain health situations that would make your loved one's life not worth living?Continued testing
      26How ready are you to sign official papers putting your wishes in writing about whether or not certain health situations would make your loved one's life not worth living?How ready are you help your loved one sign official papers putting his/her wishes in writing about whether or not there are certain health situations that would make your loved one's life not worth living?Continued testing
      27How much have you thought about the care you would want if you were very sick or near the end of life?How much have you thought about the care your loved one would want if he/she were very sick or near the end of life?Excluded due to a lack of face validity
      28How much have you thought about talking with your decision maker about the care you would want if you were very sick or near the end of life?How much have you thought about talking with your loved one about the care he/she would want if they were very sick or near the end of life?Continued testing
      29How much have you thought about talking with your doctors about the care you would want if you were very sick or near the end of life?How much have you thought about being part of a discussion with your loved one's doctors about the care your loved one would want if he/she was very sick or near the end of life?Continued testing
      30How much have you thought about talking with your other family and friends about the care you would want if you were very sick or near the end of life?How much have you thought about being part of a discussion with your loved one's other family and friends about the care your loved one would want if he/she were very sick or near the end of life?Continued testing
      31How confident are you that today you could talk with your decision maker about the care you would want if you were very sick or near the end of life?As of today, how confident are you that you could talk with your loved one about the care he/she would want if he/she were very sick or near the end of life?Continued testing
      32How confident are you that today you could talk with your doctors about the care you would want if you were very sick or near the end of life?As of today, how confident are you that you could be part of a discussion with your loved one's doctors about the care your loved one would want if he/she were very sick or near the end of life?Continued testing
      33How confident are you that today you could talk with your other family and friends about the care you would want if you were very sick or near the end of life?As of today, how confident are you that you could be part of a discussion with your loved one's other family and friends about the care your loved one would want if he/she were very sick or near the end of life?Continued testing
      34How ready are you to decide on the medical care you would want if you were very sick or near the end of life?How ready are you to help your loved one decide on the medical care he/she would want if he/she were very sick or near the end of life?Excluded due to a lack of face validity
      35How ready are you to talk to your decision maker about the kind of medical care you would want if you were very sick or near the end of life?How ready are you to talk to your loved one about the kind of medical care he/she would want if they were very sick or near the end of life?Continued testing
      36How ready are you to talk to your doctor about the kind of medical care you would want if you were very sick or near the end of life?How ready are you to be part of a discussion with your loved one's doctor about the kind of medical care your loved one would want if he/she were very sick or near the end of life?Continued testing
      37How ready are you to talk to your other family and friends about the kind of medical care you would want if you were very sick or near the end of life?How ready are you to be part of a discussion with your loved one's other family and friends about the kind of medical care he/she would want if he/she were very sick or near the end of life?Continued testing
      38How ready are you to sign official papers putting your wishes in writing about the kind of medical care you would want if you were very sick or near the end of life?How ready are you to help your loved one sign official papers putting his/her wishes in writing about the kind of medical care he/she would want if he/she very sick or near the end of life?Continued testing
      39How well informed are you about what it means to give a medical decision maker flexibility to make future decisions?UnchangedExcluded due to a lack of face validity
      40How well informed are you about the different amounts of flexibility a person can give their medical decision maker?UnchangedContinued testing
      41How much have you thought about the amount of flexibility you would want to give your medical decision maker?How much have you thought about the amount of flexibility you would have as your loved one's medical decision maker?Continued testing
      42How much have you thought about talking with your decision maker about how much flexibility you want to give a medical decision maker?How much have you thought about talking with your loved one about the amount of flexibility he/she would want to give you as a medical decision maker?Continued testing
      43How much have you thought about talking with your doctor about how much flexibility you want to give your decision maker?How much have you thought about being part of a discussion with your loved one's doctor about the amount of flexibility you would have as the medical decision maker?Continued testing
      44How much have you thought about talking with other friends and family about how much flexibility you want to give your decision maker?How much have you thought about being part of a discussion with your loved one's other family and friends about the amount of flexibility you would have as the medical decision maker?Continued testing
      45How confident are you that today you could talk with your decision maker about how much flexibility you want to give them as a medical decision maker?As of today, how confident are you that you could talk with your loved one about how much flexibility he/she would want to give you as a medical decision maker?Continued testing
      46How confident are you that today you could talk with your doctor about how much flexibility you want to give your medical decision maker?As of today, how confident are you that you could be part of a discussion with your loved one's doctor about how much flexibility your loved one would want to give you as a medical decision maker?Continued testing
      47How confident are you that today you could talk with your other family and friends about how much flexibility you want to give your medical decision maker?As of today, how confident are you that you could be part of a discussion with your loved one's other family and friends about how much flexibility your loved one would want to give you as a medical decision maker?Continued testing
      48How ready are you to talk to your decision maker about how much flexibility you want to give a medical decision maker?How ready are you to talk to your loved one about how much flexibility he/she would want to give you as a medical decision maker?Continued testing
      49How ready are you to talk to your doctor about how much flexibility you want to give your decision maker?How ready are you to be part of a discussion with your loved one's doctor about how much flexibility your loved one would want to give you as a medical decision maker?Continued testing
      50How ready are you to talk to your other family and friends about how much flexibility you want to give your medical decision maker?How ready are you to be part of a discussion with your loved one's other family and friends about how much flexibility your loved one would want to give you as a medical decision maker?Continued testing
      51How ready are you to sign official papers putting your wishes in writing about how much flexibility to give your decision maker?How ready are you to help your loved one sign official papers about how much flexibility he/she would want to give you as a medical decision maker?Continued testing
      52How well informed are you about the types of questions you can ask your doctor that will help you make a good medical decision?How well informed are you about the types of questions you and your loved one can ask his/her doctor that will help them make a good medical decision for your loved one?Excluded due to a lack of face validity
      53How much have you thought about questions you will ask your doctor to help make good medical decisions?How much have you thought about questions you will ask your loved one's doctor to help make good medical decisions for your loved one?Excluded due to a lack of face validity
      54How confident are you that today you could ask the right questions of your doctor to help make good medical decisions?How confident are you that today you could ask the right questions of your loved one's doctors to help make good medical decisions for your loved one if your loved one was unable to speak for themselves?Continued testing (italicized words were accidentally deleted from survey)
      55How ready are you to ask your doctor questions to help you make a good medical decision?How ready are you to ask your loved one's doctor questions to help you make a good medical decision for your loved one if your loved one was unable to speak for themselves?Continued testing
      ACP = advance care planning.
      Appendix 2Item-by-Item Correlation Analysis
      ItemItem 1Item 2Item 3Item 4Item 5Item 6Item 7Item 8Item 9Item 10
      Item 11.0000.7340.6720.5140.4480.5160.5160.3790.4760.500
      Item 20.7341.0000.7150.5630.5760.4950.5490.5260.5820.454
      Item 30.6720.7151.0000.4500.4410.5170.5590.5160.5320.464
      Item 40.5140.5630.4501.0000.7440.5890.4310.3670.4540.539
      Item 50.4480.5760.4410.7441.0000.7350.4290.4400.5080.459
      Item 60.5160.4950.5170.5890.7351.0000.4940.4690.5440.519
      Item 70.5160.5490.5590.4310.4290.4941.0000.8920.7380.516
      Item 80.3790.5260.5160.3670.4400.4690.8921.0000.7920.454
      Item 90.4760.5820.5320.4540.5080.5440.7380.7921.0000.472
      Item 100.5000.4540.4640.5390.4590.5190.5160.4540.4721.000
      Item 110.3860.4230.4620.5900.6080.6020.5040.4940.5370.742
      Item 120.3950.3680.4230.4930.5370.7670.5430.5310.6360.667
      Item 130.5700.4870.5230.5680.5730.5970.4670.3940.5190.713
      Item 140.4750.4780.4450.4870.5130.6110.6110.5440.5660.581
      Item 150.4200.4330.3180.5250.6770.6690.4990.4810.4730.436
      Item 160.3980.4600.4210.4690.6150.7680.5090.4990.5580.445
      Item 170.4080.4770.2140.3060.4530.3620.4420.4010.4460.292
      Item 180.4510.4660.3740.4810.6400.6210.6100.5270.5400.401
      Item 190.3900.4690.3220.3860.5650.5620.5390.5080.6280.327
      Item 200.5040.5360.4930.4200.4460.6440.5430.4920.5160.559
      Item 210.5350.6260.5620.4680.5620.6540.5140.4290.4990.487
      Item 220.4730.5410.4340.4320.5620.7330.4920.4610.5180.475
      Item 230.5520.4780.4450.5120.5530.6100.4500.4060.5120.612
      Item 240.5390.6000.5800.5100.6300.7140.5500.4720.4630.567
      Item 250.4530.4810.4810.5000.6690.6970.5370.4620.5020.353
      Item 260.5070.5250.4040.4680.6420.7970.4770.4670.5650.415
      Item 270.5020.5830.4340.4210.5310.5070.4970.4920.5260.414
      Item 280.5580.5780.5900.5950.7700.7940.6650.6380.7270.554
      Item 290.4210.5250.4730.4610.6870.7190.5930.6270.8030.481
      Item 300.4620.5600.5050.5070.5070.6950.5100.4940.5310.671
      Item 310.5050.5230.5190.5560.5950.7720.4930.4220.4710.591
      Item 320.4870.5570.4890.4720.5980.7560.4410.4480.5160.530
      Item 330.5180.4730.5150.4250.5200.5820.3900.3480.4650.566
      Item 340.5400.4910.5390.4510.5480.5840.5200.4360.3880.351
      Item 350.3640.4320.4110.5120.6600.6880.5170.4000.5050.336
      Item 360.4100.4640.4100.4690.6460.6580.5820.4680.5420.503
      Item 370.3840.3640.3760.4150.5970.6110.5300.4320.4620.470
      Item 380.3560.4500.3360.4060.6260.7000.5180.4750.5170.467
      Item 390.4510.5220.3570.2390.3930.3990.4430.4030.3930.431
      Item 400.4510.5080.4950.4270.6130.6010.6360.5220.5750.411
      Item 410.4190.4880.4460.3200.5480.5650.5950.4840.6470.347
      Item 420.5490.5420.5990.5540.6170.6680.5180.4430.4870.733
      Item 430.4500.4780.4750.4380.5670.6100.4100.3720.4490.462
      Item 440.4950.5550.5280.4840.6190.7020.4470.4210.5060.457
      Item 450.5260.4780.5300.4460.5520.6310.4310.4140.4620.628
      Item 460.3500.5010.4160.3750.4720.5570.6580.6320.6650.439
      Item 470.3950.4700.4930.4170.5390.6210.4290.4610.4970.447
      ItemItem 11Item 12Item 13Item 14Item 15Item 16Item 17Item 18Item 19Item 20
      Item 10.3860.3950.5700.4750.4200.3980.4080.4510.3900.504
      Item 20.4230.3680.4870.4780.4330.4600.4770.4660.4690.536
      Item 30.4620.4230.5230.4450.3180.4210.2140.3740.3220.493
      Item 40.5900.4930.5680.4870.5250.4690.3060.4810.3860.420
      Item 50.6080.5370.5730.5130.6770.6150.4530.6400.5650.446
      Item 60.6020.7670.5970.6110.6690.7680.3620.6210.5620.644
      Item 70.5040.5430.4670.6110.4990.5090.4420.6100.5390.543
      Item 80.4940.5310.3940.5440.4810.4990.4010.5270.5080.492
      Item 90.5370.6360.5190.5660.4730.5580.4460.5400.6280.516
      Item 100.7420.6670.7130.5810.4360.4450.2920.4010.3270.559
      Item 111.0000.7700.6660.5750.5310.4620.1320.3580.2990.486
      Item 120.7701.0000.6170.5950.5720.6710.1490.4140.4500.516
      Item 130.6660.6171.0000.5390.5210.5180.3090.5200.4300.612
      Item 140.5750.5950.5391.0000.7460.7020.5300.7190.6360.744
      Item 150.5310.5720.5210.7461.0000.8060.4670.7880.6340.671
      Item 160.4620.6710.5180.7020.8061.0000.4880.6880.6930.689
      Item 170.1320.1490.3090.5300.4670.4881.0000.6830.6690.579
      Item 180.3580.4140.5200.7190.7880.6880.6831.0000.8740.727
      Item 190.2990.4500.4300.6360.6340.6930.6690.8741.0000.664
      Item 200.4860.5160.6120.7440.6710.6890.5790.7270.6641.000
      Item 210.5910.5170.5770.6650.6430.6230.4330.6690.6370.792
      Item 220.5510.6990.4640.6880.7400.8130.4150.6680.6950.781
      Item 230.5490.5210.8960.5260.6430.5980.4280.6610.5780.722
      Item 240.4890.4890.4990.6780.6540.7050.6220.6350.5150.743
      Item 250.4820.5070.4670.5740.7220.7670.5080.6080.5440.591
      Item 260.4340.6680.4840.5630.7530.8650.3880.6400.6540.578
      Item 270.2890.3260.3700.5230.4200.5170.8480.5590.5220.551
      Item 280.6070.6990.6380.6910.7240.7300.4720.7760.7120.609
      Item 290.5430.7240.5700.6180.5590.7380.4870.6430.7790.569
      Item 300.5900.6200.6160.6690.5650.6680.4980.5540.4940.884
      Item 310.6880.7260.5310.5080.6620.6820.2500.5000.4420.638
      Item 320.6210.7640.5060.5580.6250.7810.3180.5190.5710.659
      Item 330.5240.4840.7980.3560.5240.4940.3010.4770.4010.646
      Item 340.3870.3890.4050.2350.3640.4580.4090.4450.3860.460
      Item 350.4220.4860.4270.4350.5930.6860.3910.5900.5100.526
      Item 360.5280.5760.4440.5350.5570.6510.5160.5350.4820.596
      Item 370.5430.6180.4220.4110.5480.5930.2790.4310.4200.483
      Item 380.5080.6440.4230.5190.6640.7440.3160.5420.5240.523
      Item 390.2570.2340.3050.3750.3160.3500.7460.4780.3830.522
      Item 400.4120.4400.4870.4200.5370.5550.5490.7480.6630.601
      Item 410.3600.4870.4620.4200.4930.5950.5300.6650.6920.526
      Item 420.7470.6740.7470.5790.5500.4990.3210.4530.3110.636
      Item 430.5470.5590.5340.3300.5480.5020.2710.4030.3030.458
      Item 440.5130.5970.5260.3980.5510.6320.3150.4430.4160.531
      Item 450.5850.5280.7600.4940.6170.5520.2980.5720.4430.697
      Item 460.4550.4550.4240.5320.5870.6170.4100.6700.5750.579
      Item 470.6050.5880.4380.3530.5360.4910.2310.3610.2990.508
      ItemItem 20Item 21Item 22Item 23Item 24Item 25Item 26Item 27Item 28Item 29
      Item 10.5040.5350.4730.5520.5390.4530.5070.5020.5580.421
      Item 20.5360.6260.5410.4780.6000.4810.5250.5830.5780.525
      Item 30.4930.5620.4340.4450.5800.4810.4040.4340.5900.473
      Item 40.4200.4680.4320.5120.5100.5000.4680.4210.5950.461
      Item 50.4460.5620.5620.5530.6300.6690.6420.5310.7700.687
      Item 60.6440.6540.7330.6100.7140.6970.7970.5070.7940.719
      Item 70.5430.5140.4920.4500.5500.5370.4770.4970.6650.593
      Item 80.4920.4290.4610.4060.4720.4620.4670.4920.6380.627
      Item 90.5160.4990.5180.5120.4630.5020.5650.5260.7270.803
      Item 100.5590.4870.4750.6120.5670.3530.4150.4140.5540.481
      Item 110.4860.5910.5510.5490.4890.4820.4340.2890.6070.543
      Item 120.5160.5170.6990.5210.4890.5070.6680.3260.6990.724
      Item 130.6120.5770.4640.8960.4990.4670.4840.3700.6380.570
      Item 140.7440.6650.6880.5260.6780.5740.5630.5230.6910.618
      Item 150.6710.6430.7400.6430.6540.7220.7530.4200.7240.559
      Item 160.6890.6230.8130.5980.7050.7670.8650.5170.7300.738
      Item 170.5790.4330.4150.4280.6220.5080.3880.8480.4720.487
      Item 180.7270.6690.6680.6610.6350.6080.6400.5590.7760.643
      Item 190.6640.6370.6950.5780.5150.5440.6540.5220.7120.779
      Item 201.0000.7920.7810.7220.7430.5910.5780.5510.6090.569
      Item 210.7921.0000.8180.6140.6720.5870.6020.4410.6250.596
      Item 220.7810.8181.0000.5220.6330.5910.7830.4280.6350.669
      Item 230.7220.6140.5221.0000.5720.5720.5710.4240.6620.567
      Item 240.7430.6720.6330.5721.0000.8420.6590.7210.6580.546
      Item 250.5910.5870.5910.5720.8421.0000.7510.5780.6690.587
      Item 260.5780.6020.7830.5710.6590.7511.0000.4480.7060.716
      Item 270.5510.4410.4280.4240.7210.5780.4481.0000.6200.596
      Item 280.6090.6250.6350.6620.6580.6690.7060.6201.0000.850
      Item 290.5690.5960.6690.5670.5460.5870.7160.5960.8501.000
      Item 300.8840.7340.7360.6500.7950.5650.5440.6150.6140.606
      Item 310.6380.7410.7850.5610.7320.7060.7130.3660.6340.549
      Item 320.6590.7600.9220.5010.6140.5610.7900.4150.6360.696
      Item 330.6460.6200.4720.8400.5630.5570.4840.3850.5850.503
      Item 340.4600.4580.4430.4290.6050.6480.4860.5520.5310.452
      Item 350.5260.5150.5670.4910.6790.7840.6550.4190.6240.564
      Item 360.5960.5350.5900.4410.7450.7330.5790.5440.6150.572
      Item 370.4830.4440.5650.4160.6030.6750.5630.3140.5490.532
      Item 380.5230.4630.6870.4770.5820.6350.7100.3460.6430.606
      Item 390.5220.3830.3470.3520.6990.4640.3110.7710.4450.398
      Item 400.6010.5970.5200.5690.6580.6270.5630.5600.7290.651
      Item 410.5260.5320.5190.5120.6120.6380.6320.5110.6890.720
      Item 420.6360.6240.5360.6570.6910.5730.4290.4470.6470.518
      Item 430.4580.5700.5490.5280.6200.6230.5660.3900.5090.460
      Item 440.5310.6310.6640.5110.6550.6420.6590.4160.5700.576
      Item 450.6970.5950.5410.8320.6340.5180.5120.3450.6240.495
      Item 460.5790.5640.5910.5440.6150.5930.5890.4310.6430.571
      Item 470.5080.5060.5430.4920.6380.6740.5510.3930.5320.479
      ItemItem 29Item 30Item 31Item 32Item 33Item 34Item 35Item 36Item 37Item 38Item 39
      Item 10.4210.4620.5050.4870.5180.5400.3640.4100.3840.3560.451
      Item 20.5250.5600.5230.5570.4730.4910.4320.4640.3640.4500.522
      Item 30.4730.5050.5190.4890.5150.5390.4110.4100.3760.3360.357
      Item 40.4610.5070.5560.4720.4250.4510.5120.4690.4150.4060.239
      Item 50.6870.5070.5950.5980.5200.5480.6600.6460.5970.6260.393
      Item 60.7190.6950.7720.7560.5820.5840.6880.6580.6110.7000.399
      Item 70.5930.5100.4930.4410.3900.5200.5170.5820.5300.5180.443
      Item 80.6270.4940.4220.4480.3480.4360.4000.4680.4320.4750.403
      Item 90.8030.5310.4710.5160.4650.3880.5050.5420.4620.5170.393
      Item 100.4810.6710.5910.5300.5660.3510.3360.5030.4700.4670.431
      Item 110.5430.5900.6880.6210.5240.3870.4220.5280.5430.5080.257
      Item 120.7240.6200.7260.7640.4840.3890.4860.5760.6180.6440.234
      Item 130.5700.6160.5310.5060.7980.4050.4270.4440.4220.4230.305
      Item 140.6180.6690.5080.5580.3560.2350.4350.5350.4110.5190.375
      Item 150.5590.5650.6620.6250.5240.3640.5930.5570.5480.6640.316
      Item 160.7380.6680.6820.7810.4940.4580.6860.6510.5930.7440.350
      Item 170.4870.4980.2500.3180.3010.4090.3910.5160.2790.3160.746
      Item 180.6430.5540.5000.5190.4770.4450.5900.5350.4310.5420.478
      Item 190.7790.4940.4420.5710.4010.3860.5100.4820.4200.5240.383
      Item 200.5690.8840.6380.6590.6460.4600.5260.5960.4830.5230.522
      Item 210.5960.7340.7410.7600.6200.4580.5150.5350.4440.4630.383
      Item 220.6690.7360.7850.9220.4720.4430.5670.5900.5650.6870.347
      Item 230.5670.6500.5610.5010.8400.4290.4910.4410.4160.4770.352
      Item 240.5460.7950.7320.6140.5630.6050.6790.7450.6030.5820.699
      Item 250.5870.5650.7060.5610.5570.6480.7840.7330.6750.6350.464
      Item 260.7160.5440.7130.7900.4840.4860.6550.5790.5630.7100.311
      Item 270.5960.6150.3660.4150.3850.5520.4190.5440.3140.3460.771
      Item 280.8500.6140.6340.6360.5850.5310.6240.6150.5490.6430.445
      Item 291.0000.6060.5490.6960.5030.4520.5640.5720.5320.6060.398
      Item 300.6061.0000.7400.7310.6450.4860.5520.6520.5480.5550.590
      Item 310.5490.7401.0000.8380.6420.5630.6720.6680.6890.6530.415
      Item 320.6960.7310.8381.0000.5050.4600.5370.5840.5890.6760.332
      Item 330.5030.6450.6420.5051.0000.4990.5310.4730.4840.4150.376
      Item 340.4520.4860.5630.4600.4991.0000.6390.6820.6470.5120.621
      Item 350.5640.5520.6720.5370.5310.6391.0000.8430.7920.7350.418
      Item 360.5720.6520.6680.5840.4730.6820.8431.0000.8590.7980.613
      Item 370.5320.5480.6890.5890.4840.6470.7920.8591.0000.7870.415
      Item 380.6060.5550.6530.6760.4150.5120.7350.7980.7871.0000.350
      Item 390.3980.5900.4150.3320.3760.6210.4180.6130.4150.3501.000
      Item 400.6510.5740.5840.4990.5830.6860.7010.6850.6350.5520.675
      Item 410.7200.4880.5660.5100.5290.5600.6810.6710.6170.5560.635
      Item 420.5180.7490.7580.6000.7820.5060.5570.6560.6490.5260.504
      Item 430.4600.5630.7360.6220.6350.5860.5770.6020.7060.5510.475
      Item 440.5760.6420.8000.7350.6480.5940.6880.6730.6890.6410.471
      Item 450.4950.7050.6450.5660.8110.4830.5040.5350.5550.5770.442
      Item 460.5710.6020.5750.5340.4740.4730.6480.5300.4920.5810.454
      Item 470.4790.6040.7610.6140.6180.5730.5930.5980.7080.5530.466
      ItemItem 40Item 41Item 42Item 43Item 44Item 45Item 46Item 47
      Item 10.4510.4190.5490.4500.4950.5260.3500.395
      Item 20.5080.4880.5420.4780.5550.4780.5010.470
      Item 30.4950.4460.5990.4750.5280.5300.4160.493
      Item 40.4270.3200.5540.4380.4840.4460.3750.417
      Item 50.6130.5480.6170.5670.6190.5520.4720.539
      Item 60.6010.5650.6680.6100.7020.6310.5570.621
      Item 70.6360.5950.5180.4100.4470.4310.6580.429
      Item 80.5220.4840.4430.3720.4210.4140.6320.461
      Item 90.5750.6470.4870.4490.5060.4620.6650.497
      Item 100.4110.3470.7330.4620.4570.6280.4390.447
      Item 110.4120.3600.7470.5470.5130.5850.4550.605
      Item 120.4400.4870.6740.5590.5970.5280.4550.588
      Item 130.4870.4620.7470.5340.5260.7600.4240.438
      Item 140.4200.4200.5790.3300.3980.4940.5320.353
      Item 150.5370.4930.5500.5480.5510.6170.5870.536
      Item 160.5550.5950.4990.5020.6320.5520.6170.491
      Item 170.5490.5300.3210.2710.3150.2980.4100.231
      Item 180.7480.6650.4530.4030.4430.5720.6700.361
      Item 190.6630.6920.3110.3030.4160.4430.5750.299
      Item 200.6010.5260.6360.4580.5310.6970.5790.508
      Item 210.5970.5320.6240.5700.6310.5950.5640.506
      Item 220.5200.5190.5360.5490.6640.5410.5910.543
      Item 230.5690.5120.6570.5280.5110.8320.5440.492
      Item 240.6580.6120.6910.6200.6550.6340.6150.638
      Item 250.6270.6380.5730.6230.6420.5180.5930.674
      Item 260.5630.6320.4290.5660.6590.5120.5890.551
      Item 270.5600.5110.4470.3900.4160.3450.4310.393
      Item 280.7290.6890.6470.5090.5700.6240.6430.532
      Item 290.6510.7200.5180.4600.5760.4950.5710.479
      Item 300.5740.4880.7490.5630.6420.7050.6020.604
      Item 310.5840.5660.7580.7360.8000.6450.5750.761
      Item 320.4990.5100.6000.6220.7350.5660.5340.614
      Item 330.5830.5290.7820.6350.6480.8110.4740.618
      Item 340.6860.5600.5060.5860.5940.4830.4730.573
      Item 350.7010.6810.5570.5770.6880.5040.6480.593
      Item 360.6850.6710.6560.6020.6730.5350.5300.598
      Item 370.6350.6170.6490.7060.6890.5550.4920.708
      Item 380.5520.5560.5260.5510.6410.5770.5810.553
      Item 390.6750.6350.5040.4750.4710.4420.4540.466
      Item 401.0000.9100.5710.6040.6220.5890.6470.582
      Item 410.9101.0000.4940.5860.6440.5200.5650.558
      Item 420.5710.4941.0000.6950.6760.7560.4480.719
      Item 430.6040.5860.6951.0000.8670.7120.4710.875
      Item 440.6220.6440.6760.8671.0000.6820.4960.759
      Item 450.5890.5200.7560.7120.6821.0000.5480.679
      Item 460.6470.5650.4480.4710.4960.5481.0000.519
      Item 470.5820.5580.7190.8750.7590.6790.5191.000
      Appendix 3Final 17-Item ACP Engagement Survey–Surrogate Decision Maker
      ItemTheoretical ConstructDomain (Factor)
      1. How well informed are you about who can be a medical decision maker?KnowledgeServing as surrogate decision-maker
      2. How well informed are you about what makes someone a good medical decision maker?KnowledgeServing as surrogate decision-maker
      3. How well informed are you about the types of decisions that you may have to make for your loved one in the future?KnowledgeServing as surrogate decision-maker
      4. How much have you thought about your role as your loved one's medical decision-maker?ContemplationServing as surrogate decision-maker
      5. As of today, how confident are you that you could serve as your loved one's medical decision maker?Self-efficacyServing as surrogate decision-maker
      6. As of today, how confident are you that you could talk with your loved one about the care he/she would want if he/she were very sick or near the end of life?Self-efficacyServing as surrogate decision-maker
      7. As of today, how confident are you that you could be part of a discussion with your loved one's doctor about the care your loved one would want if he/she were very sick or near the end of life?Self-efficacyServing as surrogate decision-maker
      8. How much have you thought about talking with your loved one about whether or not there are certain health situations that would make your loved one's life not worth living?ContemplationContemplation
      9. How much have you thought about being part of a discussion with your loved one's doctors about whether or not there are certain health situations that would make your loved one's life not worth living?ContemplationContemplation
      10. How much have you thought about talking with your loved one about the care he/she would want if they were very sick or near the end of life?ContemplationContemplation
      11. How much have you thought about being part of a discussion with your loved one's doctors about the care he/she would want if they were very sick or near the end of life?ContemplationContemplation
      12. How ready are you to formally discuss with your loved one your role as their medical decision maker?ReadinessReadiness
      13. How ready are you to talk to your loved one about the kind of medical care he/she would want if they were very sick or near the end of life?ReadinessReadiness
      14. How ready are you to talk to your loved one about whether or not there are certain health situations that would make your loved one's life not worth living?ReadinessReadiness
      15. How ready are you to be part of a discussion with your loved one's doctor about whether or not there are certain health situations that would make your loved one's life not worth living?ReadinessReadiness
      16. How ready are you to be part of a discussion with your loved one's doctor about the kind of medical care your loved one would want if he/she were very sick or near the end of life?ReadinessReadiness
      17. How ready are you to ask your loved one's doctor questions to help you make a good medical decision for your loved one if your loved one was unable to speak for themselves?ReadinessReadiness
      ACP = advance care planning.

      References

        • Sudore R.L.
        • Lum H.D.
        • You J.J.
        • et al.
        Defining advance care planning for adults: a consensus definition from a multidisciplinary Delphi panel.
        J Pain Symptom Manage. 2017; 53: 821-832.e1
        • Detering K.M.
        • Hancock A.D.
        • Reade M.C.
        • Silvester W.
        The impact of advance care planning on end of life care in elderly patients: randomised controlled trial.
        BMJ. 2010; 340: 1345
        • Kirchhoff K.T.
        • Hammes B.J.
        • Kehl K.A.
        • Briggs L.A.
        • Brown R.L.
        Effect of a disease-specific advance care planning intervention on end-of-life care.
        J Am Geriatr Soc. 2012; 60: 946-950
        • Silveira M.J.
        • Kim S.Y.H.
        • Langa K.M.
        Advance directives and outcomes of surrogate decision making before death.
        N Engl J Med. 2010; 362: 1211-1218
        • Teno J.M.
        • Gozalo P.L.
        • Bynum J.P.W.
        • et al.
        Change in end-of-life care for Medicare beneficiaries: site of death, place of care, and health care transitions in 2000, 2005, and 2009.
        JAMA. 2013; 309: 470-477
        • Casarett D.
        • Karlawish J.
        • Morales K.
        • Crowley R.
        • Mirsch T.
        • Asch D.A.
        Improving the use of hospice services in nursing homes: a randomized controlled trial.
        JAMA. 2005; 294: 211-217
        • Wright A.A.
        • Keating N.L.
        • Ayanian J.Z.
        • et al.
        Family perspectives on aggressive cancer care near the end of life.
        JAMA. 2016; 315: 284-292
        • Morrison R.S.
        • Chichin E.
        • Carter J.
        • Burack O.
        • Lantz M.
        • Meier D.E.
        The effect of a social work intervention to enhance advance care planning documentation in the nursing home.
        J Am Geriatr Soc. 2005; 53: 290-294
        • Kramer B.J.
        • Boelk A.Z.
        • Auer C.
        Family conflict at the end of life: lessons learned in a model program for vulnerable older adults.
        J Palliat Med. 2006; 9: 791-801
        • Kramer B.J.
        • Kavanaugh M.
        • Trentham-Dietz A.
        • Walsh M.
        • Yonker J.A.
        Predictors of family conflict at the end of life: the experience of spouses and adult children of persons with lung cancer.
        Gerontologist. 2010; 50: 215-225
        • Rao J.K.
        • Anderson L.A.
        • Lin F.C.
        • Laux J.P.
        Completion of advance directives among U.S. consumers.
        Am J Prev Med. 2014; 46: 65-70
        • Resnick H.E.
        • Hickman S.
        • Foster G.L.
        Documentation of advance directives among home health and hospice patients: United States, 2007.
        Am J Hosp Palliat Care. 2012; 29: 26-35
        • Greenberg M.R.
        • Weiner M.
        • Greenberg G.B.
        Risk-reducing legal documents: controlling personal health and financial resources.
        Risk Anal. 2009; 29: 1578-1587
        • Howard M.
        • Bernard C.
        • Klein D.
        • et al.
        Older patient engagement in advance care planning in Canadian primary care practices: Results a multisite Survey.
        Can Fam Physician. 2018; 64: 371-377
        • Bravo G.
        • Dubois M.-F.
        • Wagneur B.
        Assessing the effectiveness of interventions to promote advance directives among older adults: a systematic review and multi-level analysis.
        Soc Sci Med. 2008; 67: 1122-1132
        • Tamayo-Velazquez M.I.
        • Simon-Lorda P.
        • Villegas-Portero R.
        • et al.
        Interventions to promote the use of advance directives: an overview of systematic reviews.
        Patient Educ Couns. 2010; 80: 10-20
        • Sudore R.L.
        • Heyland D.K.
        • Barnes D.E.
        • Howard M.
        • Fassbender K.
        • Robinson C.A.
        • et al.
        Measuring advance care planning: optimizing the advance care planning engagement survey.
        J Pain Symptom Manage. 2017; 53: 669-681.e8
        • Sudore R.L.
        • Stewart A.L.
        • Knight S.J.
        • et al.
        Development and validation of a questionnaire to detect behavior change in multiple advance care planning behaviors.
        PLoS One. 2013; 8: e72465
        • Sudore R.
        • Le G.M.
        • McMahon R.
        • Feuz M.
        • Katen M.
        • Barnes D.E.
        The advance care planning PREPARE study among older Veterans with serious and chronic illness: study protocol for a randomized controlled trial.
        Trials. 2015; 16: 570
        • Radhakrishnan K.
        • Van Scoy L.J.
        • Jillapalli R.
        • Saxena S.
        • Kim M.T.
        Community-based game intervention to improve South Asian Indian Americans' engagement with advanced care planning.
        Ethn Health. 2017; : 1-19https://doi.org/10.1080/13557858.2017.1357068
        • Van Scoy L.
        • Reading J.
        • Hopkins M.
        • et al.
        Community Game Day: using an end-of-life conversation game to encourage advance care planning.
        J Pain Symptom Manage. 2017; 54: 680-6891
        • Zapata C.
        • Wistar E.
        • Horton C.
        • Lum H.
        • Sudore R.
        Using a video-based advance care planning (ACP) website to facilitate group visits for diverse older adults in primary care is feasible and improves ACP engagement (TH307D).
        J Pain Symptom Manage. 2017; 53: 318-319
      1. Howard M RC, McKenzie M, Fyles G, et al. Effectiveness of an interactive website to engage patients in advance care planning in outpatient settings: a multicenter, prospective, before-after study. Ann Fam Med. [Under Review].

        • Iezzoni L.I.
        Multiple chronic conditions and disabilities: implications for health services research and data demands.
        Health Serv Res. 2010; 45: 1523-1540
        • Wong A.
        • Nyenhuis D.
        • Black S.E.
        • et al.
        Montreal cognitive assessment 5-minute protocol is a brief, valid, reliable, and feasible cognitive screen for telephone administration.
        Stroke. 2015; 46: 1059-1064
        • Fabrigar L.R.
        • Wegener D.T.
        Exploratory factor analysis.
        Oxford University Press, New York2011
        • Jimenez G.
        • Tan W.S.
        • Virk A.K.
        • Low C.K.
        • Car J.
        • Yan Ho A.H.
        Overview of systematic reviews of advance care planning: summary of evidence and global lessons.
        J Pain Symptom Manage. 2018; 56: 436-459