Advertisement

Development and Psychometric Properties of a Survey to Assess Barriers to Implementing Advance Care Planning in Primary Care

Open ArchivePublished:August 09, 2017DOI:https://doi.org/10.1016/j.jpainsymman.2017.08.003

      Abstract

      Context

      Valid and reliable measurement of barriers to advance care planning (ACP) in health care settings can inform the design of robust interventions.

      Objective

      This article describes the development and psychometric evaluation of an instrument to measure the presence and magnitude of perceived barriers to ACP discussion with patients from the perspective of family physicians.

      Methods

      A questionnaire was designed through literature review and expert input, asking family physicians to rate the importance of barriers (0 = not at all a barrier and 6 = an extreme amount) to ACP discussions with patients and administered to 117 physicians. Floor effects and missing data patterns were examined. Item-by-item correlations were examined using Pearson correlation. Exploratory factor analysis was conducted (iterated principle factor analysis with oblique rotation), internal consistency (Cronbach's alpha) overall and within factors was calculated, and construct validity was evaluated by calculating three correlations with related questions that were specified a priori.

      Results

      The questionnaire included 31 questions in three domains relating to the clinician, patient/family and system or external factors. No items were removed due to missing data, floor effects, or high correlation with another item. A solution of three factors accounted for 71% of variance. One item was removed because it did not load strongly on any factor. All other items except one remained in the original domain in the questionnaire. Cronbach's alpha for the three factors ranged from 0.84 to 0.90. Two of three a priori correlations with related questions were statistically significant.

      Conclusion

      This questionnaire to assess barriers to ACP discussion from the perspective of family physicians demonstrates preliminary evidence of reliability and validity.

      Key Words

      Background

      Advance care planning (ACP) is a communication process wherein people plan for a time when they cannot make decisions for themselves. It includes reflection, deliberation, and determination of a person's values and wishes or preferences for treatments at the end of life and communication between an individual and his or her loved ones, future substitute decision makers, and health care providers about these values and wishes.
      • Sinuff T.
      • Dodek P.
      • You J.J.
      • et al.
      Improving end-of-life communication and decision making: the development of conceptual framework and quality Indicators.
      When people have engaged in ACP, they are more likely to report higher quality of life near end of life, rates of depression and anxiety among bereaved family members are likely to be lower,
      • Detering K.M.
      • Hancock A.D.
      • Reade M.C.
      • Silvester W.
      The impact of advance care planning on end of life care in elderly patients: randomised controlled trial.
      and there are significantly lower health care costs.
      • Zhang B.
      • Wright A.A.
      • Huskamp H.A.
      • et al.
      Health care costs in the last Week of life.
      Family practice is the setting where longitudinal relationship-centered care across the life cycle is provided for most patients.
      The College of Family Physicians of Canada
      A vision for canada: Family practice - the patient's medical home. Position paper: College of family physicians of canada.
      ACP discussions initiated in the primary care setting could ensure that patients and families are better prepared to make future in-the-moment decisions they may face elsewhere in the health care system. Patients have an expectation that their family physician will initiate such discussions.
      • Johnston S.C.
      • Pfeifer M.P.
      • McNutt R.
      The discussion about advance directives. Patient and physician opinions regarding when and how it should be conducted. End of Life Study Group.
      • Robinson C.
      • Kolesar S.
      • Boyko M.
      • Berkowitz J.
      • Calam B.
      • Collins M.
      Awareness of do-not-resuscitate orders: what do patients know and want?.
      • De Vleminck A.
      • Houttekier D.
      • Pardon K.
      • et al.
      Barriers and facilitators for general practitioners to engage in advance care planning: a systematic review.
      In a 2014 systematic review of the effectiveness of ACP interventions, only 10 of 113 included studies were conducted in community or outpatient settings, the remainder being conducted in institutional settings such as hospital.
      • Brinkman-Stoppelenburg A.
      • Rietjens J.A.C.
      • van der Heide A.
      The effects of advance care planning on end-of-life care: a systematic review.
      It is recognized that ACP discussions should happen in the community and in primary care,
      • Johnston S.C.
      • Pfeifer M.P.
      • McNutt R.
      The discussion about advance directives. Patient and physician opinions regarding when and how it should be conducted. End of Life Study Group.
      Canadian Hospice Palliative Care Association
      Advance care planning in Canada: National Framework and Implementation Task Group.
      • Howard M.
      • Bernard C.
      • Tan A.
      • Slaven M.
      • Klein D.
      • Heyland D.K.
      Advance care planning: let's start sooner.
      however, such discussions occur infrequently between patients and health care providers in primary care.
      • Robinson C.
      • Kolesar S.
      • Boyko M.
      • Berkowitz J.
      • Calam B.
      • Collins M.
      Awareness of do-not-resuscitate orders: what do patients know and want?.
      • Musa I.
      • Seymour J.
      • Narayanasamy M.J.
      • Wada T.
      • Conroy S.
      A survey of older peoples' attitudes towards advance care planning.
      • O'Sullivan R.
      • Mailo K.
      • Angeles R.
      • Agarwal G.
      Advance directives: survey of primary care patients.
      Approximately half of Canadian adults have engaged in some type of end-of-life discussion with family members or friends,
      • Teixeira A.A.
      • Hanvey L.
      • Tayler C.
      • Barwich D.
      • Baxter S.
      • Heyland D.K.
      What do Canadians think of advanced care planning? Findings from an online opinion poll.
      however across numerous countries, estimates of occurrence of patient-physician discussions range from 9% in the general adult population to less than 50% among patients in the last three months of life.
      • Musa I.
      • Seymour J.
      • Narayanasamy M.J.
      • Wada T.
      • Conroy S.
      A survey of older peoples' attitudes towards advance care planning.
      • Teixeira A.A.
      • Hanvey L.
      • Tayler C.
      • Barwich D.
      • Baxter S.
      • Heyland D.K.
      What do Canadians think of advanced care planning? Findings from an online opinion poll.
      • Evans N.
      • Pasman H.R.
      • Vega Alonso T.
      • et al.
      End-of-life decisions: a cross-national study of treatment preference discussions and surrogate decision-maker appointments.
      To increase the quantity and quality of ACP in primary care, interventions guided by knowledge of the specific barriers to and facilitators of ACP in this setting are needed. Reducing barriers that impede best practices is optimal for improving health care delivery.
      • Cochrane L.J.
      • Olson C.A.
      • Murray S.
      • Dupuis M.
      • Tooman T.
      • Hayes S.
      Gaps between knowing and doing: understanding and assessing the barriers to optimal health care.
      To adequately understand such barriers, valid, reliable assessment methods are needed.
      • Harrison M.B.
      • Legare F.
      • Graham I.D.
      • Fervers B.
      Adapting clinical practice guidelines to local context and assessing barriers to their use.
      A systematic review of studies that examined perceived barriers of ACP for general practitioners rated the quality of studies but did not report information on the rigor of development of the questionnaire instruments used in the studies, and the magnitude of perceived barriers could not be reported due to the variability in methods across studies.
      • De Vleminck A.
      • Houttekier D.
      • Pardon K.
      • et al.
      Barriers and facilitators for general practitioners to engage in advance care planning: a systematic review.
      Surveys to measure barriers that will inform the design of robust interventions should have reasonable psychometric properties.
      • Chaudoir S.R.
      • Dugan A.G.
      • Barr C.H.
      • et al.
      Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures.
      The objective of this report is to describe the development and psychometric evaluation of a questionnaire instrument to measure the presence and magnitude of perceived barriers to ACP discussion with patients from the perspective of primary care physicians.

      Methods

      Questionnaire Development

      The purpose of the questionnaire was to identify and quantify the barriers to having ACP discussions in primary care as perceived by family physicians. We followed the structure of a previously published questionnaire on barriers to goals of care discussions that was administered to hospital physicians, residents, and nurses, using the same formatting and instructions for completion, the same response scales and conceptual grouping of categories of barriers, and a similar open-ended question asking about what would help enable discussions. Details of the development and face and content validity of the hospital setting questionnaires have been described previously.
      • You J.J.
      • Downar J.
      • Fowler R.A.
      • et al.
      Barriers to goals of care discussions with seriously ill hospitalized patients and their families: a multicenter survey of clinicians.
      We provided a preamble to the primary care questionnaire to indicate it was asking primary care providers about ACP in their practice. The first page provided a definition of ACP based on a published framework of end-of-life (EOL) communication and decision-making as deliberation and determination of a person's values and wishes or preferences for treatments at the end of life, communication among an individual, his or her loved ones and future substitute decision maker(s), communication among an individual and his or her health care provider(s), and documentation of wishes.
      • Sinuff T.
      • Dodek P.
      • You J.J.
      • et al.
      Improving end-of-life communication and decision making: the development of conceptual framework and quality Indicators.

      Item Generation

      To generate the barriers items for the first draft, we used items from the acute care version that were of relevant to any setting and health care provider (e.g., lack of time and lack of training). Next, we consulted a systematic review of 15 studies of barriers to doing ACP as reported by primary care physicians.
      • De Vleminck A.
      • Houttekier D.
      • Pardon K.
      • et al.
      Barriers and facilitators for general practitioners to engage in advance care planning: a systematic review.
      We reviewed the tabulated barriers in that review, and two authors (M.H. and D.K.H.) generated items to correspond to them. We grouped the items into three areas relating to the clinician, patient/family, and system or external factors, with a separate heading for each group of questions.
      With the questionnaire first draft, we conducted two focus groups; one with researchers from our study team (family physicians, critical care physicians, a geriatrician, and palliative care physicians) conducted first, and a second with family physicians in a group practice in the community to further refine items to assure content validity and wording clarity. Clinicians taking part in the focus groups completed the questionnaires themselves in order to stimulate discussion. Two of the authors (M.H. and D.K.H.) facilitated the groups, asking whether items were clearly worded and how to revise if unclear, whether there were redundant or irrelevant items in the context of primary care that should be removed, and whether there were items that should be added. A formal qualitative analysis of the focus groups was not done. The authors recorded notes on the feedback on items and wording.

      Survey Administration

      Details of the questionnaire administration have been described previously.

      Howard M, Bernard C, Klein D, et al. Barriers to and enablers of advance care planning with patients in primary care: a survey of healthcare providers. Can Fam Physician. In press.

      In brief, we conducted a self-administered survey of health care professionals in primary care in the provinces of Ontario, Alberta, and British Columbia from November 2014 to June 2015. We recruited primary care clinicians in several ways: some were taking part in our larger study on ACP
      • Howard M.
      • Bernard C.
      • Tan A.
      • Slaven M.
      • Klein D.
      • Heyland D.K.
      Advance care planning: let's start sooner.
      and some were recruited through relevant organizations, events, or rounds. In addition, family physician professional organizations included the survey invitation in e-mail newsletters during late 2014. There were no specific eligibility criteria but the introductory letter and preamble noted that the questionnaire was intended for physicians working in a primary care setting. For clinicians who indicated an interest, we used an established method for questionnaire distribution to maximize response rates, which included sending an invitation letter with the questionnaire, and sending up to two reminders, including resending the unique link for nonresponders.
      • Schaefer D.R.
      • Dillman D.A.
      Development of a standard e-mail Methodology: results of an Experiment.
      • Dillman D.
      Mail and internet surveys: The tailored design method.
      Ethical approval for the study was obtained from the Research Ethics Board of each participating institution (Hamilton Integrated Research Ethics Board, University of Alberta Health Research Ethics Board, and University of British Columbia Office of Research Ethics). As approved by the boards, the survey included information about participating, and consent to participate was implied by completing the survey.

      Statistical Analysis

      The goals of the analyses were to 1) systematically reduce or revise items by identifying those items that were potentially noninformative, redundant, or frequently unanswered or responded to as “do not know”, 2) determine whether the items correlated in a way that suggested they were reflecting underlying latent constructs (domains), and 3) examine the construct validity and internal consistency reliability of the final domains.

      Missing Data, Floor Effects, and Item-by-Item Correlation

      First, we conducted a descriptive analysis to examine the extent of missing data and “do not know” responses to identify items that might require removal or rewording. A priori, we planned to consider eliminating items with a high proportion of missing data or “do not know” responses (10% or more for any item). The standard frequency distributions of responses to each item in the questionnaire were then examined for floor effects. That is, items to which 80% or more of respondents endorsed the lowest rating of barrier (“not at all a barrier”) were considered a priori for removal because there would be little ability for an intervention to positively influence the rating of barrier.
      • Streiner D.L.
      • Norman G.R.
      Health Measurement Scales: A practical guide to their development and use.
      Next, we calculated item-by-item Pearson correlation coefficients, considering a correlation of 0.8 or greater to be suggestive of potentially redundant items, and if so, one of these items would be flagged for potential removal.

      Exploratory Factor Analysis

      We employed exploratory factor analysis in an attempt to identify the underlying structure of the barriers' questionnaire items. When selecting the number of factors, we considered the proportion of total variance explained by the factors and the interpretability of the factor solutions with preference for parsimony. We estimated the common factor model by iterated principle axis factor analysis with an oblique Promax rotation. Items for which the highest factor loading in any domain was 0.4 or lower were considered for removal. We considered an item to be included in the domain in which it had the highest loading as long as the highest loading was greater than 0.4.

      Internal Consistency

      Following factor analysis to establish constructs, Cronbach's alpha was used to measure the internal consistency (reliability) of the items in each domain and the correlation between the individual items and the overall domain total. This step was done to determine whether internal consistency of the domains could be improved by removal of any items and to measure the internal consistency of the resulting scales.

      Construct Validity

      To test construct validity, we chose three a priori hypotheses regarding the relationship between rating of magnitude of barriers and current practice and willingness to do ACP, expecting that the barriers' statement ratings would be related to ratings of willingness and current practice. Three family physician co-authors (C.B., A.T., and D.K.) independently chose three pairs of variables they felt would have the strongest correlations, based on their clinical knowledge. Another author (M.H.) reviewed the choices and selected three hypotheses that were endorsed by two or more of the co-authors: 1) rating of barrier “patients think ACP is not relevant to them because they are too healthy right now” and “willingness to initiate ACP with patient”, 2) rating of barrier “my difficulties dealing with uncertainty of prognosis for patients with chronic illness” and “willingness to be an ACP decision coach for patient”, and 3) rating of barrier “insufficient time during scheduled appointments to deal with this topic” current practice regarding “initiate ACP with patients.” Hypotheses were tested using Pearson correlation coefficient. We considered the hypotheses confirmed if the Pearson correlation was less than −0.25 and the one-sided P-value against the null hypothesis of no correlation was <0.025. All analyses were performed using SAS, version 9.4 (SAS Institute, Inc. Cary, NC).

      Results

      Based on feedback from the focus groups, all items in the section on barriers related to the physician were reworded with “My lack of knowledge…My difficulty … My belief that…” etc. For example, the draft item “difficulties with advising patients in expressing their wishes” was changed to “My lack of knowledge about how to elicit values, beliefs, and preferences related to end-of-life care.” As a result of the focus groups, we also added suggested items relating to challenges involving family members, lack of ready access to forms and resources in the family practice, and lack of access to other health care professionals in the family practice (e.g., nurses). We also clarified an item relating to patient understanding of why the physician was raising the topic. Other changes to the items related to wording changes from the goals of care discussion context to the topic of ACP and removal of items specific to the hospital setting such as “lack of appropriate location to maintain patient confidentiality.” The revised questionnaire was reviewed for content and clarity by the research team.
      The final questionnaire asked physicians to rate the importance of 31 barriers to ACP discussions in their practice over the past month. Items were grouped conceptually into three domains: there were 14 barriers related to the doctor, nine barriers related to characteristics of patient, and eight barriers related to health care system or external factors. Responses were on a seven-point scale: 0 = not at all; 1 = very little; 2 = little; 3 = a moderate amount; 4 = a lot; 5 = a great deal; and 6 = an extreme amount. Questions were also asked (on seven-point response scales of “not at all” to “very” or “all the time”) on current engagement, confidence, and willingness to engage in each aspect of ACP: initiating discussions, exchanging information, being a decision coach, participation in finalization of the ACP plan, helping patients communicate ACP with family, and communicate with other health care professionals (18 questions). Demographic and practice characteristics of the respondents were also queried (13 questions).
      Of 162 physicians surveyed, 117 (72%) responded. We excluded one physician who only answered the first 14 questions of the entire survey. The remaining 116 physicians (72%) responded to all questions of the Barriers to ACP Discussions questionnaire and are included in this analysis. Eleven physicians responded “Do not know” to at least one of the barrier items leaving data from 105 used in the factor and reliability analysis. The physician characteristics are based on the 110 physicians who completed the personal demographics section at the end of the survey. Table 1 shows the characteristics of respondents.
      Table 1Demographic Characteristics of the Survey Respondents (n = 110)
      CharacteristicsN (%)
      Age yrs, mean ± SD47.6 ± 11.9
      Sex
       Female54 (49)
      Ethnicity
       Caucasian/white85 (77)
      Years in practice, mean ± SD19.2 ± 12.8
      Self-rated skills in advance care planning discussions
       Limited7 (6)
       Fair11 (10)
       Average52 (47)
       Very good28 (25)
       Expert12 (11)
      Priority for learning to do advance care planning
       Low priority (not of interest or already mastered) 16 (5)
       27 (6)
       332 (29)
       446 (42)
       High priority (#1 on personal learning agenda) 519 (17)
      Have undertaken extratraining or certification in palliative care24 (22)

      Missing Data, Floor and Ceiling Effects, and Item-by-Item Correlation

      Table 2 shows the distribution of the item responses. There were no items for which 10% or more of respondents had missing data or answered “do not know”; therefore, we did not remove items on this basis. There were no floor effects. The largest item-by-item Pearson correlation was 0.67, suggesting that no items should be removed due to redundancy.
      Table 2Descriptive Results of Questionnaire Items (n = 116)
      ItemNot at all

      n (%)
      Very Little

      n (%)
      A Little

      n (%)
      A Moderate Amount

      n (%)
      A Lot

      n (%)
      A Great Deal

      n (%)
      An Extreme Amount

      n (%)
      Mean, SDMedianInterquartile RangeRange
      Barriers related to the physician
      1a) My lack of knowledge about ACP and its relationship to advance directives and goals of care discussions27 (23)29 (25)35 (30)16 (14)6 (5)3 (3)0 (0)1.6, 1.321–20–5
      1b) My lack of knowledge about the legal status of ACP documents in the province*24 (21)29 (25)30 (26)18 (16)9 (8)5 (4)1 (1)1.8, 1.421–30–6
      1c) My lack of knowledge about how to elicit values, beliefs, and preferences related to end-of-life care24 (21)35 (30)39 (34)12 (10)4 (3)2 (2)0 (0)1.5, 1.111–20–5
      1d) My difficulties in dealing with patients changing preferences for medical treatments at the end of life27 (23)29 (25)37 (32)17 (15)4 (3)2 (2)0 (0)1.6, 1.221–20–5
      1e) My difficulties in dealing with uncertainty of prognosis for patients with chronic illness19 (16)26 (22)29 (25)34 (29)5 (4)2 (2)1 (1)1.9, 1.321–30–6
      1f) My difficulties with defining the right moment to engage patients in ACP14 (12)19 (16)37 (32)23 (20)13 (11)7 (6)3 (3)2.3, 1.521–30–6
      1g) My having to deal with the emotional impact of ACP conversations in patients35 (30)31 (27)28 (24)14 (12)7 (6)1 (1)0 (0)1.4, 1.310–20–5
      1h) My belief that it is my job to cure people80 (69)19 (16)6 (5)6 (5)3 (3)2 (2)0 (0)0.6, 1.100–10–5
      1i) My belief that other health care professionals are better positioned to initiate ACP62 (53)22 (19)15 (13)12 (10)3 (3)2 (2)0 (0)0.9, 1.300–20–5
      1j) My belief that patients should initiate this type of discussion65 (56)26 (22)16 (14)7 (6)2 (2)0 (0)0 (0)0.8, 1.100–10–5
      1k) My belief that advance care plans are too simplified for complicated medical scenarios41 (35)31 (27)28 (24)11 (9)1 (1)4 (3)0 (0)1.2, 1.210–20–5
      1l) My doubts about the availability or accessibility of ACP documents when they are needed in the future to make medical decisions*17 (15)26 (22)43 (37)16 (14)5 (4)2 (2)7 (6)2.0, 1.521–30–6
      1m) My fear that these conversations will diminish hope in patients with serious illness42 (36)35 (30)26 (22)8 (7)3 (3)2 (2)0 (0)1.1, 1.210–20–5
      1n) My fear that ACP will negatively impact my relationship with patients62 (53)33 (28)13 (11)5 (4)2 (2)1 (1)0 (0)0.8, 1.000–10–5
      Barriers related to characteristics of patients
      2a) Patients difficulty accepting their poor prognosis7 (6)15 (13)27 (23)46 (40)14 (12)5 (4)2 (2)2.6, 1.332–30–6
      2b) Patients difficulty understanding the limitations and complications of life-sustaining therapies (mechanical ventilation, CPR, vasopressors, and so forth) at the end of life3 (3)9 (8)25 (22)43 (37)18 (16)15 (13)3 (3)3.0, 1.332–40–6
      2c) Patients think ACP is not relevant to them because they are too healthy right now7 (6)16 (14)24 (21)43 (37)12 (10)12 (10)2 (2)2.7, 1.432–30–6
      2d) Patients not understanding or misinterpreting my reasons for bringing up the topic11 (9)24 (21)24 (21)39 (34)14 (12)3 (3)1 (1)2.3, 1.321–30–6
      2e) Patients lack of understanding about how treatment decisions are made at the end of life6 (5)12 (10)31 (27)32 (28)19 (16)13 (11)3 (3)2.8, 1.432–40–6
      2f) Patients fear of upsetting their families by discussing the topic9 (8)20 (17)29 (25)33 (28)12 (10)8 (7)3 (3)2.5, 1.421–30–6
      2g) The patients strong religious convictions26 (22)30 (26)32 (28)22 (19)3 (3)2 (2)1 (1)1.6, 1.321–20–6
      2h) Incapacity of patient because of diminished consciousness or dementia or other cognitive disability11 (9)15 (13)20 (17)34 (29)15 (13)12 (10)6 (5)2.8, 1.632–40–6
      2i) Family unwillingness to support me in engaging the patient in ACP discussions14 (12)30 (26)27 (23)28 (24)9 (8)5 (4)2 (2)2.1, 1.421–30–6
      Barriers related to the health care system or external factors
      3a) Insufficient time during scheduled appointments to deal with this topic1 (1)5 (4)13 (11)17 (15)18 (16)37 (32)25 (22)4.2, 1.553–50–6
      3b) Limited capacity to honor patients expectations for care that arise from ACP discussions10 (9)18 (16)31 (27)32 (28)10 (9)10 (9)3 (3)2.5, 1.522–30–6
      3c) Decreased interaction with patients at end of life due to transfer of care to specialists or others11 (9)9 (8)24 (21)28 (24)15 (13)20 (17)9 (8)3.1, 1.732–50–6
      3d) Patients getting different messages from the GP and the other specialists involved in patients care7 (6)19 (16)30 (26)25 (22)21 (18)6 (5)7 (6)2.7, 1.532–40–6
      3e) Lack of financial remuneration for ACP30 (26)22 (19)19 (16)17 (15)7 (6)8 (7)9 (8)2.1, 1.920–30–6
      3f) Insufficient access to or availability of other health care professionals (social works, nurses, or others) to help with ACP13 (11)25 (22)30 (26)17 (15)14 (12)12 (10)5 (4)2.4, 1.721–40–6
      3g) Lack of ready access to forms and resources for patients14 (12)19 (16)28 (24)24 (21)13 (11)12 (10)6 (5)2.5, 1.721–40–6
      3h) Inability to electronically transfer patients advance care plan to acute care11 (9)6 (5)20 (17)25 (22)16 (14)17 (15)19 (16)3.4, 1.832–50–6
      ACP = advance care planning; GP = general practitioner; CPR = cardiopulmonary resuscitation.
      There were no missing responses.
      Number of respondents with a “do not know” response = 11.

      Exploratory Factor Analysis

      We selected the three-factor solution which explained 71% of the total variance because it provided the most parsimonious interpretable solution while explaining most of the total variance. Other less intuitive solutions with a greater number of factors explained more variance; however, a scree plot did not suggest a clear optimal number of factors. All items except one (lack of knowledge about legal status of ACP documents) had a factor loading in at least one domain of greater than 0.4 (Table 3). This item had the lowest communality (0.25), loaded roughly equally on both the doctor and health care system or external factors domain and was the only item whose removal increased the Cronbach's alpha presented in the next section. For these reasons, we removed this item from the remainder of the analysis. In the three-factor solution, all other items except one remained in our original domains: “My doubts about the availability or accessibility of ACP documents when they are needed in the future to make medical decisions” loaded on the domain relating to health care system or external factors. We moved this item to the health care system or external factors domain for the remainder of the analysis.
      Table 3Results of Exploratory Factor Analysis With Oblique Rotation for the Three-Factor Solution (n = 105)
      ItemBarriers Related to the DoctorBarriers Related to Characteristics of PatientsBarriers Related to the Health Care System or External Factors
      Factor loadings
       Barriers related to the doctor
      1a) My lack of knowledge about ACP and its relationship to advance directives and goals of care discussions0.550.07−0.01
      1b) My lack of knowledge about the legal status of ACP documents in the province
      Item does not load on domain where it was originally placed in the questionnaire.
      0.32−0.060.32
      1c) My lack of knowledge about how to elicit values, beliefs, and preferences related to end-of-life care0.650.12−0.04
      1d) My difficulties in dealing with patients changing preferences for medical treatments at the end of life0.540.060.13
      1e) My difficulties in dealing with uncertainty of prognosis for patients with chronic illness0.50−0.080.16
      1f) My difficulties with defining the right moment to engage patients in ACP0.530.000.14
      1g) My having to deal with the emotional impact of ACP conversations in patients0.760.01−0.02
      1h) My belief that it is my job to cure people0.74−0.170.01
      1i) My belief that other health care professionals are better positioned to initiate ACP0.690.040.02
      1j) My belief that patients should initiate this type of discussion0.690.040.02
      1k) My belief that advance care plans are too simplified for complicated medical scenarios0.620.07−0.01
      1l) My doubts about the availability or accessibility of ACP documents when they are needed in the future to make medical decisions*0.30−0.020.46
      1m) My fear that these conversations will diminish hope in patients with serious illness0.75−0.01−0.07
      1n) My fear that ACP will negatively impact my relationship with patients0.800.03−0.19
       Barriers related to characteristics of the patient
      2a) Patients difficulty accepting their poor prognosis0.170.540.06
      2b) Patients difficulty understanding the limitations and complications of life sustaining therapies (mechanical ventilation, CPR, vasopressors, and so forth) at the end of life0.070.460.12
      2c) Patients think ACP is not relevant to them because they are too healthy right now0.080.530.05
      2d) Patients not understanding or misinterpreting my reasons for bringing up the topic0.140.600.03
      2e) Patients lack of understanding about how treatment decisions are made at the end of life−0.040.700.01
      2f) Patients fear of upsetting their families by discussing the topic−0.110.85−0.08
      2g) The patients strong religious convictions0.030.640.05
      2h) Incapacity of patient because of diminished consciousness or dementia or other cognitive disability−0.190.580.25
      2i) Family unwillingness to support me in engaging the patient in ACP discussions0.030.68−0.16
       Barriers related to the health care system or external factors
      3a) Insufficient time during scheduled appointments to deal with this topic−0.100.040.59
      3b) Limited capacity to honor patients expectations for care that arise from ACP discussions0.220.110.58
      3c) Decreased interaction with patients at end of life due to transfer of care to specialists or others0.190.120.54
      3d) Patients getting different messages from the GP and the other specialists involved in patients care0.130.060.53
      3e) Lack of financial remuneration for ACP−0.12−0.100.69
      3f) Insufficient access to or availability of other health care professionals (social works, nurses, or others) to help with ACP0.20−0.010.52
      3g) Lack of ready access to forms and resources for patients0.180.040.49
      3h) Inability to electronically transfer patients advance care plan to acute care−0.20−0.010.66
      ACP = advance care planning; GP = general practitioner; CPR = cardiopulmonary resuscitation.
      a Item does not load on domain where it was originally placed in the questionnaire.

      Internal Consistency

      The Cronbach's alpha for the barriers related to the doctor domain was 0.90 (high reliability; Table 4). Furthermore, all items in this domain correlated with the total domain score, and there were no additional items that could be deleted to increase the overall Cronbach's alpha. Cronbach's alpha for the “Barriers Related to Characteristics of the Patient” and “Barriers Related to the Healthcare System or External factors” were 0.87 and 0.84, respectively, and again items were at least moderately correlated with the domain total, and no items could be removed to increase the overall Cronbach's alpha.
      Table 4Internal Consistency of Domains by Cronbach's Alpha Test (n = 105)
      ItemCorrelation with Domain TotalAlpha if Item Removed
      Barriers related to the Doctor (overall Cronbach's alpha = 0.90)
       1a) My lack of knowledge about ACP and its relationship to advance directives and goals of care discussions0.520.90
       1c) My lack of knowledge about how to elicit values, beliefs, and preferences related to end-of-life care0.660.89
       1d) My difficulties in dealing with patients changing preferences for medical treatments at the end of life0.630.89
       1e) My difficulties in dealing with uncertainty of prognosis for patients with chronic illness0.540.90
       1f) My difficulties with defining the right moment to engage patients in ACP0.570.90
       1g) My having to deal with the emotional impact of ACP conversations in patients0.730.89
       1h) My belief that it is my job to cure people0.640.89
       1i) My belief that other health care professionals are better positioned to initiate ACP0.670.89
       1j) My belief that patients should initiate this type of discussion0.640.89
       1k) My belief that advance care plans are too simplified for complicated medical scenarios0.600.89
       1m) My fear that these conversations will diminish hope in patients with serious illness0.670.89
       1n) My fear that ACP will negatively impact my relationship with patients0.660.89
      Barriers related to characteristics of the patient (overall Cronbach's alpha = 0.87)
       2a) Patients difficulty accepting their poor prognosis0.640.85
       2b) Patients difficulty understanding the limitations and complications of life sustaining therapies (mechanical ventilation, CPR, vasopressors, and so forth) at the end of life0.550.86
       2c) Patients think ACP is not relevant to them because they are too healthy right now0.550.86
       2d) Patients not understanding or misinterpreting my reasons for bringing up the topic0.650.85
       2e) Patients lack of understanding about how treatment decisions are made at the end of life0.630.85
       2f) Patients fear of upsetting their families by discussing the topic0.680.84
       2g) The patients strong religious convictions0.610.85
       2h) Incapacity of patient because of diminished consciousness or dementia or other cognitive disability0.560.86
       2i) Family unwillingness to support me in engaging the patient in ACP discussions0.540.86
      Barriers related to the health care system or external factors (overall Cronbach's alpha = 0.84)
       1l) My doubts about the availability or accessibility of ACP documents when they are needed in the future to make medical decisions
      Item moved to the domain on which it loaded in factor analysis.
      0.550.83
       3a) Insufficient time during scheduled appointments to deal with this topic0.480.83
       3b) Limited capacity to honor patients expectations for care that arise from ACP discussions0.700.81
       3c) Decreased interaction with patients at end of life due to transfer of care to specialists or others0.640.82
       3d) Patients getting different messages from the GP and the other specialists involved in patients care0.570.83
       3e) Lack of financial remuneration for ACP0.480.84
       3f) Insufficient access to or availability of other health care professionals (social works, nurses, or others) to help with ACP0.580.82
       3g) Lack of ready access to forms and resources for patients0.580.82
       3h) Inability to electronically transfer patients advance care plan to acute care0.480.84
      ACP = advance care planning; GP = general practitioner; CPR = cardiopulmonary resuscitation.
      a Item moved to the domain on which it loaded in factor analysis.

      Construct Validity

      The correlation for “patients think ACP is not relevant to them because they are too healthy right now” and “willingness to initiate ACP with patient” was −0.18 (P = 0.06), which did not support construct validity. The other two hypotheses were statistically significant and supported construct validity: the correlation for “my difficulties dealing with uncertainty of prognosis for patients with chronic illness” and “willingness to be an ACP decision coach for patient” was −0.28 (P = 0.003) and the correlation between “insufficient time during scheduled appointments to deal with this topic” and current behavior regarding “initiate ACP with patients” was −0.28 (P = 0.003).

      Discussion

      This study describes the initial assessment of the reliability and validity of a questionnaire to elicit barriers to ACP discussions reported by primary care physicians. We found that the questionnaire has face and content validity and reasonable psychometric properties. The development was grounded in an expert consensus-derived end-of-life communication and decision-making conceptual framework.
      • Sinuff T.
      • Dodek P.
      • You J.J.
      • et al.
      Improving end-of-life communication and decision making: the development of conceptual framework and quality Indicators.
      The face and content validity were established by the research team, some of whom published the conceptual framework and an international consensus definition of ACP.
      • Sudore R.L.
      • Lum H.D.
      • You J.J.
      • et al.
      Defining advance care planning for adults: a consensus definition from a Multidisciplinary Delphi Panel.
      Face and content validity were also verified through focus groups with primary care physicians representative of the questionnaire's audience. After administering the questionnaire to 117 family physicians, we found that none of the items had high rates of missing data and there were no floor effects that required deletion of items. The factor analysis suggested that the original three domains were reasonable and an alternate solution did not clearly emerge, although the proportion of variance could have been improved by considering a greater number of factors. The factor analysis suggested that the “lack of knowledge about legal status of ACP documents” item did not belong to any one of the three underlying domains and that “my doubts about the availability or accessibility of ACP documents when they are needed in the future to make medical decisions” belonged more to the health care system or external factor domain than the doctor domain as originally expected. All other items, loaded as expected based on the original three domains.
      In the Knowledge-to-Action framework of knowledge translation, the identification of barriers to acting on knowledge underpins the knowledge-to-action process by informing the nature of interventions and strategies for a specific setting or context.
      • Graham I.D.
      • Logan J.
      • Harrison M.B.
      • et al.
      Lost in knowledge translation: time for a map?.
      Our questionnaire was designed to identify the presence and magnitude of barriers that would illuminate priorities for interventions and strategies. As such, we conceptually grouped items into three domains of barriers: those related to physician, those related to characteristics of the patient, and those related to the system or external factors, and these domains were supported by the psychometric analyses. The domains may provide insight into where to target interventions. For example, the framework of Michie et al.
      • Michie S.
      • van Stralen M.M.
      • West R.
      The behaviour change wheel: a new method for characterising and designing behaviour change interventions.
      describes three essential conditions for a behavior to occur, linked to nine categories of strategies that can be tailored to the behavioral barrier(s). The barriers in our questionnaire could be mapped to such a framework when designing interventions.
      Overall, the magnitude of barriers to having ACP discussions with patients was rated low on the seven-point scale, although there were no floor effects. The responses may reflect social desirability bias whereby barriers are underrated, if there is a perception that ACP discussions are valuable but not occurring. Due to the convenience sample studied, the participants may have been more interested in ACP than the general population of primary care physicians and respondents could have underestimated the magnitude of barriers if they had previously considered ways to overcome them.
      Limitations of this study are the convenience sample of participants whose responses may not be representative of all primary care physicians, the sample size was limited for factor analysis, and we did not assess test-retest reliability of the questionnaire. Although the limited sample size could result in unstable estimates, the observed three-factor solution was sensible and consistent with our prior expectations. Further validation of the questionnaire, perhaps employing confirmatory factor analysis, should be undertaken with a larger representative sample and test-retest reliability should be examined.
      Valid reliable tools to measure barriers to ACP discussions can provide information to tailor and improve the effectiveness of intervention strategies. We have developed a questionnaire to assess barriers to ACP discussion from the perspective of primary care physicians with preliminary evidence of validity and internal consistency. This questionnaire represents a first step to understanding how to plan interventions for the primary care setting and monitor their effectiveness, whether it be a policy change or a clinic quality improvement process.

      Disclosures and Acknowledgments

      This work was supported by the Technology Evaluation in the Elderly Network, which is supported by the Government of Canada through the Networks of Centers of Excellence (NCE) program (Grant CORE2013–13A), and also by the Ministry of Health and Long Term Care in Ontario, Canada (Grant 06687).
      The funders had no role in study design, data collection, analysis or interpretation of data or in the writing of the report, or the decision to submit the article for publication.

      References

        • Sinuff T.
        • Dodek P.
        • You J.J.
        • et al.
        Improving end-of-life communication and decision making: the development of conceptual framework and quality Indicators.
        J Pain Symptom Manage. 2015; 49: 1070-1080
        • Detering K.M.
        • Hancock A.D.
        • Reade M.C.
        • Silvester W.
        The impact of advance care planning on end of life care in elderly patients: randomised controlled trial.
        BMJ. 2010; 340: c1345
        • Zhang B.
        • Wright A.A.
        • Huskamp H.A.
        • et al.
        Health care costs in the last Week of life.
        Arch Intern Med. 2009; 169: 480-488
        • The College of Family Physicians of Canada
        A vision for canada: Family practice - the patient's medical home. Position paper: College of family physicians of canada.
        2011 (Available from) (Accessed June 30, 2017)
        • Johnston S.C.
        • Pfeifer M.P.
        • McNutt R.
        The discussion about advance directives. Patient and physician opinions regarding when and how it should be conducted. End of Life Study Group.
        Arch Intern Med. 1995; 155: 1025-1030
        • Robinson C.
        • Kolesar S.
        • Boyko M.
        • Berkowitz J.
        • Calam B.
        • Collins M.
        Awareness of do-not-resuscitate orders: what do patients know and want?.
        Can Fam Physician. 2012; 58: e229-233
        • De Vleminck A.
        • Houttekier D.
        • Pardon K.
        • et al.
        Barriers and facilitators for general practitioners to engage in advance care planning: a systematic review.
        Scand J Prim Health Care. 2013; 31: 215-226
        • Brinkman-Stoppelenburg A.
        • Rietjens J.A.C.
        • van der Heide A.
        The effects of advance care planning on end-of-life care: a systematic review.
        Palliat Med. 2014; 28: 1000-1025
        • Canadian Hospice Palliative Care Association
        Advance care planning in Canada: National Framework and Implementation Task Group.
        Advance care planning in Canada: a national framework for consultation, 2010 (Available from) (Accessed June 30, 2017)
        • Howard M.
        • Bernard C.
        • Tan A.
        • Slaven M.
        • Klein D.
        • Heyland D.K.
        Advance care planning: let's start sooner.
        Can Fam Physician. 2015; 61: 663-665
        • Musa I.
        • Seymour J.
        • Narayanasamy M.J.
        • Wada T.
        • Conroy S.
        A survey of older peoples' attitudes towards advance care planning.
        Age Ageing. 2015; 44: 371-376
        • O'Sullivan R.
        • Mailo K.
        • Angeles R.
        • Agarwal G.
        Advance directives: survey of primary care patients.
        Can Fam Physician. 2015; 61: 353-356
        • Teixeira A.A.
        • Hanvey L.
        • Tayler C.
        • Barwich D.
        • Baxter S.
        • Heyland D.K.
        What do Canadians think of advanced care planning? Findings from an online opinion poll.
        BMJ Support Palliat Care. 2015; 5: 40-47
        • Evans N.
        • Pasman H.R.
        • Vega Alonso T.
        • et al.
        End-of-life decisions: a cross-national study of treatment preference discussions and surrogate decision-maker appointments.
        PLoS One. 2013; 8: e57965
        • Cochrane L.J.
        • Olson C.A.
        • Murray S.
        • Dupuis M.
        • Tooman T.
        • Hayes S.
        Gaps between knowing and doing: understanding and assessing the barriers to optimal health care.
        J Contin Educ Health Prof. 2007; 27: 94-102
        • Harrison M.B.
        • Legare F.
        • Graham I.D.
        • Fervers B.
        Adapting clinical practice guidelines to local context and assessing barriers to their use.
        Can Med Assoc J. 2010; 182: E78-E84
        • Chaudoir S.R.
        • Dugan A.G.
        • Barr C.H.
        • et al.
        Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures.
        Implement Sci. 2013; 8: 22
        • You J.J.
        • Downar J.
        • Fowler R.A.
        • et al.
        Barriers to goals of care discussions with seriously ill hospitalized patients and their families: a multicenter survey of clinicians.
        JAMA Intern Med. 2015; 175: 549-556
      1. Howard M, Bernard C, Klein D, et al. Barriers to and enablers of advance care planning with patients in primary care: a survey of healthcare providers. Can Fam Physician. In press.

        • Schaefer D.R.
        • Dillman D.A.
        Development of a standard e-mail Methodology: results of an Experiment.
        Public Opin Q. 1998; 62: 378-397
        • Dillman D.
        Mail and internet surveys: The tailored design method.
        Wiley, New York, NY2000
        • Streiner D.L.
        • Norman G.R.
        Health Measurement Scales: A practical guide to their development and use.
        4th ed. Oxford University Press, Oxford2008
        • Sudore R.L.
        • Lum H.D.
        • You J.J.
        • et al.
        Defining advance care planning for adults: a consensus definition from a Multidisciplinary Delphi Panel.
        J Pain Symptom Manage. 2017; 53: 821-832
        • Graham I.D.
        • Logan J.
        • Harrison M.B.
        • et al.
        Lost in knowledge translation: time for a map?.
        J Contin Educ Health Prof. 2006; 26: 13-24
        • Michie S.
        • van Stralen M.M.
        • West R.
        The behaviour change wheel: a new method for characterising and designing behaviour change interventions.
        Implement Sci. 2011; 6: 42