Advertisement

Development and Pilot Testing of a Simulation to Study How Physicians Facilitate Surrogate Decision Making Based on Critically Ill Patients' Values and Preferences

Open ArchivePublished:November 05, 2018DOI:https://doi.org/10.1016/j.jpainsymman.2018.10.513

      Abstract

      Context

      There are no evidence-based programs to train physicians to facilitate shared decision making based on incapacitated intensive care unit patients' values and preferences.

      Objectives

      The objective of this study was to develop a high-fidelity simulation to fill this gap.

      Methods

      Case development involved six steps: 1) drafting a case about an elderly patient receiving prolonged mechanical ventilation; 2) engaging an expert advisory board to optimize case content; 3) revising the case based on advisory board input; 4) training actors to portray the case patient's daughter; 5) obtaining physician feedback on the simulation; and 6) revising the case based on their feedback. We conducted a cross-sectional pilot study with 50 physicians to assess feasibility and acceptability, defined a priori as an enrollment rate >40 physicians/year, study procedures <75 minutes/participant, >95% actor adherence to standardization rules, and high physician ratings of realism and acceptability.

      Results

      Advisory panel feedback yielded two modifications: 1) refocusing the case on decision making about tracheostomy and percutaneous gastrostomy and 2) making the patient's values more authentic. Physician feedback yielded two additional modifications: 1) reducing how readily the actor divulged the patient's values and 2) making her more emotional. All 50 physicians enrolled in the pilot study over 11 months completed study procedures in <75 minutes. Actor adherence to standardization rules was 95.8%. Physicians' mean ratings of realism and acceptability were 8.4 and 9.1, respectively, on a 10-point scale.

      Conclusion

      Simulation is feasible, is acceptable, and can be adequately standardized to study physicians' skills for facilitating surrogate decision making based on an incapacitated intensive care unit patient's values and preferences.

      Key Words

      Introduction

      Incapacitated, critically ill patients near the end of life often receive burdensome, expensive treatment that they would not want.
      • Shalowitz D.I.
      • Garrett-Mayer E.
      • Wendler D.
      The accuracy of surrogate decision makers: a systematic review.
      • Smucker W.D.
      • Houts R.M.
      • Danks J.H.
      • Ditto P.H.
      • Fagerlin A.
      • Coppola K.M.
      Modal preferences predict elderly patients' life-sustaining treatment choices as well as patients' chosen surrogates do.
      • Frey R.
      • Hertwig R.
      • Herzog S.M.
      Surrogate decision making: do we have to trade off accuracy and procedural satisfaction?.
      • Heyland D.K.
      • Dodek P.
      • Rocker G.
      • et al.
      What matters most in end-of-life care: perceptions of seriously ill patients and their family members.
      • Barnato A.E.
      • Herndon M.B.
      • Anthony D.L.
      • et al.
      Are regional variations in end-of-life care intensity explained by patient preferences?: a Study of the US Medicare Population.
      Such treatment violates the fundamental principle of person-centered care and harms patients, their families, and society.
      Committee on quality of health care in America
      Crossing the Quality Chasm: A New Health System for the 21st Century.
      Part of the problem is that clinicians and families struggle to communicate effectively about how to incorporate the patient's values and preferences into a treatment plan.
      • Scheunemann L.P.
      • Arnold R.M.
      • White D.B.
      The facilitated values history: helping surrogates make authentic decisions for incapacitated patients with advanced illness.
      • Chiarchiaro J.
      • Ernecoff N.C.
      • Scheunemann L.P.
      • et al.
      Physicians rarely elicit critically ill patients' previously expressed treatment preferences in intensive care units.
      • Scheunemann L.P.
      • Cunningham T.V.
      • Buddadhumaruk P.
      • Arnold R.M.
      • White D.B.
      How clinicians discuss critically ill patients' preferences and values with surrogates: an empirical analysis.
      Improving these communication skills is an important goal, but evidence-based interventions do not exist.
      Before interventions can be tested, there must be a way of assessing change in clinicians' communication skills. One strong possibility for doing so is simulation, which theoretically standardizes clinical cases to limit variability, permit intervention on specific skills, and test the impact on patient outcomes.
      • Littlewood K.E.
      High fidelity simulation as a research tool.
      • McGaghie W.C.
      • Issenberg S.B.
      • Cohen E.R.
      • Barsuk J.H.
      • Wayne D.B.
      Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence.
      • Barsuk J.H.
      • McGaghie W.C.
      • Cohen E.R.
      • O'Leary K.J.
      • Wayne D.B.
      Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit.
      However, no methodology is available for testing whether medical simulations are adequately standardized for communication skills of interest.
      • Mariani B.
      • Doolen J.
      Nursing simulation research: what are the perceived gaps?.
      For example, a simulation focusing on surrogate decision making about an incapacitated patient's values and preferences should require that actors discuss a defined set of patient values in a reliable way in response to clinicians' questions. Such a methodology would allow researchers to study differences in clinicians' communication skills that might influence decision outcomes, assess changes in clinicians' communication skills in response to interventions, and demonstrate that the specific communication skills impact patient outcomes. Clearly, this is a critical gap.
      Therefore, we developed and pilot-tested a high-fidelity simulation to study how physicians facilitate surrogate decision making based on the values and preferences of an incapacitated, critically ill patient near the end of life. Our main aims were as follows: 1) to develop a methodology for testing the standardization of the simulation and 2) to apply it as part of an overall assessment of the simulation's feasibility and acceptability.

      Methods

      We chose simulation methodology because it is safe, is efficient, permits standardization, and can be applied across the range of observational and interventional communication research. The University of Pittsburgh IRB approved our study protocol, which was based on published guidelines for simulation development. The funding sources had no role in the study.

      Development Phase

      We followed published frameworks for standardized case development
      • Jeffries P.R.
      A framework for designing, implementing, and evaluating simulations as teaching strategies.
      • Valentin E.C.
      Guidelines for designing simulation building blocks.
      in 6 steps (Fig. 1):
      • 1.
        Drafting. The case focused on a 78-year-old woman on Day 12 of mechanical ventilation with a relatively poor prognosis receiving prolonged mechanical ventilation for Acute Respiratory Distress Syndrome (ARDS).
      • 2.
        Expert advisory panel. We shared the draft with an expert advisory panel consisting of clinicians and researchers from critical care medicine, critical care nursing, geriatrics, and palliative care. We conducted semistructured interviews
        • Galetta A.
        Mastering the semi-structured interview and beyond.
        to identify the strengths and weaknesses of the case for studying surrogate decision making about patient values and preferences in the intensive care unit (ICU).
      • 3.
        Case revision. We made two changes based on the expert advisory panel's feedback. First, we focused the case on whether the patient should undergo tracheostomy and percutaneous gastrostomy because these decisions involve trade-offs between deeply held values such as prolonging life, avoiding burdensome treatments, maintaining independence, and honoring spiritual beliefs.
        • Hawkins N.A.
        • Ditto P.H.
        • Danks J.H.
        • Smucker W.D.
        Micromanaging death: process preferences, values, and goals in end-of-life medical decision making.
        • Patrick D.L.
        • Starks H.E.
        • Cain K.C.
        • Uhlmann R.F.
        • Pearlman R.A.
        Measuring preferences for health states worse than death.
        • Karel M.J.
        • Moye J.
        • Bank A.
        • Azar A.R.
        Three methods of assessing values for advance care planning: comparing persons with and without dementia.
        Second, we made the patient's values more authentic to the way families talk.
      • 4.
        Actor training. We trained two experienced medical actors who were Caucasian women in their 30s–40s to portray the patient's daughter. They studied written information about their character and learned two standardization rules: 1) do not volunteer information about the patient's values and preferences unless the clinician asks and 2) give standardized information to specific types of questions. Although families do not operate under such rules, simulation always requires calculated trade-offs with realism. We considered this trade-off acceptable for two reasons. First, empiric evidence suggests that families do not talk much about patients' values and preferences in ICU family conferences if not asked.
        • Scheunemann L.P.
        • Cunningham T.V.
        • Buddadhumaruk P.
        • Arnold R.M.
        • White D.B.
        How clinicians discuss critically ill patients' preferences and values with surrogates: an empirical analysis.
        Second, we designed the rules to require that clinicians ask about patients' values and preferences in order for them to be discussed and to reward them for asking, allowing them to move the conversation forward toward a patient-centered treatment decision. Thus, the rules support the overarching goal of developing a method of assessing clinicians' communication skills and change in response to interventions.
      • We also trained the actors not to express too much emotion, so that clinicians did not spend the whole simulation on emotional support. They practiced in a series of four hours of role play with study investigators and two hours with volunteer physicians. The role plays included feedback about following the rules, as well as their authenticity to families encountered in clinical practice.
      • 5.
        Clinician panel. Once the actors had learned the role, we recruited 16 fellows and attendings from critical care and palliative care to pretest it. They went through the simulation procedures described in the section below on the pilot phase and provided structured feedback about how to improve it.
      • 6.
        Case revision. We made two modifications to the case based on the clinician panel's feedback. First, we made her more emotional because she was initially so reserved that she was unrealistic compared to actual practice. Second, we reduced how much information the actors divulged in response to any single question about the patient's values to one to two statements. This addressed both physicians' sense that the actors shared more than most family members and our goal of studying how clinicians facilitate shared decision making based on an incapacitated patient's values and preferences; they would not have to use these skills if the actor provided information too freely.
      Figure thumbnail gr1
      Fig. 1Simulation development. ARDS = Acute Respiratory Distress Syndrome; MV = mechanical ventilation.

      Pilot Phase

      Once we had completed these six steps, the simulation was ready for pilot testing. Although ICU family conferences typically include multiple family members and multiple interdisciplinary team members, we followed the precedent of prior research simulations about ICU decision making which included one to two family members and one physician.
      • Barnato A.E.
      • Hsu H.E.
      • Bryce C.L.
      • et al.
      Using simulation to isolate physician variation in ICU admission decision making for critically ill elders with end-stage cancer: a pilot feasibility study.
      • Uy J.
      • White D.B.
      • Mohan D.
      • Arnold R.M.
      • Barnato A.E.
      Physicians' decision-making roles for an acutely unstable critically and terminally ill patient.
      • Chiarchiaro J.
      • Schuster R.A.
      • Ernecoff N.C.
      • Barnato A.E.
      • Arnold R.M.
      • White D.B.
      Developing a simulation to study conflict in intensive care units.
      • Chiarchiaro J.
      • White D.B.
      • Ernecoff N.C.
      • Buddadhumaruk P.
      • Schuster R.A.
      • Arnold R.M.
      Conflict management strategies in the ICU differ between palliative care.
      This choice was feasible for a pilot study, minimized participant scheduling burdens, permitted us to work on the methods for studying and measuring communication between case family member–clinician dyads before expanding to families and teams, and permitted us to study physicians' communication skills in depth (analysis pending). We started with physicians because they strongly influence decision making
      • Garland A.
      • Connors A.F.
      Physicians' influence over decisions to forego life support.
      • Curtis J.R.
      • Barnato A.E.
      Variability in decisions to limit life sustaining treatment is it all about the physician chest.
      • Frost D.W.
      • Cook D.J.
      • Heyland D.K.
      • Fowler R.A.
      Patient and healthcare professional factors influencing end-of-life decision-making during critical illness: a systematic review.
      • Abbo E.D.
      • Yuen T.C.
      • Buhrmester L.
      • et al.
      Physicians' influence over decisions to forego life support.
      and improving the quality of their communication skills is a high priority to improve patient outcomes. The intent of these choices was to build a foundation for future work incorporating nonphysician team members, consistent with guidelines for shared decision making near the end of life.
      • Davidson J.E.
      • Powers K.
      • Hedayat K.M.
      • et al.
      Clinical practice guidelines for support of the family in the patient-centered intensive care unit: American College of Critical Care Medicine Task Force 2004-2005.
      We recruited a cross-sectional convenience sample of attendings or fellows in palliative or critical care via e-mail and advertising at local conferences and offered a $25 gift card in appreciation for participation. We typically scheduled three to four of those who responded to participate in the same morning and ran simulations back-to-back in the family conference room of our medical ICU to avoid conflicts with real conferences which typically happen in the afternoons. In the week before participation, we e-mailed physicians a copy of the patient's medical record including an off-service note from the prior intensivist as well as the morning's vitals, laboratories, and chest x-ray report (Online Supplement 1). We obtained informed consent at the time of enrollment.
      Enrolled physicians completed a demographic questionnaire in the physician workroom in our medical ICU and had a chance to review the patient's medical record. We instructed them to conduct the family meeting as if this were their patient. They then went to the ICU family meeting room where the audio- and video-recorders were already running, met the simulated daughter, and led the family meeting. Simulations were capped at 30 minutes for feasibility, at which time the investigator knocked on the door and called the physician out. Afterward, the physicians completed a questionnaire and a brief interview about the feasibility and acceptability of participation. The actors completed a brief questionnaire from the case daughter's perspective. We noted whether physicians completed study procedures in <75 minutes.

      Measurement

      We considered feasibility to have three parts: 1) that participants were available, willing, and able to participate; 2) that the simulation was sufficiently standardized with regard to the target communication skills; and 3) that the simulation demonstrated some ability to discriminate between physicians' skill for discussing patients' values and preferences.
      For the first goal, we measured physicians' availability as the enrollment rate (number of physicians participating per year); their willingness to participate based on responses to previously used questionnaires
      • Chiarchiaro J.
      • Schuster R.A.
      • Ernecoff N.C.
      • Barnato A.E.
      • Arnold R.M.
      • White D.B.
      Developing a simulation to study conflict in intensive care units.
      about realism and acceptability of participating in the simulation; and efficiency of participation as the time required to complete study procedures (to ensure study procedures would fit in physicians' schedules and to keep actor payments within the study budget).
      To test the second goal, we conducted a mixed-methods analysis of simulation transcripts to test how well the actors adhered to the standardization rules. The qualitative analysis used a previously developed coding scheme according to Crabtree and Miller's template coding strategy,
      focusing on two key aspects of shared decision making: 1) discussing past expressions of the patient's values and preferences and 2) deliberating about how to apply them to treatment planning.
      • Kon A.A.
      • Davidson J.E.
      • Morrison W.
      • Danis M.
      • White D.B.
      Shared decision making in ICUs.
      To test actors' standardization, we added three subcodes indicating how actors' statements related to physicians' statements: 1) volunteering (which has correct content but does not require that physicians use the target communication skills); 2) answering incorrectly/not sharing when asked (which does not reward the communication skills physicians utilize); and 3) answering appropriately (responding to a physician prompt about the patients' values and preferences with correct information from the case). Online Supplement 2 provides exemplars.
      Two internal medicine residents who were masked to physician specialty trained in the coding scheme by coding a subset of five transcripts line-by-line and resolving any differences by discussion with the principal investigator. Next, they demonstrated excellent interrater reliability compared to each other and the principal investigator (average kappas ≥ 0.83) on 20-question tests constructed from a bank of exemplars. After this training, they each independently coded half of the transcripts. To ensure accurate coding, they conducted qualitative data cleaning with the principal investigator. This step involved reviewing all coding line-by-line to ensure appropriate inclusion of all eligible statements, exclusion of statements not fitting the code definitions, and categorization of all statements according to the codebook rules.
      We approached the third goal, testing the simulation's ability to discriminate between physicians' skill in discussing the patient's values and preferences, in two ways
      • Shalowitz D.I.
      • Garrett-Mayer E.
      • Wendler D.
      The accuracy of surrogate decision makers: a systematic review.
      : describing physicians' communication skill use to assess the range and distribution for floor or ceiling effects
      • Smucker W.D.
      • Houts R.M.
      • Danks J.H.
      • Ditto P.H.
      • Fagerlin A.
      • Coppola K.M.
      Modal preferences predict elderly patients' life-sustaining treatment choices as well as patients' chosen surrogates do.
      ; assessing whether actors' ratings of patient-centered communication on a modified Patient-Centeredness of Care-Surrogate
      • Stewart M.
      • Brown J.B.
      • Donner A.
      • et al.
      The impact of patient-centered care on outcomes.
      questionnaire were associated with physicians' communication skill use. The Patient-Perceived Patient-Centeredness of Care scale has 14 items on a four-point Likert scale; it has demonstrated construct validity and been adapted for surrogate decision makers.
      • White D.B.
      • Cua S.M.
      • Walk R.
      • et al.
      Nurse-led intervention to improve surrogate decision making for patients with advanced critical illness.
      We had to remove two of its items for use in the simulation because of its cross-sectional design. The rationale for these two strategies for testing discrimination was that they balanced a quantitative assessment of how frequently physicians used skills with a qualitative assessment of how it felt from the case daughter's perspective.

      Statistical Analysis

      We calculated descriptive statistics for all measures. We considered that feasibility required an enrollment rate of at least 40 physicians/year, retention of ≥95% of physicians for all study procedures, keeping the time burden of participation ≤75 minutes for all participants, median realism and acceptability scores ≥8/10, and <5% rate of errors for actors in following the standardization rules. To determine the error rate, we calculated the total number of opportunities for discussing the patient's values and preferences by adding all the statements in which the actor volunteered, answered correctly, answered incorrectly, or failed to answer physicians' questions about them. Any statement that was not answered correctly was counted as an error, and the error rate was the number of errors divided by the total number of opportunities. To assess whether the simulation discriminated between physicians' use of skills for discussing patients' values and preferences, we first calculated descriptive statistics of their skill use. Because it was nonnormally distributed, we used Spearman correlation to test its association with actor-rated Patient-Centeredness of Care.

      Results

      Study Population

      Table 1 shows the demographics of our study population. Overall, they were representative of palliative and critical care physicians at a large, urban academic tertiary care center in the U.S. The young mean age with a wide standard deviation reflects the mix of fellows and attendings.
      Table 1Participant Characteristics
      CharacteristicDevelopment Phase N = 16% or Mean (SD)Pilot Phase N = 50% or Mean (SD)
      Mean age (SD)37.4 (9.0)39.8 (9.8)
      % Male3852
      Race
       % Asian2512
       % Caucasian7574
       % Other014
      % Latino or Hispanic126
      % Trainees4432
      Mean years in practice (SD)8.3 (9.3)8.9 (8.8)
      Area of practice
       % Critical care8872
       % Palliative care1228

      Feasibility of Study Procedures

      Study procedures demonstrated the feasibility of recruiting and retaining physicians to completion of data collection: the physician enrollment rate was 50 physicians/year, the study completion rate was 100% for enrolled physicians, and 100% of physicians completed the study in <75 minutes. Physician ratings of the simulation's realism and acceptability of participation were also consistently high (Table 2). Semistructured interviews with physicians suggested several reasons why the simulation did not seem perfectly realistic, including that there was only one family member and only one clinician (when most family meetings have multiple family members and clinicians); the daughter character was not as emotional as a typical family member although they had experience with such reserved family members; and although the patient's values and preferences seemed similar to real patients, the daughter expressed them more readily and clearly than most families.
      Table 2Realism and Acceptability
      To What Extent Were the Following Similar to What You Encounter in Your Practice?Pilot Phase Median Scale 1–10 (IQR)
      RealismFamily conference8 (7–10)
      Actor portraying the family member9 (7–10)
      Emotions expressed9 (8–10)
      Discussion about the patient's values and treatment preferences8.5 (7–10)
      Conference room10 (9–10)
      Clinical information10 (8–10)
      AcceptabilityHow acceptable did you find participation overall?9.75 (8–10)
      IQR = interquartile range.

      Feasibility of Standardizing Communication

      Transcript analysis revealed the feasibility of standardizing communication of actors within medical simulations (Table 3). All of the simulations contained some discussion of the patient's values and preferences. Overall, there were 1068 physician-actor statements about the patient's values and preferences, which counted as opportunities for success or error. The overall error rate was 4.2%, with most errors reflecting problems volunteering or incorrectly responding (giving information that was not in the case, exemplar in Online Supplement 2), to physicians' questions about past expressions of the patient's values and preferences.
      Table 3Frequency of Actor Errors During the Pilot Phase
      Error TypesN Errors% Errors
      Values and preferences
       Volunteered232.1
       Didn't share/answered incorrectly111.0
      Treatment plans
       Volunteered1<0.001
       Didn't share/answered incorrectly100.9
      Total (1068 opportunities)454.2

      Discrimination Among Physicians' Use of Communication Skills

      The simulation demonstrated the ability to discriminate differences in physicians' communication skill use in both assessment strategies (Table 4). First, the lowest number of physician statements about the patient's values and preferences in a conference was 1 and the maximum was 42, with a mean of 12.7, an SD of 8.4, and a nonnormal distribution with an interquartile range of 2–25. There was a similar variation within the individual communication skills assessed; because deliberation is contingent on elicitation, the maximum number of statements was less, but the distribution was still nonnormal. Second, physicians' use of communication skills for discussing patients' values and preferences showed significant association with Patient-Centeredness of Care-Surrogate (Spearman's rho = 0.36, P = 0.01). Two questions on the Patient-Centeredness of Care-Surrogate are specifically related to communication about patients' values and preferences: 1) to what extent did the doctor ask about your loved one's goals/preferences for treatment? and 2) how much would you say that the health care team(s) care about your loved one as a person? In a sensitivity analysis testing the association of just these two statements with physicians' use of communication skills for discussing patients' values and preferences, the correlation remained significant (Spearman's rho = 0.43, P = 0.01).
      Table 4Physicians' Use of Communication Skills Within the Simulation
      Communication Skills Used by PhysiciansSimulation Characteristics (N = 50)
      Elicited the patient's previously expressed values and preferences
       Any question—n (%)50 (100)
       Number of questions—mean (SD, total range)9.8 (7.3, 1–36)
      Deliberated about how to apply the patient's values and preferences in the current situation
       Discussed how the patient would think or feel about the current situation or possible futures
      Any discussion—n (%)40 (80)
      Mean (SD, total range)2.2 (1.8, 0–6)
       Offered a treatment recommendation based on the patient's values and preferences
      Any recommendation—n (%)23 (46)
      Number of recommendations—mean (SD, total range)0.68 (0.94, 0–4)
      Total communication about patients' values and preferences
       Any question—n (%)50 (100)
       Number of statements—mean (SD, range)12.7 (8.4, 1–42)
       Association with Patient-Centeredness of Care—Spearman's rho (P-value)0.36 (0.01)

      Discussion

      This study demonstrated that simulation is a feasible and acceptable methodology for studying physician-surrogate communication about an incapacitated patient's values and preferences. Physicians were willing and able to participate and agreed that the simulation was realistic and an acceptable use of their time. The design of the simulation for targeting communication about patients' values and preferences was successful, in that 100% of the simulations contained some discussion of them compared to about 70% in usual practice.
      • Scheunemann L.P.
      • Cunningham T.V.
      • Buddadhumaruk P.
      • Arnold R.M.
      • White D.B.
      How clinicians discuss critically ill patients' preferences and values with surrogates: an empirical analysis.
      However, the simulation still discriminated both among physicians' quantitative use of skills (i.e., how many times did they use each skill) and the qualitative impact of those skills (i.e., how patient-centered did it feel to the actor portraying the case daughter). The nonnormal distribution of physicians' skill use ranging from a minimum of 1 to a maximum of 42 statements for discussing patients' values and preferences suggests low likelihood of important floor or ceiling effects. And despite the complexity of these conversations, the actors successfully followed standardization rules with <5% errors.
      This last result deserves highlighting. Our novel analysis showed the feasibility of standardizing actor responses in medical communication simulations. As simulation moves from a medical education tool to a research tool, demonstrating its standardization takes on increased importance. The overall error rate of 4.2% has face validity for good standardization for such complex communication skills. Future research should refine methods for ensuring that simulations are standardized with regard to target communication skills. For example, if clinicians trained in facilitating communication based on the patients' values and preferences demonstrated increased use of these skills in our simulation, it would support the simulation's convergent validity as a tool for studying those skills. Another key refinement is increasing the efficiency of testing both simulation standardization and clinicians' skill use, for example, by a checklist or other tool that could be completed in real time during simulations. Finally, guidelines recommend that interdisciplinary teams facilitate shared decision making with families.
      • Davidson J.E.
      • Powers K.
      • Hedayat K.M.
      • et al.
      Clinical practice guidelines for support of the family in the patient-centered intensive care unit: American College of Critical Care Medicine Task Force 2004-2005.
      Future studies should include nonphysician team members and refine the methodology for assessing standardization when multiple participants are present.
      Medical education research has long used simulation to train physicians in complex skills like communication based on the understanding that adult learners require practice and feedback to incorporate and use such skills.
      • Back A.L.
      • Arnold R.M.
      • Baile W.F.
      • et al.
      Efficacy of communication skills training for giving bad news and discussing transitions to palliative care.
      • Arnold R.M.
      • Back A.L.
      • Barnato A.E.
      • et al.
      The Critical Care Communication project: improving fellows' communication skills.
      Our study, a simulation studying physicians' communication skills during critical care triage,
      • Barnato A.E.
      • Hsu H.E.
      • Bryce C.L.
      • et al.
      Using simulation to isolate physician variation in ICU admission decision making for critically ill elders with end-stage cancer: a pilot feasibility study.
      • Mohan D.
      • Alexander S.C.
      • Garrigues S.K.
      • Arnold R.M.
      • Barnato A.E.
      Communication practices in physician decision-making for an unstable critically ill patient with end-stage cancer.
      and a simulation studying how physicians manage conflict during family conferences
      • Chiarchiaro J.
      • Schuster R.A.
      • Ernecoff N.C.
      • Barnato A.E.
      • Arnold R.M.
      • White D.B.
      Developing a simulation to study conflict in intensive care units.
      • Chiarchiaro J.
      • White D.B.
      • Ernecoff N.C.
      • Buddadhumaruk P.
      • Schuster R.A.
      • Arnold R.M.
      Conflict management strategies in the ICU differ between palliative care.
      represent first steps in adapting this methodology for critical care outcomes research. The long-term goal of this work is translational
      • Littlewood K.E.
      High fidelity simulation as a research tool.
      : identify the “active ingredients” of communication that improve patient- and family-centered outcomes and develop, test, and disseminate interventions focused on those active ingredients to improve patient and family outcomes. Because the skills and aims of medical educators and intervention researchers are compatible but not overlapping,
      • Munroe B.
      • Buckley T.
      • Curtis K.
      • Morris R.
      Designing and implementing full immersion simulation as a research tool.
      this use of simulation methodology is an ideal model for interdisciplinary research collaboration.
      This study's most important limitation is that it was conducted with a convenience sample of physicians at a single center that has shown a strong commitment to communication training and research for critical care physicians.
      • Arnold R.M.
      • Back A.L.
      • Barnato A.E.
      • et al.
      The Critical Care Communication project: improving fellows' communication skills.
      Whether physicians with less strong institutional support would be as available to participate or would rate its acceptability as highly is unknown. It seemed important that we held the study in an easily accessible location, were committed to fitting physicians' schedules, and committed to not running over 75 minutes. Second, although most of the communication simulations in critical care have focused on single-episode communication with one provider and one simulator, usual practice typically includes multiple family members and multiple interdisciplinary team members over time. This was a reason why physicians downgraded the simulation's realism. Future research should use this simulation to study interprofessional communication and adapt it to study the process of communication over time. A third limitation is that we used a single case representing a “typical” patient, an “easy” family member, and easily elicited, clear values. These choices were reasonable to begin research in this area; we made them under the guidance of our expert advisory panel according to simulation guidelines despite the fact that they decreased the case's realism. Future work should investigate a range of ages, genders, ethnicities, clinical scenarios, family behaviors, and patient values. Finally, the standardization rules prevented the case daughter from volunteering information or changing the topic, which is not how families operate. However, physicians scored the realism well, indicating that they did not perceive a significant difference from practice. Again, the goal of the simulation is not to perfectly mirror actual practice but to provide a good laboratory for learning and practicing new communication skills. The ultimate test of whether it is realistic enough will be whether clinicians demonstrate improved use of communication skills for discussing patients' values and preferences in the simulation, transfer them to practice, and improve patient-centeredness of care—which are important questions for future research.
      In conclusion, this study developed a feasible and reliable simulation to study physician communication with surrogate decision makers about incapacitated, critically ill patients' values and preferences. Its methods for demonstrating the simulation's standardization are novel and may be useful to other researchers studying communication using simulation. The next step is to use the simulation to test the efficacy of a communication skills training intervention to improve how clinicians elicit and integrate patients' values into goals of care decisions in ICUs.

      Disclosures and Acknowledgments

      This work was supported by the National Institutes of Health (grant numbers NIH T32HL007820-16A1, NIH T32HL007563-27, and NIA F32 AG047806-01A1); the Jewish Healthcare Foundation; and the Arthur Vining Davis Foundation. The funding agreement ensured the authors' independence in designing the study, interpreting the data, writing, and publishing the report.
      The authors appreciate the contributions of Douglas B. White, MD, MAS, to the conception and design of this research.

      Online Supplement 1. Case Patient Electronic Medical Record: Vitals, Laboratory Data, Off-Service Progress Note, and Radiologist Chest X-Ray Interpretation

      Tabled 1
      3Flowsheet Print Request
      Patient: JEFFERSON, MARYPrinted by: PULMONARY FELLOW
      MRN: 100004025Date Range: m/d-1/yyyy 12:00 am to m/d/yyyy 08:00 amPrinted on: m/d/yyyy 08:00 am.
      Tabled 1
      Vitalsm/d/yyyy

      2:32 am
      m/d/yyyy

      4:47 am
      m/d/yyyy

      5:54 am
      m/d/yyyy

      6:13 am
      m/d/yyyy

      7:52 am
      Vital signs
      Temperature conversion (C)37.3
      Temperature (F)99.1
      Heart rate7668738286
      Pulse locationMonitorMonitorMonitorMonitorMonitor
      Pulse characterRegularRegularRegularRegularRegular
      Respiratory rate2424252624
      Respiratory patternAssistedAssistedAssistedAssistedAssisted
      Ventilator rate2424242424
      Systolic/diastolic BP106/46114/50116/50119/60104/49
      Mean blood pressure6272728369
      SpO29194939591
      Oxygen % (FiO2)6060606060
      Respiratory devices/methodEndotracheal tubeEndotracheal tubeEndotracheal tubeEndotracheal tubeEndotracheal tube
      Tabled 1
      Flowsheet Print Request
      Patient: JEFFERSON, MARYPrinted by: PULMONARY FELLOW
      MRN: 100004025Date range: m/d-1/yyyy 12:00 am to m/d/yyyy 08:00 amPrinted on: m/d/yyyy 08:00 am
      Tabled 1
      Laboratorym/d/yyyy

      4:00 am
      Blood gases
       pHa7.34
       PaCO257
       PaO274
       HCO3a28
       Base excess, arterial3
       SaO291
      Common Chem
       Na132
       K3.5
       Cl98
       CO228
       Anion gap10
       BUN28
       Cr1.6
       Glucose142
       Ca (adjusted)7.7
       Mg1.9
       Phos1.8
       Albumin1.9
       Total protein4.6
       Bili, total1.8
       Bili, direct0.5
       ALT/SGPT15
       AST/SGOT26
       Alk Phos133
      Hematology
       WBC13.8
       RBC2.72
       Hgb8.7
       Hct26.4
       MCV92.8
       MCH30.2
       MCHC32.5
       RDW16.4
       MPV9.4
       Platelets246
       Neutrophils75
       Lymphs11
       Monocytes13
       Eosinophils1
       Basophils0
       PT14.8
       INR1.3
       PTT31.5
      Result type: IP-Progress
      Performed date: Month d-1, yyyy
      Result status: final
      Result title: PACCM progress note
      Performed by: PULMONARY FELLOW on Month d-1, yyyy 10:16 am
      Verified by: PULMONARY FELLOW on Month d-1, yyyy 10:16 am
      Encounter info: 007395720372, UPMCPUH, inpatient, m/d-12/yyyy –
      * Final Report *
      Document Has Been Updated and Contains Addenda
      Addendum by PULMONARY ATTENDING on Month d-1, yyyy 6:27 pm (Verified)
      I have evaluated this patient with the fellow and clinical team and agree with the examination and plan documented.
      A 78-year-old woman was admitted 11 days ago with pneumonia and ARDS complicated by septic shock and acute kidney injury. Initially required paralytics and pressors to maintain oxygenation and mean arterial pressure, respectively, although these agents have now been weaned off. Although she has been stabilized from a respiratory standpoint, she is weaning slowly. Will continue low tidal volume strategy with daily sedation interruption and pressure support trials as tolerated. Will also attempt to minimize sedation/deliriogenic medications and begin mobilization.
      Her prognosis remains guarded, with approximately 30% expected in-hospital mortality. She will most likely require long-term trach/PEG, with ∼50% chance of liberation from mechanical ventilation. In addition, she is likely to have ongoing physical dysfunction, and possibly also cognitive dysfunction, with associated need assistance for basic daily activities. Plan for family discussion tomorrow.
      PACCM Progress Note
      University of Pittsburgh Medical Center
      Patient: JEFFERSON, MARYMRN: 100004025FIN: 007395720372
      Age: 78 yearsSex: femaleDOB: 10/14/1935
      Associated diagnoses: None
      Author: PULMONARY FELLOW
      History of present illness
      A 78-year-old woman with h/o severe COPD (baseline FEV1 48% predicted) who called EMS the afternoon of Dec 4 for respiratory distress secondary to pneumonia. They intubated her or room air SaO2 in the 70s, at which time they found her to have copious purulent secretions. On arrival at the UPMC Cranberry ED, she still had marginal SaO2 (high 80s) on 100% FiO2 and was hypotensive (bp's 70s/40s). She received zosyn, azithro, and vanc for community-acquired pneumonia and norepi for hypotension. Sats remained tenuous despite increased PEEP, so she was transferred to Presby for further management. On arrival, she was agitated and dyssynchronous with the vent with sats in the low 80s on 100% FiO2 and 18 of PEEP and has required intermittent boluses of paralytics to maintain SaO2 > 88%. CXR is consistent with ARDS, with increased consolidation in the RLL suggesting pneumonia as the inciting cause.
      Histories
      • PMH:
        • -
          COPD (baseline FEV1 48% predicted, temporarily on home O2 after an exacerbation last year)
        • -
          Hypertension
        • -
          Hyperlipidemia
        • -
          Osteoporosis
        • -
          Arthritis
      • PSH:
        • -
          s/p hysterectomy 1962.
      SH: Former smoker (45+ pack years), quit 12 years ago. Social alcohol (∼2–3 drinks/month). No illicits. Lives independently in Cranberry. Daughter Angela lives close by.
      FH: Father died of a heart attack in his 80s. Mother died of “old age.” No significant sick contacts. No family history of cancer.
      Health status
      Allergies
      NKDA
      Inpatient Medications
      Scheduled Medications
      Albuterol-ipratropium (Combivent 103 mcg to 18 mcg/inh aerosol with adapter) 4Puff(s) Inhalation Q4H.
      aspirin 81 mg by mouth daily.
      atorvastatin (Lipitor) 20 mg by mouth daily.
      chlorhexidine topical 0.12% (chlorhexidine topical 0.12% liquid) 15 mL SwishSpit BID.
      docusate (Colace) 100 mg NG tube BID.
      heparin 5,000 unit(s) subQ Q8H.
      lansoprazole (Prevacid SoluTab) 30 mg NG tube daily.
      olanzapine (ZyPREXA Zydis) 5 mg NG tube BID.
      senna 10 mL by mouth at bedtime
      PRN Medications
      albuterol 0.083% (albuterol 0.083% inhalation soln) 2.5 mg aerosol Q4H.
      lorazepam 2 mg IV Q2H.
      olanzapine (ZyPREXA Zydis) 5 mg NG tube daily.
      Recently Discontinued Medications
      norepinephrine 16 mg [0.1 mcg/kg/minute] + Base solution 250 mL (norepinephrine 16 mg/NS 250 ... 250 mL 6.09 mL/hour IV.
      Continuous Infusions
      fentanyl 2500 mcg [50 mcg/hr] + base solution 100 mL initial rate = 2 mL/hour IV.
      Review of Systems
      Unable to obtain 2/2 respiratory failure/sedation.
      Interval History
      Overnight events: Mild agitation overnight resolved with prn Zyprexa. Desaturates 2/2 dyssynchrony when agitated. Otherwise stable.
      Physical Examination
      Vital Signs (Last 7 in Past 36 Hours)
      Tabled 1
      VitalsTempCBPPulseRRSaO2FiO2
      mm/dd-1 08:1437.698/5490259460%
      mm/dd-1 10:56105/52889360%
      mm/dd 12:0495/51909160%
      mm/dd 02:4636.692/5191249460%
      mm/dd 04:12102/59889360%
      mm/dd 06:57119/58949160%
      mm/dd 08:23
      24 hours Max Temp: 37.7 at mm/dd-1 15:45. Dosing Wt: 64 kg.
      36 hours Max Temp: 37.7 at mm/dd-1 15:45. BMI: No data found.
      General: Restless and agitation at night and during daily sedation interruptions. Ramsay sedation scale: Level 4 patient asleep, brisk response to loud auditory stimulus, or light glabellar tap.
      Eye: Pupils are equal, round, and reactive to light. Normal conjunctiva.
      HENT: Normocephalic, dry oral mucosa, orally intubated.
      Neck: Supple, no carotid bruit, ∼10 cm JVD.
      Respiratory: Breath sounds are equal, coarse breath sounds throughout with occasional expiratory wheezing. Synchronous with the ventilator, small amount of thin white secretions.
      Cardiovascular: Normal rate. Regular rhythm, no murmur, no gallop. Thready pulses with sluggish capillary refill.
      Gastrointestinal: Soft, nontender, nondistended, normal bowel sounds.
      Lymphatics: No lymphadenopathy.
      Integumentary: Warm, dry, pink.
      Neurologic: Gag reflex normal, intubated, sedated. Weakness and difficulty raising arms off bed noted (upper and lower extremity strength 3/5) during sedation interruption.
      Cognition and speech: Intubated, sedated.
      Impression and Plan
      A 78-year-old woman with severe COPD presenting 11 days ago with pneumonia and bilateral airspace infiltrates with P/F ratio <200 consistent with ARDS. Initially required paralytics to manage ventilator dyssynchrony/breath stacking and hypoxemia.
      Neurologic: Intermittently agitated and delirious. Unclear if her weakness is a consequence of critical illness myoneuropathy (especially with steroid and paralytic use earlier in her ICU stay) or simple deconditioning.
      • -
        Continue sedation with fentanyl
      • -
        Delirium management with scheduled and prn zyprexa
      • -
        Prn haldol in case agitation results in dyssynchrony/desaturation; reserve lorazepam only in case agitation does not respond to haldol
      • -
        MICU mobility protocol
      Pulmonary: Has course of antibiotics and steroids for pneumonia/COPD exacerbation and now has minimal secretions. Appears to be entering the fibrotic phase of ARDS, with stabilizing SaO2. However, she is weaning slowly and is unlikely to be liberated from the ventilator soon, especially given her severe underlying COPD.
      • -
        continue low tidal volume ventilation (6 mg/kg)
      • -
        wean oxygen as tolerated to maintain SaO2 >88%
      • -
        combivent, prn albuterol to manage obstruction
      • -
        will discuss tracheostomy with daughter before consulting surgery
      Cardiovascular: Initially had severe sepsis (altered mentation, reduce UOP, lactic acidosis), possibly also with a component of pulmonary hypertension (elevated JVP, large RV on bedside US, rapidly worsening hypotension with auto-PEEP). Now off pressors and maintaining adequate MAP.
      • -
        continue home ASA, statin
      Renal: Creatinine is somewhat elevated but improving, as is her UOP. Expect she will avoid dialysis this hospitalization.
      • -
        Strict I/O
      • -
        Renally dose medications and avoid nephrotoxins
      GI: Has duotube (see below)
      • -
        Colace, senna while on fentanyl gtt
      • -
        Tube feeds at goal
      Access: RIJ TLC (placed 12/4)
      Tubes: Foley, duotube
      Prophylaxis: Prevacid, SC heparin.
      Code status: full code. Advance directive on chart.
      DISPO: Family meeting scheduled for tomorrow morning to discuss trach/PEG. Prognosis for long-term recovery is guarded; she is progressing toward chronic critical illness. Given her weakness and delirium, she has a low (probably <5%) chance of being functional enough to live with her daughter at six months and would still require assistance for walking/transfers, bathing, and dressing and most likely will require long-term care at a skilled nursing facility. Have updated daughter, Angela, nearly every day, but because she has been so unstable, most of our decision making has focused on acute issues like the possible need for dialysis, and we have not addressed more long-term goals of care.
      Diagnosis: Acute respiratory distress syndrome (ICD9 518.52, final, diagnosis), septic shock (ICD9 785.52, final, diagnosis), CAP (community-acquired pneumonia) (ICD9 486, final, diagnosis), COPD, severe (ICD9 496, final, diagnosis).
      Professional Services
      Visit information: Patient seen on 12/d-1/2014.
      Credentials: MD.
      Title: Fellow.
      Supervising MD: PULMONARY ATTENDING.
      Critical care time: I spent 39 minutes personally attending to this patient's critical care needs. This critical care time is exclusive of any time for separately billable procedures.
      Result type: Chest anterior X-ray.
      Performed date: Month d, yyyy 4:23 am.
      Result status: final.
      Result title: XRAY CHEST FRONTAL VIEW.
      Encounter info: 007395720372, UPMCPUH, inpatient, mm/d-12/yyyy –
      * Final Report *
      XRCH1V
      CLINICAL HISTORY:
      Respiratory failure, ARDS.
      COMPARISON:
      Yesterday.
      TECHNIQUE:
      Portable AP projection of the chest.
      FINDINGS:
      Diffuse bilateral patchy opacities are grossly unchanged and are consistent with slowly resolving acute respiratory distress syndrome.
      The endotracheal tube remains in good position, as does the right internal jugular central venous catheter with the tip in the distal SVC. Feeding tube courses below the diaphragm and appears unchanged from yesterday.
      IMPRESSION:
      • 1.
        Diffuse alveolar infiltrates in all lobes of both lungs, likely representing ARDS
      • 2.
        Unchanged tubes and lines
      End of impression:
      Dictated by: RADIOLOGY ATTENDING.
      Signed by: RADIOLOGY ATTENDING.
      Signed on: mm/d/yyyy at 6:12 am.

      Online Supplement 2. Exemplars of Correct Responses and Errors According to the Case Rules

      Tabled 1
      Type of StatementExemplarExplanation of Case Rule/Violation
      Values and preferences
       Answered correctlyPhysician: And was she like an independent person, or what was she like?

      Case daughter: Oh, [chuckles] fiercely independent.
      The case daughter was supposed to talk about the patient's values and preferences in response to all physician questions about them.
       VolunteeredPhysician: OK. And so you've been taking time from work to come here?

      Case daughter: Um, I have a certain amount of flexibility. So yeah, and I also want to say, I mean, we had—talking to my mom about if there ever came a time when it was too rough for her to live by herself that—how she would feel about living with us. And she welcomed that idea, she wasn't resistant to it.
      The case daughter was never supposed to talk about the case patient's values and preferences without the physician asking. Here, she volunteers the patient's values related to living independently.
       Didn't sharePhysician: Now I did want to kind of look at the big picture with you and kind of get a sense of what she's expressed to you in the past about long-term support like that, right, because if we did a tracheostomy, that would be a long-term commitment to being on the ventilator essentially.

      Case daughter: Um, you, you—so once you do that you can't ever come off that, is that what you're saying?
      Whenever the physician asked about the patient's values and preferences, the case daughter should talk about them. Here, she changed the subject to talk about treatment options.
       Answered incorrectlyPhysician: Maybe you can tell me a little bit about where things are with your mom now, what you're hearing from the doctors, how you're looking at things.

      Case daughter: Yeah, um, well, you know, she's been in here about 12 days now.

      Physician: Yeah.

      Case daughter: A long time. And, uh, this is very unexpected by all of us. And I know she's on antibiotics for the pneumonia and for the sepsis that she seemed to develop, but that's under control. And it was good news to hear that she didn't have to go on dialysis.

      Physician: Yeah. Yep, yep.

      Case daughter: Right? And they might have cleared up a sort of kidney infection along the way and that's good.
      The case provided details about the medical history. The case daughter was supposed to adhere to them. This was incorrect because the case patient did not have a urinary tract infection or acute kidney injury/any indication for dialysis.
      Treatment plans
       Answered correctlyPhysician: And you mentioned that you had had some conversations with her about critical care and what she would want. What kind of things had she said?

      Case daughter: Well, she said that like after what my dad had gone through, she said that like having a feeding tube for the rest of her life would not be something that she would want to have done.
      The case daughter was required to directly answer a question about what the patient would want by saying she didn't know, or that the patient would not want a tracheostomy/feeding tube.
       VolunteeredPhysician: But because of her kidneys being a little bit hurt in the process—most of the medicines are filtered through the kidneys, and some patients like her take a little bit longer to wake up. So it's expected that she will take a little longer than a younger person, and a little longer than someone with normal kidney function.

      Case daughter: Yeah, it would be really um, great to be able to talk to her about all this.

      Physician: Right.

      Case daughter: Um, I'm not exactly sure how … how she would feel about [sighs lightly] some of these things we were talking about.
      The case daughter was only supposed to talk about what the patient might want if the physician asked. Here, she volunteered that she didn't know how her mom would feel, which violates the rule.
       Didn't sharePhysician: And what you're telling me about what she said about your dad and watching your dad's illness, it sounds like that's not really an OK alternative or choice for her. So then the question is, what do we do if we know that this option is not OK with her, it's not an OK quality of life? Does that make sense?

      Case daughter: I guess I just wanna ask again, I mean, is there any ounce of hope … we wouldn't have to go that route?
      The case daughter was required to directly answer a question about what the patient would want by saying she didn't know, or that the patient would not want a tracheostomy/feeding tube. Here, she changed the subject to ask about prognosis and express emotions.
       Answered incorrectlyPhysician: Do you have any thoughts on whether we should go forward with the tracheostomy at this point?

      Case daughter: At this point … if there's a chance, I would think that my mom would say, “there's a chance, do it, then see what happens.”
      The case daughter was required to directly answer a question about what the patient would want by saying she didn't know, or that the patient would not want a tracheostomy/feeding tube. She could never say the patient would want a tracheostomy. She violated that rule here.

      Appendix A. Supplementary data

      References

        • Shalowitz D.I.
        • Garrett-Mayer E.
        • Wendler D.
        The accuracy of surrogate decision makers: a systematic review.
        Arch Intern Med. 2006; 166: 493-497
        • Smucker W.D.
        • Houts R.M.
        • Danks J.H.
        • Ditto P.H.
        • Fagerlin A.
        • Coppola K.M.
        Modal preferences predict elderly patients' life-sustaining treatment choices as well as patients' chosen surrogates do.
        Med Decis Mak. 2000; 20: 271-280
        • Frey R.
        • Hertwig R.
        • Herzog S.M.
        Surrogate decision making: do we have to trade off accuracy and procedural satisfaction?.
        Med Decis Mak. 2013; 34: 258-269
        • Heyland D.K.
        • Dodek P.
        • Rocker G.
        • et al.
        What matters most in end-of-life care: perceptions of seriously ill patients and their family members.
        CMAJ [Internet]. 2006; 174: 627-633
        • Barnato A.E.
        • Herndon M.B.
        • Anthony D.L.
        • et al.
        Are regional variations in end-of-life care intensity explained by patient preferences?: a Study of the US Medicare Population.
        Med Care. 2007; 45: 386-393
        • Committee on quality of health care in America
        Crossing the Quality Chasm: A New Health System for the 21st Century.
        National Academies Press, Washington, D.C.2001: 337
        • Scheunemann L.P.
        • Arnold R.M.
        • White D.B.
        The facilitated values history: helping surrogates make authentic decisions for incapacitated patients with advanced illness.
        Am J Respir Crit Care Med. 2012; 186: 480-486
        • Chiarchiaro J.
        • Ernecoff N.C.
        • Scheunemann L.P.
        • et al.
        Physicians rarely elicit critically ill patients' previously expressed treatment preferences in intensive care units.
        Am J Respir Crit Care Med. 2017; 196: 242-245
        • Scheunemann L.P.
        • Cunningham T.V.
        • Buddadhumaruk P.
        • Arnold R.M.
        • White D.B.
        How clinicians discuss critically ill patients' preferences and values with surrogates: an empirical analysis.
        Crit Care Med. 2015; 43: 757-764
        • Littlewood K.E.
        High fidelity simulation as a research tool.
        Best Pract Res Clin Anaesthesiol [internet]. 2011; 25 (Available from:): 473-487
        • McGaghie W.C.
        • Issenberg S.B.
        • Cohen E.R.
        • Barsuk J.H.
        • Wayne D.B.
        Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence.
        Acad Med. 2011; 86: 706-711
        • Barsuk J.H.
        • McGaghie W.C.
        • Cohen E.R.
        • O'Leary K.J.
        • Wayne D.B.
        Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit.
        Crit Care Med. 2009; 37: 2697-2701
        • Mariani B.
        • Doolen J.
        Nursing simulation research: what are the perceived gaps?.
        Clin Simul Nurs [internet]. 2016; 12: 30-36
        • Jeffries P.R.
        A framework for designing, implementing, and evaluating simulations as teaching strategies.
        Nurs Educ Perspect. 2005; : 96-103
        • Valentin E.C.
        Guidelines for designing simulation building blocks.
        2002: 563-571
        • Galetta A.
        Mastering the semi-structured interview and beyond.
        New York University Press, New York2013
        • Hawkins N.A.
        • Ditto P.H.
        • Danks J.H.
        • Smucker W.D.
        Micromanaging death: process preferences, values, and goals in end-of-life medical decision making.
        Gerontologist. 2005; 45: 107-117
        • Patrick D.L.
        • Starks H.E.
        • Cain K.C.
        • Uhlmann R.F.
        • Pearlman R.A.
        Measuring preferences for health states worse than death.
        Med Decis Mak. 1994; 14: 9-18
        • Karel M.J.
        • Moye J.
        • Bank A.
        • Azar A.R.
        Three methods of assessing values for advance care planning: comparing persons with and without dementia.
        J Aging Health. 2007; 19: 123-151
        • Barnato A.E.
        • Hsu H.E.
        • Bryce C.L.
        • et al.
        Using simulation to isolate physician variation in ICU admission decision making for critically ill elders with end-stage cancer: a pilot feasibility study.
        Crit Care Med. 2008; 36: 3156-3163
        • Uy J.
        • White D.B.
        • Mohan D.
        • Arnold R.M.
        • Barnato A.E.
        Physicians' decision-making roles for an acutely unstable critically and terminally ill patient.
        Crit Care Med. 2013; 41: 1511-1517
        • Chiarchiaro J.
        • Schuster R.A.
        • Ernecoff N.C.
        • Barnato A.E.
        • Arnold R.M.
        • White D.B.
        Developing a simulation to study conflict in intensive care units.
        Ann Am Thorac Soc. 2015; 12: 526-532
        • Chiarchiaro J.
        • White D.B.
        • Ernecoff N.C.
        • Buddadhumaruk P.
        • Schuster R.A.
        • Arnold R.M.
        Conflict management strategies in the ICU differ between palliative care.
        Specialists and Intensivists. 2016; 44: 934-942
        • Garland A.
        • Connors A.F.
        Physicians' influence over decisions to forego life support.
        J Palliat Med. 2007; 10: 1298-1305
        • Curtis J.R.
        • Barnato A.E.
        Variability in decisions to limit life sustaining treatment is it all about the physician chest.
        Chest. 2014; 146: 532-534
        • Frost D.W.
        • Cook D.J.
        • Heyland D.K.
        • Fowler R.A.
        Patient and healthcare professional factors influencing end-of-life decision-making during critical illness: a systematic review.
        Crit Care Med [internet]. 2011; 39: 1174-1189
        • Abbo E.D.
        • Yuen T.C.
        • Buhrmester L.
        • et al.
        Physicians' influence over decisions to forego life support.
        Crit Care Med [Internet]. 2013; 32 (Second ed. New York, NY: Cambridge University Press): 48-53
        • Davidson J.E.
        • Powers K.
        • Hedayat K.M.
        • et al.
        Clinical practice guidelines for support of the family in the patient-centered intensive care unit: American College of Critical Care Medicine Task Force 2004-2005.
        Crit Care Med. 2007; 35: 605-622
      1. Clinical Practice Guidelines for Quality Palliative Care. Second. National Consensus Project for Quality Palliative Care, Pittsburgh, Pennsylvania2009
      2. Crabtree B.F. Miller W.L. Doing Qualitative Research. 2nd ed. Sage Publications, Inc, Thousand Oaks, California1999
        • Kon A.A.
        • Davidson J.E.
        • Morrison W.
        • Danis M.
        • White D.B.
        Shared decision making in ICUs.
        Crit Care Med [internet]. 2016; 44: 188-201
        • Stewart M.
        • Brown J.B.
        • Donner A.
        • et al.
        The impact of patient-centered care on outcomes.
        J FamPract. 2000; 49: 796-804
        • White D.B.
        • Cua S.M.
        • Walk R.
        • et al.
        Nurse-led intervention to improve surrogate decision making for patients with advanced critical illness.
        Am J Crit Care. 2012; 21: 396-409
        • Back A.L.
        • Arnold R.M.
        • Baile W.F.
        • et al.
        Efficacy of communication skills training for giving bad news and discussing transitions to palliative care.
        Arch Intern Med. 2007; 167: 453-460
        • Arnold R.M.
        • Back A.L.
        • Barnato A.E.
        • et al.
        The Critical Care Communication project: improving fellows' communication skills.
        J Crit Care [internet]. 2015; 30 (Elsevier Inc.): 250-254
        • Mohan D.
        • Alexander S.C.
        • Garrigues S.K.
        • Arnold R.M.
        • Barnato A.E.
        Communication practices in physician decision-making for an unstable critically ill patient with end-stage cancer.
        J Palliat Med. 2010; 13: 949-956
        • Munroe B.
        • Buckley T.
        • Curtis K.
        • Morris R.
        Designing and implementing full immersion simulation as a research tool.
        Australas Emerg Nurs J [Internet]. Coll Emerg Nurs Australasia. 2016; 19: 90-105