If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Address correspondence to: Rashmi K. Sharma, MD, MHS, Division of General Internal Medicine, University of Washington, 1959 NE Pacific Street, Campus Box #356526, Seattle, WA 98195, USA.
Division of Hospital Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USASection of Palliative Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
Division of Hospital Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USASection of Palliative Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
Division of Hospital Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USASection of Palliative Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USADepartment of Medical Education, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
Although code status discussions (CSD) occur frequently in the hospital setting, discussions often lack content necessary for informed decision making. Simulation-based mastery learning (SBML) has been used to improve clinical skills among resident physicians and may provide a novel way to improve hospitalists' CSD skills.
Objectives
The objective of this pilot randomized controlled trial was to develop and evaluate a CSD SBML intervention for hospitalists.
Methods
Twenty hospitalists were randomized to control vs. a CSD SBML intervention. Hospitalists conducted a baseline standardized patient encounter (pretest) that was scored using a 19-item CSD checklist and controls completed a repeat standardized patient encounter six months later (post-test). Intervention group hospitalists received at least one two-hour training session featuring deliberate practice and feedback and were expected to meet a minimum passing score (MPS) on the post-test of 84% set by an expert panel.
Results
Only two of the 20 hospitalists met the MPS at pretest. Seventy percentage of intervention hospitalists achieved the MPS after a single training session. Post-test median checklist scores were higher for intervention hospitalists compared with controls (16.5 vs. 12.0, P = 0.0001). Intervention hospitalists were significantly more likely to ask about previous experiences with end-of-life decision making (70% vs. 20%, P = 0.03), explore values/goals (100% vs. 50%, P = 0.01), ask permission to make a recommendation regarding code status (60% vs. 0%, P = 0.003), and align recommendations with patient values/goals (90% vs. 40%, P = 0.02) than controls.
Conclusion
Few hospitalists demonstrated mastery of CSD skills at baseline; SBML was an effective way to improve these skills.
Physician-patient code status discussions (CSDs) are a routine part of inpatient care particularly in the setting of serious illness. Unfortunately, CSDs led by resident and practicing physicians often lack exploration of the patient's values and goals, understanding of disease severity and prognosis, and discussion about resuscitation outcomes.
Communication and decision making about life-sustaining treatment: examining the experiences of resident physicians and seriously-ill hospitalized patients.
This approach has been successfully used in procedural training (e.g., insertion of central venous catheters) to improve both educational and patient outcomes.
Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence.
The novel application of SBML principles to CSD education may provide a standardized way to achieve proficiency while providing time to practice skills in a safe environment.
In academic medical centers, where hospitalists frequently supervise residents on inpatient teaching services, suboptimal communication may not only negatively affect patient outcomes but also has the potential to affect resident communication skills. Training of hospitalists might improve their skills and also reinforce skills taught to residents. However, SBML has primarily been used with trainees, and it is not known whether CSD SBML would be feasible and effective among hospitalists at an academic medical center. The aim of this study was to develop a CSD SBML intervention for hospitalists and conduct a pilot randomized controlled trial to evaluate the impact of the program on hospitalists' CSD skills.
Methods
Study Design and Participants
We conducted a pilot randomized controlled trial of a CSD SBML educational intervention in hospitalists who attend on the inpatient medicine teaching service at Northwestern Memorial Hospital (NMH), an 885 bed tertiary care hospital in Chicago, Illinois. The study ran from October 2014 to January 2016 and was approved by the Northwestern University Institutional Review Board. All participants provided written informed consent.
Checklist Evaluation and Standard Setting Process
We used a modified version of a 19-item checklist previously developed for resident assessment of CSD skills.
To pilot the checklist, we asked a sample of 10 palliative care physicians to conduct a simulated CSD with a trained standardized patient portraying a hospitalized man with advanced colorectal cancer. Based on these results, hospitalists were provided up to 30 minutes to complete their simulated encounter.
A multidisciplinary panel of 10 board-certified physicians and one nurse representing palliative medicine (4), internal medicine (4), hematology/oncology (1), pulmonary/critical care medicine (1), and emergency medicine (1) determined the CSD MPS. Panelists received instruction in standard setting and used the Angoff (item based) and Hofstee (group based) methods to assign pass/fail standards.
The mean of the Angoff and Hofstee scores (84% or 16 of 19) was used as the final MPS.
Study Procedure
Eligible participants were NMH hospitalists who attended on the inpatient medicine teaching service at least once per year. Hospitalists were randomized to either the control group, which consisted of usual clinical practice, or intervention group, which included didactics, role-play, and feedback from a palliative care physician. All participants completed a simulated patient encounter at baseline (pretest) and six months later (posttest). The intervention was adapted from previous CSD SBML interventions and is described in detail elsewhere.
Briefly, intervention group participants were given reading materials reviewing key concepts in CSDs along with the scored checklist and video from their pretest discussion one week before their training session. They then met individually with one of three palliative care faculty members to review the videotape and practice CSD skills with the standardized patient before conducting their simulated CSD posttest. Training sessions lasted approximately two hours. Consistent with the mastery model, intervention group participants who did not meet or exceed the MPS at post-test completed additional deliberate practice with feedback and retesting until the MPS was achieved. A single rater scored all the simulated CSDs. A second rater rescored a 50% random sample of the video-recorded CSDs to assess inter-rater reliability.
Analysis
We used the chi-square tests and student t-tests to compare demographic factors between control and intervention groups. Chi-square tests were conducted to compare individual checklist items between groups. We used the Wilcoxon rank-sum test to compare median checklist scores between groups at pretest and post-test. All analyses were conducted using Stata, version 11.0 (StataCorp, College Station, TX).
Results
Forty-eight hospitalists were eligible to participate. Twenty (42%) consented and completed the entire protocol. Mean age of participants was 37.2 ± 5.1 years and mean years in practice were 8.0 ± 4.9. There were no statistically significant differences in demographic characteristics between the intervention and control groups (Table 1).
Table 1Study Participant Demographics
Participant Characteristics
Control Group (n = 10)
Intervention Group (n = 10)
Age, yrs, mean (SD)
37.3 (5.0)
37.0 (5.5)
Female gender (%)
50
50
Race/ethnicity (%)
Non-Hispanic white
40
60
Asian/Pacific Islander
40
30
Black
20
10
Years in practice, mean (SD)
8.6 (4.9)
7.4 (5.0)
Previously received formal code status communication skills training (% yes)
20
30
There were no statistically significant differences between the control and intervention groups for any of these characteristics.
Inter-rater reliability was high for checklist items (mean kappa 0.82). Median scores on the pretest were 11 of 19 (interquartile range 10-12) for the intervention and 12 of 19 (interquartile range 11, 13) for the control group (Fig. 1). Only two hospitalists (10%), both randomly assigned to the intervention group, met the MPS of 84% on the pretest. On the post-test, intervention group median scores were significantly higher than control scores (16.5 of 19 vs. 12.0 of 19, P = 0.0001). Seventy percentage of intervention hospitalists achieved the MPS after a single two-hour training session. The remaining 30% achieved the MPS after a second two-hour training session. No control group hospitalists achieved the MPS on either the pretest or post-test (Fig. 1).
Fig. 1Median pretest vs. posttest checklist scores by study group.
Table 2 displays the percentage of hospitalists in each group that correctly completed each checklist item at post-test. Intervention group hospitalists were significantly more likely to complete the following items than controls: ask about previous experiences with end-of-life decision making (70% vs. 20%, P = 0.03), explore values/goals (100% vs. 50%, P = 0.01), ask permission to make a recommendation regarding code status (60% vs. 0%, P = 0.003), and align recommendations with the patient's values/goals (90% vs. 40%, P = 0.02).
We found that few practicing hospitalists demonstrated mastery of CSD skills at baseline, and only those provided SBML met the MPS at post-test. Training ensured that hospitalists reliably explored patient's previous experiences with end-of-life decision making, asked about a health care power of attorney, and aligned recommendations about code status with the patient's values and goals. These results were achieved in less than four hours of training per participant (2.6 hours on average). Our findings suggest that despite frequently caring for seriously-ill patients, experienced hospitalists may need to improve their CSD skills and that SBML ensures that they can do so competently.
Beyond ensuring that hospitalists have the skills to facilitate CSDs, we must also ensure that academic hospitalists are in the best position to train resident physicians. Academic hospitalists frequently supervise residents on inpatient teaching services and residents rate the knowledge, teaching, and feedback of hospitalists higher than other faculty.
As many hospitalists perform CSDs themselves, they are particularly well suited to role model skills and provide feedback to residents. Training residents to master CSDs but not ensuring that supervising faculties are equally competent may negatively affect resident skill retention and reinforce suboptimal communication behaviors.
Although training supervising faculty is a worthy goal, our study illustrates several challenges to this approach. First, slightly less than half of eligible hospitalists agreed to participate in the study. Although being observed and receiving feedback are expected in the course of medical training, faculty members may be less comfortable being rated. Second, coordinating schedules between full-time hospitalists, the standardized patient, and the trainers for the intervention group was challenging and took almost 16 months to complete. Our results suggest that incorporating rigorous communication skills training into the busy clinical schedules of practicing hospitalists may require alternative incentives or strategies such as the provision of protected education time or offering continuing medical education credit.
Our study has several limitations. First, we had a small sample size of volunteers from a single academic medical center. Second, despite our efforts to create a realistic simulation experience, we acknowledge that hospitalists may have performed differently in actual clinical encounters. However, we postulate that being observed would have likely made participants use better communication techniques, which, if anything, would have increased checklist scores. Third, the checklist, although used in earlier published research,
may not completely define a high-quality CSD that is acceptable to patients. Fourth, participants reviewed the checklist as part of the training which may affect external validity. Although this is a possibility in any simulation-based intervention, this does not diminish the profound skills improvement demonstrated after rigorous SBML. Although the same case was used for the pre- and post-test, the scoring rubric focused on specific communication skills that would likely not differ with a different clinical scenario. Fifth, the first rater was present at the training sessions and, thus, not blinded to study group assignment on the post-tests which may have biased scores. However, inter-rater reliability was high, making this less likely. Finally, we have not yet studied hospitalist skill retention although this has been demonstrated in residents.
Hospitalists in this study displayed poor baseline CSD communication skills. SBML was highly effective in boosting skills in this important area. Further research is needed to evaluate associations between clinician skill level and downstream patient outcomes and between supervising faculty CSD skills and resident performance.
Disclosures and Acknowledgments
This study was supported by an Augusta Webster Educational Innovation Grant from the Northwestern University Department of Medical Education. Dr. Sharma is supported by an American Cancer Society Mentored Research Scholar Grant (MRSG 14–058-01-PCSM). The authors have no conflicts of interest to disclose.
References
Gehlbach T.G.
Shinkunas L.A.
Forman-Hoffman V.L.
et al.
Code status orders and goals of care in the medical ICU.
Communication and decision making about life-sustaining treatment: examining the experiences of resident physicians and seriously-ill hospitalized patients.
Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence.
31242-8&id=gr1.jpg){kind=link}