Advertisement

Determining goal concordant care in the intensive care unit using Electronic Health Records

Open AccessPublished:November 15, 2022DOI:https://doi.org/10.1016/j.jpainsymman.2022.11.002

      Abstract

      Background

      Goal concordant care (GCC) is the alignment of care to patient values and preferences. GCC is a major outcome of communication with patients and families in serious/critical illness. Using the electronic health record (EHR) to study the provision of GCC would be pragmatic and cost-effective for research and quality improvement efforts.

      Research question

      Do EHRs contain information to identify GCC?

      Methods

      This is a feasibility retrospective chart review performed by two independent reviewers. An existing framework containing 4 questions for identifying GCC was adopted. Two clinicians reviewed multi-disciplinary notes and extracted pertinent information. The primary outcomes were whether the 4 key questions for determining goal concordance could be answered using information in the EHR. The secondary outcome was the type of goals identified. Cohen's kappa was used to measure agreement between two reviewers.

      Results

      Patient care was considered goal concordant in 35 (85%) of 41 patients in a random sample comprising of 36 survivors and 5 who died in hospital. Inter-rater agreement on identifying data to determine GCC was excellent (Kappa 0.70). Patient goals were identified in 80% of charts reviewed. Note sources informative of patient preferences, included social work (39%), hospital progress notes (29%), palliative care (20%), and physical/occupational therapy (15%). ‘Returning home’ and ‘getting better/ stronger’ were among the most common patient goals captured in EHR.

      Conclusion

      The EHR can be used to understand patient goals, but the information is scattered across the multi-disciplinary notes. Improving EHR and external validation will facilitate ascertainment of goal concordance as an important outcome measure.

      Key message

      : This feasibility study demonstrates that EHR likely contains sufficient information to assess patient goals for most patients. A framework of questions to structure EHR review was helpful in this process.

      Keywords

      Introduction

      Patients, families, and providers often differ in their goals and expectations of outcomes following critical illness. Knowing patient's goals and values is vital to delivering appropriate and individualized care. Patient's goals for quality of life, physical functioning, and family and social engagement can vary broadly; what is acceptable for some may be “worse than death” for others [
      • McPeake JM
      • Harhay MO
      • Devine H
      • Iwashyna TJ
      • MacTavish P
      • Mikkelsen M
      • Shaw M
      • Quasim T
      Exploring Patients' Goals Within the Intensive Care Unit Rehabilitation Setting.
      ,
      • Rubin EB
      • Buehler AE
      • Halpern SD
      States Worse Than Death Among Hospitalized Patients With Serious Illnesses.
      ]. Care designed to maximally meet a patient goal is termed Goal Concordant Care (GCC). GCC is likely to be as important for patients who are expected to survive the hospitalization as well as for those expected to die and is a preferred outcome for evaluating advance care planning interventions and communication interventions in the setting of the intensive care unit (ICU) [
      • Sanders JJ
      • Miller K
      • Desai M
      • Geerse OP
      • Paladino J
      • Kavanagh J
      • Lakin JR
      • Neville BA
      • Block SD
      • Fromme EK
      • et al.
      Measuring Goal-Concordant Care: Results and Reflections From Secondary Analysis of a Trial to Improve Serious Illness Communication.
      ,
      • Modes ME
      • Heckbert SR
      • Engelberg RA
      • Nielsen EL
      • Curtis JR
      • Kross EK
      Patient-Reported Receipt of Goal-Concordant Care Among Seriously Ill Outpatients-Prevalence and Associated Factors.
      ,
      • Turnbull AE
      • Hartog CS
      Goal-concordant care in the ICU: a conceptual framework for future research.
      ,
      • Sudore RL
      • Heyland DK
      • Lum HD
      • Rietjens JAC
      • Korfage IJ
      • Ritchie CS
      • Hanson LC
      • Meier DE
      • Pantilat SZ
      • Lorenz K
      • et al.
      Outcomes That Define Successful Advance Care Planning: A Delphi Panel Consensus.
      ,
      • Cooper Z
      • Koritsanszky LA
      • Cauley CE
      • Frydman JL
      • Bernacki RE
      • Mosenthal AC
      • Gawande AA
      • Block SD
      Recommendations for Best Communication Practices to Facilitate Goal-concordant Care for Seriously Ill Older Patients With Emergency Surgical Conditions.
      ,
      • Narang AK
      • Wright AA
      • Nicholas LH
      Trends in Advance Care Planning in Patients With Cancer: Results From a National Longitudinal Survey.
      ]. Treatment inconsistent with patient goals results in worse ratings on care, pain management, and communication with clinician among patients at the end of life [
      • Khandelwal N
      • Curtis JR
      • Freedman VA
      • Kasper JD
      • Gozalo P
      • Engelberg RA
      • Teno JM
      How Often Is End-of-Life Care in the United States Inconsistent with Patients' Goals of Care?.
      ].
      Measuring GCC is challenging, as there is no gold standard methodology. Caution has been suggested when attempting to reliably measure ‘care consistent with goals’ [
      • Sudore RL
      • Heyland DK
      • Lum HD
      • Rietjens JAC
      • Korfage IJ
      • Ritchie CS
      • Hanson LC
      • Meier DE
      • Pantilat SZ
      • Lorenz K
      • et al.
      Outcomes That Define Successful Advance Care Planning: A Delphi Panel Consensus.
      ,
      • Bernacki R
      • Hutchings M
      • Vick J
      • Smith G
      • Paladino J
      • Lipsitz S
      • Gawande AA
      • Block SD
      Development of the Serious Illness Care Program: a randomised controlled trial of a palliative care communication intervention.
      ], because patients often have multiple competing goals that may change during the course of their illness [
      • Berntsen GK
      • Gammon D
      • Steinsbekk A
      • Salamonsen A
      • Foss N
      • Ruland C
      • Fonnebo V
      How do we deal with multiple goals for care within an individual patient trajectory? A document content analysis of health service research papers on goals for care.
      ]. In serious illness, different ways of measuring goal concordance have included comparison of prospectively documented patient preferences with end-of life (EOL) health resources utilization, comparison of patient preferences with patient assessment of goal concordance, and retrospective assessment of GCC from bereaved caregivers [
      • Modes ME
      • Heckbert SR
      • Engelberg RA
      • Nielsen EL
      • Curtis JR
      • Kross EK
      Patient-Reported Receipt of Goal-Concordant Care Among Seriously Ill Outpatients-Prevalence and Associated Factors.
      ,
      • Glass DP
      • Wang SE
      • Minardi PM
      • Kanter MH
      Concordance of End-of-Life Care With End-of-Life Wishes in an Integrated Health Care System.
      ]. Using a life-priority survey to help identify what goals matter most to patients is another way to ascertain the goals of patients with serious or critical illness [
      • Sanders JJ
      • Miller K
      • Desai M
      • Geerse OP
      • Paladino J
      • Kavanagh J
      • Lakin JR
      • Neville BA
      • Block SD
      • Fromme EK
      • et al.
      Measuring Goal-Concordant Care: Results and Reflections From Secondary Analysis of a Trial to Improve Serious Illness Communication.
      ]. Another approach uses a framework developed by Turnbull et al. focusing on whether patient goals and limits are known, considered achievable by the clinical team and whether the patient receives treatment expected to help accomplish those goals [
      • Turnbull AE
      • Hartog CS
      Goal-concordant care in the ICU: a conceptual framework for future research.
      ,
      • Turnbull AE
      • Sahetya SK
      • Colantuoni E
      • Kweku J
      • Nikooie R
      • Curtis JR
      Inter-Rater Agreement of Intensivists Evaluating the Goal Concordance of Preference-Sensitive ICU Interventions.
      ]. This framework has been used previously to assess whether preference-sensitive interventions in critical illness such as feeding tubes and tracheostomy were goal-concordant for individual patients [
      • Turnbull AE
      • Sahetya SK
      • Colantuoni E
      • Kweku J
      • Nikooie R
      • Curtis JR
      Inter-Rater Agreement of Intensivists Evaluating the Goal Concordance of Preference-Sensitive ICU Interventions.
      ].
      Methods for evaluating goal concordance using the electronic health record (EHR) have been proposed [
      • Turnbull AE
      • Hartog CS
      Goal-concordant care in the ICU: a conceptual framework for future research.
      ,
      • Halpern SD
      Goal-Concordant Care - Searching for the Holy Grail.
      ]. Some of the challenges in developing a scalable tool to measure GCC in the EHR includes documentation of patient goals, reliable classification by reviewers of goals being concordant or not, training of researchers, and developing an algorithm using natural language processing (NLP) to help review EHR notes [
      • Halpern SD
      Goal-Concordant Care - Searching for the Holy Grail.
      ]. If demonstrated to be feasible, these methods could enable clinical researchers to use GCC as an outcome in serious/critical illness. Therefore, this is a feasibility study to determine if the EHR contains sufficient information about patient goals and key features to determine goal concordance.

      Methods

      Study design

      This study is a secondary analysis of a single center retrospective review of a random sample of ICU patients treated between 2015 and 2019 at the Mayo Clinic in Rochester, Minnesota exploring changes in health trajectory before and after critical illness [
      • Ahmad SR TA
      • Budahn L
      • Lemahieu AM
      • Anderson B
      • Vashistha K
      • Karnatovskaia L
      • Gajic O
      Feasibility of Extracting Meaningful Patient Centered Outcomes From the Electronic Health Record Following Critical Illness in the Elderly.
      ]. The study was approved by the Institutional Review Board. The EHR system was changed to Epic in May of 2018. EHRs of patients prior to 2018 were transitioned to Epic. The Anesthesia and Critical Care Research Unit (ACRU) at the Mayo Clinic assisted with data extraction and with manual chart review.
      A REDCap data base was created to collect patient demographics, diagnosis, and specific ICU interventions received. The REDCap data collection questions pertaining to patient goals were adapted from Turnbull et al. [
      • Turnbull AE
      • Hartog CS
      Goal-concordant care in the ICU: a conceptual framework for future research.
      ] and included 1) are patient goals known, 2) are limitations to treatment known, 3) are goals achievable or realistic, and 4) were goals achieved. The term ‘goal’ is inclusive of patient specific values and preferences.
      We reviewed EHR charts to identify the types of patient goals expressed, and to answer questions related to GCC. We focused on the notes from the ICU stay and the associated hospital encounter but also reviewed those before or after that encounter. Charts were continually revisited until the relevant content was identified by both reviewers. This review was undertaken with a key assumption that what was stated in the chart is an accurate representation of a patient's true goals.
      Two reviewers, SRA and LB, a physician, and a nurse, completed 41 chart reviews. Using the framework of questions by Turnbull et al. each reviewer conducted independent chart analysis and shared challenges and progress. We studied notes and identified high yield areas where information about patient goals were found. We compared subjective impression of each chart in the recurring meetings. Any differences were resolved by a third physician reviewer, ADT. If after reviewing a variety of note sources, we were able to answer ‘yes’ to the 4 questions, we concluded care to be goal concordant.

      Participants

      Patients over 65 years old admitted to the ICU between August of 2015 and August 2019 for greater than 24 hours and requiring one or more of the following: high-flow nasal cannula, bilevel or continuous positive pressure ventilation, invasive mechanical ventilation, and vasopressor support were eligible for inclusion in this study. The patients were residents of Olmsted County, Minnesota, with research authorization on file. Patients in a vegetative state at baseline or with intubation for elective surgical procedures only were excluded. In total, 41 patient chart reviews were completed in the primary analysis, with 23 having an index ICU admission prior to the introduction of Epic and 18 after. Charts were revisited and studied to extract goals in free text identified by each reviewer (see Figure 1).
      Figure 1:
      Figure 1Consort diagram of patient chart review

      Outcomes

      The primary outcome was ability to determine GCC using EHR. The secondary outcome was the type of patient preference identified.

      Statistical analysis

      Patient characteristics, interventions, and outcomes were summarized using median and interquartile range (IQR) for continuous variables and frequency counts and percentages for categorical variables. Cohen's kappa coefficient was used to measure agreement between the two reviewers regarding GCC and patient preferences [
      • McHugh ML
      Interrater reliability: the kappa statistic.
      ].
      Data management and statistical analysis were performed in SAS Studio 3.8 (SAS Institute Inc, Cary, North Carolina) and Microsoft Excel 365 (Microsoft Corporation, Redmond, WA).

      Results

      This secondary analysis included 41 randomly selected patients, among whom 36 survived their hospital stay and 5 died during the hospitalization (Table 1). All but two patients had no pre-existing mechanical ventilation, tracheostomy, or feeding tube. No patient had prior renal replacement therapy in this sample. Common ICU interventions included mechanical ventilation and vasopressor use as per our inclusion criteria. Family meetings were identified in about half of the charts (N=22), and palliative care consults were identified in 24% (N = 10) of charts (Table 2).
      Table 1Patient demographics and pre-ICU characteristics
      CharacteristicsOverall (N=41)
      Gender, n (%)
      Female12 (29%)
      Male29 (71%)
      Age, median (Q1, Q3)75.0 (69.0, 84.0)
      Pre-ICU residence, n (%)
      Assisted living5 (12%)
      Home30 (73%)
      Nursing home5 (12%)
      Short-term Nursing facility1 (2%)
      Pre-existing chronic mechanical ventilation, n (%)1 (2%)
      Pre-existing tracheostomy, n (%)1 (2%)
      Pre-existing feeding tube, n (%)1 (2%)
      Chronic hemodialysis, n (%)0 (0%)
      ICU Interventions for study inclusion
      Mechanical ventilation (new), n (%)33 (80%)
      Non-invasive mechanical ventilation (new), n (%)16 (48%)
      Vasopressor support, n (%)29 (71%)
      Hospital outcomes
      Status at hospital discharge, n (%)
      Alive36 (88%)
      Dead5 (12%)
      Hospital length of stay, median (Q1, Q3)12.0 (7.0, 17.5)
      Status at ICU discharge, n (%)
      Alive39 (95%)
      Dead2 (5%)
      ICU length of stay, median (Q1, Q3)5.0 (2.3, 7.0)
      Table 2ICU Interventions
      InterventionsOverall (N=41)
      Mechanical ventilation (new), n (%)33 (80%)
      Non-invasive mechanical ventilation (new), n (%)16 (48%)
      Vasopressor support, n (%)29 (71%)
      Family meetings, n (%)
      Yes22 (55%)
      No12 (30%)
      Unable to determine6 (15%)
      Palliative care consult, n (%)10 (24%)
      Ethics consult, n (%)3 (7%)
      We tested whether information could be extracted during chart review to answer the four questions proposed by the framework from Turnbull et al. for assessing the presence of goal concordant care. [
      • Turnbull AE
      • Hartog CS
      Goal-concordant care in the ICU: a conceptual framework for future research.
      ]
      Of the notes eliciting patient preferences, those from social work were most informative (16, 39%) followed by progress notes inclusive of advanced care planning notes (12, 29%), palliative care (8, 20%), and physical/occupational therapy (6, 15%). One patient had preferences documented in a chaplain note and one in the ethics note. Eight patients had no documentation of their preferences; 12 patients had preferences documented in more than one type of note. Overall, we did not find one consistent spot to locate ‘patient goals’, even for charts reviewed after the EHR change.
      An example is a case of an elderly man with a do-not-resuscitate and do- not intubate order admitted for empyema and subsequent hemorrhagic shock. He received a feeding tube during hospitalization. Social work notes documented patient goals as ‘returning home’ and ‘getting stronger’. The EHR revealed home discharge after ICU/ hospital course and short-term rehabilitation with no recurrent hospitalizations. In this case, when applying the framework of GCC questions, we answered that goals were known, limitations were known, goals were realistic, and goals were achieved. Hence, care in the ICU/ hospital course was considered goal concordant.
      We had at least substantial agreement between the two reviewers on whether care was goal concordant, partially concordant, not concordant, or concordance could not be determined (Table 3). There was almost perfect inter-rater agreement in answering most questions except for the impression on whether the ‘goals were achievable/realistic’. While the agreement was substantial, there was still uncertainty in 7 (18%) charts.
      Table 3Goal Concordance in EHR
      EHR review to answer questions of goal concordanceReviewer 1 (N=41)Reviewer 2 (N=41)Kappa
      Patient goals documented, n (%)0.926
      Yes33 (80%)32 (78%)
      No8 (20%)9 (22%)
      Limitations to treatment known, n (%)0.950
      Yes17 (41%)18 (44%)
      No24 (59%)23 (56%)
      Goals achievable/ realistic, n (%)0.807
      Yes30 (75%)32 (80%)
      No3 (8%)3 (8%)
      Maybe7 (18%)5 (13%)
      Goal achieved, n (%)1.000
      Yes32 (80%)32 (80%)
      No8 (20%)8 (20%)
      Goal concordant care per documentation, n (%)0.701
      Yes35 (85%)36 (88%)
      No2 (5%)0 (0%)
      Partial2 (5%)3 (7%)
      Unable to determine2 (5%)2 (5%)
      Kappa values from 0.81 to 1.00 represent ‘Almost perfect agreement’, 0.61-0.8 represents ‘Substantial agreement’
      Table 4 provides examples of specific preferences expressed which were captured in over 80% of the charts reviewed. ‘Returning home’ and ‘getting better’ were the two most common patient preferences. Some of the social work and physical/occupational therapy notes incorporated a detailed template to capture these goals as well as relevant dialogue.
      Table 4
      Common themes of preferences found in EMR chart reviewReviewer 1

      (N=41)
      Reviewer 2

      (N=41)
      Examples
      Continue with life prolonging measures8 (19.5%)8 (19.5%)-Wants everything done at this moment.

      -Asks that we proceed with aggressive care as necessary to improve outcome.
      Being with family8 (19.5%)8 (19.5%)

      -Her values are to be at home with husband. She values her family, friends, and pets.

      -His ultimate goal is to be home with his beloved wife.
      Return home18 (44%)18 (44%)-“I'm going home one way or another.”

      -“I want to go home and stop coming to the hospital.”
      Symptom control7 (17%)8 (19.5%)-Patients’ family, including his wife, spoke on behalf of patient, with the goal of focusing on comfort and preventing any harm.

      -Goal was pain control.
      Treat with respect1 (2.4%)1 (2.4%)-He had wished to be treated kindly amongst others.
      Get better/stronger/back to baseline12 (29.3%)14 (34.1%)

      -To be mobile and get better.

      -He is understandably frustrated and tearful in the changes he has in his independence due to his surgery.
      Limit hospital care2 (5%)2 (2.4%)

      -He does not want to be intubated or resuscitated long-term, specifically if his outcome is going to be poor.
      Recreation3 (7%)3 (7%)-He wanted to be able to play music again.

      -Patient would like to be able to be out in the snow with his dog.

      -He would like to remain independent with most of his activities and do things such as fishing and camping as best he can.
      Other6 (14.6%)4 (9.8%)-Hope of no longer needing oxygen, “it gets in the way”.

      -Keeping his stress level as low as possible.

      -A meaningful quality of life would consist mostly of the ability to verbally interact in a meaningful way with friends and family.
      No preferences8 (19.5%)9 (22%)
      Eight charts did not contain any specific patient preferences in the ICU/hospitalization data. In those cases, review of patient's clinical trajectory post-ICU and documented outpatient dialogue reflecting provider-patient interaction assisted the reviewers in understanding whether patients received goal concordant care. Here, we assumed that there were no limitations to care and considered goal concordance to be equivalent to anticipated outcomes of medical/surgical care. An example is of a patient needing urgent valvular surgery, needing intensive care, eventually discharged home without long term complications, where during the hospital stay no specific patient preferences were documented.
      There were 2 cases where determination of GCC was challenging, and reviewers differed in their final impression of GCC despite same response to all the 4 questions. This resulted from differing impressions of what the patient may have understood of the medical condition when expressing those goals, and the changes in goals over time. An example is of a patient with an anticipated poor prognosis who originally expressed a preference for ‘life prolongation at all costs’ and ‘not wanting to go to a skilled nursing facility’. Subsequent follow-up notes several months later documented a change in preference to ‘time at home with wife’ and a shift of focus to ‘comfort care’. There were 2 charts where it could not be determined whether GCC took place. No specific goals were found in the chart. Here the category ‘unable to determine’ was applied.

      Discussion

      In this study we demonstrated that important aspects of GCC can be found in EHR in most cases. The framework proposed by Turnbull et al [
      • Turnbull AE
      • Hartog CS
      Goal-concordant care in the ICU: a conceptual framework for future research.
      ] was helpful in structuring the chart review for this purpose.
      Themes of values were easily distinguished in the ‘free text’ in different note sources such as conversations documented by social worker, chaplain, and physical therapists. There is much similarity in themes previously described for patients at the end of life [
      • Sanders JJ
      • Miller K
      • Desai M
      • Geerse OP
      • Paladino J
      • Kavanagh J
      • Lakin JR
      • Neville BA
      • Block SD
      • Fromme EK
      • et al.
      Measuring Goal-Concordant Care: Results and Reflections From Secondary Analysis of a Trial to Improve Serious Illness Communication.
      ]. A new angle that surfaced in our exploration was the impact of socioeconomic determinants of health like challenges with family dynamics, financial constraints, and living conditions on GCC especially in the few cases when reviewers did not agree in their concluding impressions. These aspects were best described in social work notes. Besides patient factors, discrepancy in GCC could also be reflective of quality of communication.
      The meaning and implications of patient ‘goals’ have seen an evolution over the decades as we reform our care strategies to be more patient centric. From the time of the SUPPORT trial (Study to understand prognoses and preferences for outcomes and risks of treatments) in 1995, when addressing code status and symptoms at end of life were brought to general medical attention as important patient goals, clinicians strive to focus on a combination of personal, life and health goals of the critically ill [
      • Sanders JJ
      • Miller K
      • Desai M
      • Geerse OP
      • Paladino J
      • Kavanagh J
      • Lakin JR
      • Neville BA
      • Block SD
      • Fromme EK
      • et al.
      Measuring Goal-Concordant Care: Results and Reflections From Secondary Analysis of a Trial to Improve Serious Illness Communication.
      ,
      • Berntsen GK
      • Gammon D
      • Steinsbekk A
      • Salamonsen A
      • Foss N
      • Ruland C
      • Fonnebo V
      How do we deal with multiple goals for care within an individual patient trajectory? A document content analysis of health service research papers on goals for care.
      ,
      A controlled trial to improve care for seriously ill hospitalized patients. The study to understand prognoses and preferences for outcomes and risks of treatments (SUPPORT). The SUPPORT Principal Investigators.
      ].
      The use of EHR as a patient centered platform is of significant interest, as studies such as ours serve as a call to action for providers to systematically document patient goals, encouraging discussions with patients/their families and thereby fostering communication. EHR can thereby be useful in pragmatic clinical research, particularly in facilitating quality improvement projects in critical care. Generally, patient's goals are expected to be incorporated into the advance care planning (ACP) notes. Recent recommendations to improve ACP notes included incorporating goals of care conversations, undergoing training to improve documentation, having a centralized and consistent location for the information in the note, and including prompts that reflect goals of care into procedure-focused discussions [
      • Lamas D
      • Panariello N
      • Henrich N
      • Hammes B
      • Hanson LC
      • Meier DE
      • Guinn N
      • Corrigan J
      • Hubber S
      • Luetke-Stahlman H
      • et al.
      Advance Care Planning Documentation in Electronic Health Records: Current Challenges and Recommendations for Change.
      ]. Unfortunately, documentation of advance care planning has not been consistent in the EHR [
      • Lamas D
      • Panariello N
      • Henrich N
      • Hammes B
      • Hanson LC
      • Meier DE
      • Guinn N
      • Corrigan J
      • Hubber S
      • Luetke-Stahlman H
      • et al.
      Advance Care Planning Documentation in Electronic Health Records: Current Challenges and Recommendations for Change.
      ,
      • Grudzen CR
      • Buonocore P
      • Steinberg J
      • Ortiz JM
      • Richardson LD
      • Group ARCW
      Concordance of Advance Care Plans With Inpatient Directives in the Electronic Medical Record for Older Patients Admitted From the Emergency Department.
      ]. In our study, we reviewed a variety of note types to identify the ones with highest yield for patient care goals particularly those using ‘free text’. Free text is of particular interest, as the use of natural language processing (NLP) with machine learning has previously been successfully used to identify patient goals from goals of care conversations documented in provider notes, physician order for life-sustaining treatment, and hospice referrals [
      • Lee RY
      • Brumback LC
      • Lober WB
      • Sibley J
      • Nielsen EL
      • Treece PD
      • Kross EK
      • Loggers ET
      • Fausto JA
      • Lindvall C
      • et al.
      Identifying Goals of Care Conversations in the Electronic Health Record Using Natural Language Processing and Machine Learning.
      ]. In an EHR chart review of 679 patients with serious illness from sepsis, clinical documentation looking for goals of longevity, function or comfort were present for 40% of patients with care being considered as goal concordant in 68% [
      • Taylor SP
      • Kowalkowski MA
      • Courtright KR
      • Burke HL
      • Patel S
      • Hicks S
      • Hurley C
      • Mitchell S
      • Halpern SD
      Deficits in Identification of Goals and Goal-Concordant Care After Sepsis Hospitalization.
      ]. Being a feasibility study, the project was not designed to capture prevalence of goal concordance. EHR based applications are being trialed to sniff pertinent data with the aim to overcome unmet palliative care needs and include goal concordance among other outcomes [
      • Cox CE
      • Riley IL
      • Ashana DC
      • Haines K
      • Olsen MK
      • Gu J
      • Pratt EH
      • Al-Hegelan M
      • Harrison RW
      • Naglee C
      • et al.
      Improving racial disparities in unmet palliative care needs among intensive care unit family members with a needs-targeted app intervention: The ICUconnect randomized clinical trial.
      ,
      • Courtright KR
      • Dress EM
      • Singh J
      • Bayes BA
      • Chowdhury M
      • Small DS
      • Hetherington T
      • Plickert L
      • Detsky ME
      • Doctor JN
      • et al.
      Prognosticating Outcomes and Nudging Decisions with Electronic Records in the Intensive Care Unit Trial Protocol.
      ,
      • Cox CE
      • Jones DM
      • Reagan W
      • Key MD
      • Chow V
      • McFarlin J
      • Casarett D
      • Creutzfeldt CJ
      • Docherty SL
      Palliative Care Planner: A Pilot Study to Evaluate Acceptability and Usability of an Electronic Health Records System-integrated, Needs-targeted App Platform.
      ].
      The study has several limitations. First, with chart review studies there is no real time comparison to help validate our findings. It was assumed that EHR documentation was a true reflection of patient goals and discourse between patients and providers. Second, for the charts with absence of documented goals or where goals were unclear, one cannot truly ascertain if communication occurred. Impressions of goal concordance can be challenged for subjectivity by the reviewers. Third, documentation is highly variable across providers and services, and even the ACP notes are not consistent in capturing this dialogue. Even though there is a dedicated area to capture ‘patient goal’ in EPIC, this area remains largely unrecognized. Fourth, over half of study patients’ charts were reviewed just prior to change of institutional EHR. Fifth, manual chart review itself is not efficient for large scale clinical research.
      GCC could evolve as an important measurable outcome for future studies centered on communication in serious and critical illness including in survivors. The study emphasizes the need to improve the EHR to include essential measures of communication as a patient centered outcome with the vision to integrate it with the patient portal system [
      • Dalal AK
      • Dykes P
      • Samal L
      • McNally K
      • Mlaver E
      • Yoon CS
      • Lipsitz SR
      • Bates DW
      Potential of an Electronic Health Record-Integrated Patient Portal for Improving Care Plan Concordance during Acute Care.
      ]. This project can serve to expand the ‘free text’ needed for the natural language processing and machine learning to build an NLP ontology. At this time, the ability of NLP to capture nuanced and dynamic goals needed for GCC is unknown. Therefore, the future of using GCC accurately as an outcome measure in artificial intelligence is currently uncertain.
      As this is a feasibility study, similar exploration will need to be repeated across different hospital set ups with varying resources of multi-disciplinary ICU teams to account for differences in communication, documentation, and inter- rater agreements on GCC in reviewers of differing backgrounds in the health sciences research. Amongst some questions related to GCC, is its value in patients of diverse cultural, racial, and socioeconomic backgrounds as well its bearing on quality of dying and survival.

      Funding

      • 1
        Small Grant Award, Mayo Clinic, Rochester, FP00100342
      • 2
        Critical Care Research Subcommittee Grant, Mayo Clinic, Rochester, PAU 43306

      Declaration of Competing Interest

      The authors have no relevant financial or non- financial interests to disclose

      References

        • McPeake JM
        • Harhay MO
        • Devine H
        • Iwashyna TJ
        • MacTavish P
        • Mikkelsen M
        • Shaw M
        • Quasim T
        Exploring Patients' Goals Within the Intensive Care Unit Rehabilitation Setting.
        American journal of critical care: an official publication, American Association of Critical-Care Nurses. 2019; 28: 393-400
        • Rubin EB
        • Buehler AE
        • Halpern SD
        States Worse Than Death Among Hospitalized Patients With Serious Illnesses.
        JAMA Intern Med. 2016; 176: 1557-1559
        • Sanders JJ
        • Miller K
        • Desai M
        • Geerse OP
        • Paladino J
        • Kavanagh J
        • Lakin JR
        • Neville BA
        • Block SD
        • Fromme EK
        • et al.
        Measuring Goal-Concordant Care: Results and Reflections From Secondary Analysis of a Trial to Improve Serious Illness Communication.
        Journal of pain and symptom management. 2020; 60 (e882): 889-897
        • Modes ME
        • Heckbert SR
        • Engelberg RA
        • Nielsen EL
        • Curtis JR
        • Kross EK
        Patient-Reported Receipt of Goal-Concordant Care Among Seriously Ill Outpatients-Prevalence and Associated Factors.
        Journal of pain and symptom management. 2020; 60: 765-773
        • Turnbull AE
        • Hartog CS
        Goal-concordant care in the ICU: a conceptual framework for future research.
        Intensive care medicine. 2017; 43: 1847-1849
        • Sudore RL
        • Heyland DK
        • Lum HD
        • Rietjens JAC
        • Korfage IJ
        • Ritchie CS
        • Hanson LC
        • Meier DE
        • Pantilat SZ
        • Lorenz K
        • et al.
        Outcomes That Define Successful Advance Care Planning: A Delphi Panel Consensus.
        Journal of pain and symptom management. 2018; 55 (e248): 245-255
        • Cooper Z
        • Koritsanszky LA
        • Cauley CE
        • Frydman JL
        • Bernacki RE
        • Mosenthal AC
        • Gawande AA
        • Block SD
        Recommendations for Best Communication Practices to Facilitate Goal-concordant Care for Seriously Ill Older Patients With Emergency Surgical Conditions.
        Ann Surg. 2016; 263: 1-6
        • Narang AK
        • Wright AA
        • Nicholas LH
        Trends in Advance Care Planning in Patients With Cancer: Results From a National Longitudinal Survey.
        JAMA Oncol. 2015; 1: 601-608
        • Khandelwal N
        • Curtis JR
        • Freedman VA
        • Kasper JD
        • Gozalo P
        • Engelberg RA
        • Teno JM
        How Often Is End-of-Life Care in the United States Inconsistent with Patients' Goals of Care?.
        Journal of palliative medicine. 2017; 20: 1400-1404
        • Bernacki R
        • Hutchings M
        • Vick J
        • Smith G
        • Paladino J
        • Lipsitz S
        • Gawande AA
        • Block SD
        Development of the Serious Illness Care Program: a randomised controlled trial of a palliative care communication intervention.
        BMJ Open. 2015; 5e009032
        • Berntsen GK
        • Gammon D
        • Steinsbekk A
        • Salamonsen A
        • Foss N
        • Ruland C
        • Fonnebo V
        How do we deal with multiple goals for care within an individual patient trajectory? A document content analysis of health service research papers on goals for care.
        BMJ Open. 2015; 5e009403
        • Glass DP
        • Wang SE
        • Minardi PM
        • Kanter MH
        Concordance of End-of-Life Care With End-of-Life Wishes in an Integrated Health Care System.
        JAMA Netw Open. 2021; 4e213053
        • Turnbull AE
        • Sahetya SK
        • Colantuoni E
        • Kweku J
        • Nikooie R
        • Curtis JR
        Inter-Rater Agreement of Intensivists Evaluating the Goal Concordance of Preference-Sensitive ICU Interventions.
        Journal of pain and symptom management. 2018; 56 (e403): 406-413
        • Halpern SD
        Goal-Concordant Care - Searching for the Holy Grail.
        The New England journal of medicine. 2019; 381: 1603-1606
        • Ahmad SR TA
        • Budahn L
        • Lemahieu AM
        • Anderson B
        • Vashistha K
        • Karnatovskaia L
        • Gajic O
        Feasibility of Extracting Meaningful Patient Centered Outcomes From the Electronic Health Record Following Critical Illness in the Elderly.
        Front Med. 2022; (Article 826169): 9
        • McHugh ML
        Interrater reliability: the kappa statistic.
        Biochem Med (Zagreb). 2012; 22: 276-282
      1. A controlled trial to improve care for seriously ill hospitalized patients. The study to understand prognoses and preferences for outcomes and risks of treatments (SUPPORT). The SUPPORT Principal Investigators.
        Jama. 1995; 274: 1591-1598
        • Lamas D
        • Panariello N
        • Henrich N
        • Hammes B
        • Hanson LC
        • Meier DE
        • Guinn N
        • Corrigan J
        • Hubber S
        • Luetke-Stahlman H
        • et al.
        Advance Care Planning Documentation in Electronic Health Records: Current Challenges and Recommendations for Change.
        Journal of palliative medicine. 2018; 21: 522-528
        • Grudzen CR
        • Buonocore P
        • Steinberg J
        • Ortiz JM
        • Richardson LD
        • Group ARCW
        Concordance of Advance Care Plans With Inpatient Directives in the Electronic Medical Record for Older Patients Admitted From the Emergency Department.
        Journal of pain and symptom management. 2016; 51: 647-651
        • Lee RY
        • Brumback LC
        • Lober WB
        • Sibley J
        • Nielsen EL
        • Treece PD
        • Kross EK
        • Loggers ET
        • Fausto JA
        • Lindvall C
        • et al.
        Identifying Goals of Care Conversations in the Electronic Health Record Using Natural Language Processing and Machine Learning.
        Journal of pain and symptom management. 2021; 61 (e132): 136-142
        • Taylor SP
        • Kowalkowski MA
        • Courtright KR
        • Burke HL
        • Patel S
        • Hicks S
        • Hurley C
        • Mitchell S
        • Halpern SD
        Deficits in Identification of Goals and Goal-Concordant Care After Sepsis Hospitalization.
        J Hosp Med. 2021; 16: 667-670
        • Cox CE
        • Riley IL
        • Ashana DC
        • Haines K
        • Olsen MK
        • Gu J
        • Pratt EH
        • Al-Hegelan M
        • Harrison RW
        • Naglee C
        • et al.
        Improving racial disparities in unmet palliative care needs among intensive care unit family members with a needs-targeted app intervention: The ICUconnect randomized clinical trial.
        Contemp Clin Trials. 2021; 103106319
        • Courtright KR
        • Dress EM
        • Singh J
        • Bayes BA
        • Chowdhury M
        • Small DS
        • Hetherington T
        • Plickert L
        • Detsky ME
        • Doctor JN
        • et al.
        Prognosticating Outcomes and Nudging Decisions with Electronic Records in the Intensive Care Unit Trial Protocol.
        Annals of the American Thoracic Society. 2021; 18: 336-346
        • Cox CE
        • Jones DM
        • Reagan W
        • Key MD
        • Chow V
        • McFarlin J
        • Casarett D
        • Creutzfeldt CJ
        • Docherty SL
        Palliative Care Planner: A Pilot Study to Evaluate Acceptability and Usability of an Electronic Health Records System-integrated, Needs-targeted App Platform.
        Annals of the American Thoracic Society. 2018; 15: 59-68
        • Dalal AK
        • Dykes P
        • Samal L
        • McNally K
        • Mlaver E
        • Yoon CS
        • Lipsitz SR
        • Bates DW
        Potential of an Electronic Health Record-Integrated Patient Portal for Improving Care Plan Concordance during Acute Care.
        Appl Clin Inform. 2019; 10: 358-366