If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Intermountain Healthcare, in collaboration with Cerner Corporation, developed a hospital-based electronic palliative care algorithm.
Objectives
This study aims to improve identification of patients who would benefit from palliative care services, and calculate palliative care penetration rates.
Methods
This study used a mixed-methods nonrandomized retrospective study design. Three 30-day iterations of clinical data were analyzed for patients identified by the electronic algorithm. During the second and third 30-day iterations, palliative care clinicians conducted chart reviews on a weekly basis for identified patients and determined whether the patients were appropriate for a palliative care consult. Positive predictive values (PPVs) were calculated. Based on the PPV, palliative care consult penetration rates were also calculated.
Results
During the first iteration, the algorithm triggered 2995 times on 1384 unique patient encounters (69.3% of the total inpatient population). In the second iteration, the algorithm triggered 851 times on 477 unique patient encounters (26.4% of the total inpatient population). Eight hundred twenty-one chart reviews were completed on 420 unique patient encounters. The PPV was 68.3%. Based on the PPV, the projected palliative care penetration rate was 17.6%. During the third iteration, the algorithm triggered 1229 times on 539 unique patient encounters (33.3% of the total inpatient population). Nine hundred sixty-seven chart reviews were completed on 505 unique patient encounters. The PPV was 80.1%. Based on the PPV, the projected palliative care penetration rate was 26.4%.
Conclusion
This study successfully optimized an electronic palliative care identification algorithm with a PPV of 80.1% and indicates appropriate palliative care penetration rates may be as high as 26.4% of the total inpatient population.
Timely integration of palliative care is needed in the acute care hospital setting. Today, around 90 million Americans are living with serious, life-threatening illness. This number is expected to double in the next 25 years. Most seriously ill people will spend time in the hospital over the course of their illness.
Although patients with serious and life-threatening illness benefit from inpatient palliative care services, it is challenging for nonpalliative care providers to identify these patients in need of palliative care services.
In addition, it is challenging to calculate appropriate palliative care service utilization rates because of a lack of available data on identified patients.
A study completed in England, based on all English deaths from 2006 to 2008, reviewed updated ICD-10 causes of death to estimate potential palliative care service utilization rates.
The study states that in high-income countries, up to 82% of those who die need palliative care. This recommendation is based on death registration data using underlying and contributory causes to provide estimates, not reliant on symptom of hospital activity data.
In 2008, The Center to Advance Palliative Care (CAPC) and the National Palliative Care Research Center established the Palliative Care Registry which collects palliative care program data across the country.
Using CAPC registry data, hospital palliative care penetration rates are calculated by dividing palliative care service utilization by the total inpatient population. Since 2009, palliative care programs in hospitals have increased their service penetration by 78% from 2.7% to 4.8% in 2015.
Although the number of patients reached by consultation level palliative care has increased, there is still room for growth especially in the timely identification of patients who need palliative care services.
To increase the palliative care penetration rates in hospitals, it is important for physicians and care providers to appropriately identify those in need of palliative care. In 2013, the Institute for Clinical Systems Improvement updated their guidelines for palliative care.
The objectives of these guidelines are to increase the identification of patients who are in the early stages of a serious illness who would benefit from palliative care; improve the effectiveness and comfort level of the primary care clinicians in communicating the necessity and benefits of palliative care with those patients with a serious illness; improve the assessment of the identified patient's palliative care needs using the domains of palliative care; increase the percentage of patients in the early stages of a serious illness who have a care plan identified and/or documented; improve the ongoing reassessment and adjustment of the patient's plan of care as the condition warrants, using the domains of palliative care; and increase the completion, documentation, and ongoing utilization of advance directives for patients with a serious illness.
Although there are a few manual screening tools to help providers increase the identification of patients for palliative care services, there are few clinically integrated tools built into electronic medical records to help clinicians automatically identify the palliative care patient population.
Intermountain Healthcare in collaboration with Cerner Corporation developed a hospital-based electronic palliative care identification algorithm to improve identification of patients who would benefit from specialty-level palliative care services. The algorithm works as an electronic screening tool, which runs on real-time data and alerts a provider when a patient meets the identification criteria that suggest palliative care services are needed. This study evaluates the precision of the identification algorithm and, based on those results, calculates potential palliative care penetration rates.
Methods
This study used a nonrandomized, mixed-methods retrospective study design. Quantitative and qualitative approaches offered a more in-depth look at palliative care identification at one hospital within the Intermountain Healthcare system in a large metropolitan city in Utah. The palliative care identification algorithm was created as a rules-based algorithm based on a review of the literature for existing identification tools, as well as the judgment of a clinical team. Three iterations of 30 days of clinical data were analyzed for patients who received palliative care consults, as well as for patients identified by the electronic algorithm. Clinical data were extracted from Cerner's Millennium® EHR software system. The algorithm ran in “silent mode,” which is a live production domain where data were collected on actual patients in real time, but results of the algorithm were analyzed retrospectively and no notification alerts were sent to providers. Therefore, patients may be triggered more than one time during an encounter. Key informant interviews were conducted after the first 30-day iteration. Changes to the algorithm were made between the 30-day iterations based on both qualitative and quantitative data. During the second and third 30-day iterations, palliative care clinicians conducted chart reviews on a weekly basis for the algorithm-identified patients. Positive predictive values (PPVs) were calculated based on the chart reviews. Interrater reliability was determined for unique patient encounters that were reviewed more than once. Based on the PPV, appropriate palliative care penetration rates were calculated.
The Identification Algorithm
The algorithm ran on real-time clinical data from the patients' EHR within the acute setting. The algorithm's initiation criteria will begin to process when a patient is admitted to the hospital as an inpatient or as soon as they enter the emergency department. Patients are excluded if they are currently receiving hospice care or if they have an active palliative care consult during the encounter. The algorithm identified patients with a serious illness in combination with high utilization, limited functional status, or at least one palliative care uncontrolled symptom (see Table 1 for definitions). The algorithm criteria and rules were selected based on a team of palliative care clinicians in conjunction with a literature review of criteria included in existing identification tools. Suppression logic was set to reidentify patients every 24 hours if a new qualifying variable was documented. Qualifying data continued to be documented during an identified patient's length of stay, which indicates the number of algorithm alerts may be greater than the number of unique patients during the same time period (see Fig. 1).
Table 1Palliative Care Identification Algorithm Criteria Definitions
Criteria
Definition
Serious illness
A qualifying active ICD-10 diagnosis from the principal diagnosis or problem list in the EHR that was clinically identified as a serious or life-threatening illness from a list of more than 500 ICD-10 codes. The following list indicates a broad representation of the actual codes:
•
Chronic organ failure
○
Congestive heart failure
○
Lung disease (obstructive and restrictive)
○
Cirrhosis
○
Chronic renal failure
•
Chronic neurologic disease
○
Dementia (all types)
○
ALS
○
Parkinson's disease
○
Muscular dystrophy
○
Other myopathies
•
Cancer
○
Recurrent
○
Metastatic
•
Acute catastrophic medical event
○
Acute cerebrovascular accident
○
Anoxic brain injury
○
Cardiac arrest
•
Traumatic brain injury
•
Signs of chronic debility
○
Generalized weakness
○
Decubitus ulcer
High utilization
•
Two or more inpatient admission within 30 days
•
Three or more inpatient admissions within 180 days
•
Three or more emergency admissions within 180 days
Limited functional status
Any of the following indicators documented in the EHR:
•
history of falls
•
being at risk for falls
•
needing assistance for bed mobility
•
ambulation
•
gait weakness
•
having an existing or new gastrointestinal or endotracheal tube
Palliative care uncontrolled symptoms
Any of the following symptoms documented in the EHR:
During each 30-day iteration, descriptive statistics were analyzed for all variables included in the identification algorithm. Patients identified by the algorithm were compared to patients who received actual palliative care consults. In addition to descriptive statistics, chart review data were also analyzed for the second- and third-iteration data. Changes to the algorithm were made in-between each algorithm iteration. Changes were made based on a clinical review of a palliative care team, descriptive statistics for each variable, and comparing variables for the algorithm-identified patients and patients who received actual palliative care consults. After the first iteration, the diagnosis list was refined to define serious illnesses related to or indicative of a patient who may need a palliative care consult in a more clinically meaningful way. These changes were made after comparing the list of top diagnoses for algorithm-identified patients and patients who received actual palliative care consults. A similar process of excluding variables was conducted for each of the algorithm criteria. During the second and third iterations, changes to the algorithm were again made based on a clinical review of the variables for algorithm-identified patients compared to those who received palliative care consults as well as comparing the variables based on the chart reviewed appropriateness. After comparing variables based on the chart reviews, select variables were excluded if it improved the comparison statistics or was clinically meaningful.
For chart reviews, palliative care clinicians first reviewed the patients' past medical history to confirm the presence of a chronic or serious illness. After this was confirmed, notes written by a provider in the patient's chart were reviewed to confirm that the chronic or serious illness was contributing to their health care utilization. Clinicians determined “yes” or “no” for whether the identified patients were appropriate for a palliative care consult based on their review of the patient's chart. If the answer was “no,” then a reason was given for why that patient was determined as an inappropriate patient for a palliative care consult. Interrater reliability kappa statistics were calculated for charts reviewed more than once during the second and third iterations. PPVs were calculated for the second- and third-iteration chart reviews. To increase the sample size, account for any potential changes for a patient over time, and calculate an accurate value for how the algorithm performed, the PPV was calculated based on the total number of chart reviews and not the unique patients reviewed. Based on the calculated PPV, palliative care penetration rates were calculated from the total number of unique patients the algorithm would have appropriately identified divided by the total inpatient population.
Qualitative Data Analysis
Key informant interviews were recorded and transcribed. Main themes were identified and provided the context and clinical expertise needed to make changes to the algorithm, which increased the accuracy of algorithm. Chart review comments for why a patient is inappropriate for a palliative care consult were also analyzed to identify areas to improve the algorithm.
Results
First Iteration 30-Day Results
The algorithm triggered 2995 times on 1384 unique patients, which was 69.3% of the total inpatient population during the 30 days. During this same time period, 58 patients actually received palliative care consults, encompassing 2.9% of the total inpatient population. Diagnosis, hospital utilization, functional status, and symptoms were compared for patients identified by the algorithm and patients who received palliative care consults during the same time period (Fig. 2, Fig. 3).
Fig. 2Top 10 diagnoses or problems for first-iteration algorithm.
In the second iteration, the algorithm triggered 967 times on 471 unique patient encounters (26.0% of the total inpatient population). Diagnosis, hospital utilization, functional status, and symptoms were compared for patients identified by the algorithm and by the appropriateness identified in the chart reviews (Fig. 4, Fig. 5). During this same time period, 62 patients received palliative care consults (3.4% of the total inpatient population). Eight hundred twenty-one chart reviews were completed on 420 unique patient encounters. Of those chart reviews, 560 (68.3%) were considered appropriate for palliative care consults, and 260 (31.7%) were considered inappropriate. Interrater reliability for charts reviewed more than one time (n = 146) was 93% (kappa = 0.84). The PPV was 68.2% (95% CI, 66.7%–69.3%) (Table 1).
Fig. 3Top 10 functional limitation or palliative care identifier symptom for first iteration.
In the third iteration, the algorithm triggered 1229 times on 539 unique patients (33.3% of total inpatient population). Diagnosis, hospital utilization, functional status, and symptoms were compared for patients identified by the algorithm and by the appropriateness identified in the chart reviews (Fig. 6, Fig. 7). During this same time period, 68 patients received palliative care consults (4.2% of the total inpatient population). Nine hundred sixty-seven chart reviews were completed on 505 unique patient encounters. Of those chart reviews, 775 (80.1%) were considered appropriate for palliative care consults, and 192 (19.9%) were considered inappropriate. Interrater reliability for charts reviewed more than one time (n = 167) was 96.0% (kappa = 0.87). The PPV was 80.1% (95% CI, 78.5%–81.5%) (Table 2).
Fig. 5Top 10 functional limitation or palliative care identifier symptom for second iteration by chart review appropriateness.
Currently, between 3% and 4% of this hospital's total 30-day inpatient population received palliative care consults. In the second iteration, based on a PPV of 68.2%, and a 30-day inpatient population of 1811, results indicate that 17.6% (95% CI, 13.7%–21.5%) of the inpatient population would be appropriate for a palliative care consult (Table 3).
In the third iteration, based on a PPV of 80.1%, and an inpatient population of 1617, results indicate that 26.4% (95% CI, 22.9%–29.8%) of the inpatient population would be appropriate for a palliative care consult (Table 3). These results challenge the national standard palliative care penetration rates CAPC has recommended for inpatient hospital populations.
Discussion
A mixed-methods collaboration is successful for developing and optimizing a precise palliative care identification algorithm with a PPV of 80%. To date, no other electronic palliative care identification algorithm using real-time data has a PPV greater than 80%. The palliative care algorithm will bring further awareness to providers regarding who is appropriate for an inpatient palliative care consult. In addition, this algorithm may also bring awareness for identifying basic primary-level palliative care needs, which can help foster discussions of symptom management and completion of advanced directives early in a patients' illness trajectory.
Optimizing the identification algorithm was an iterative process involving data-driven decisions made in collaboration with palliative care clinicians. In order for the algorithm to function most accurately, clinical expertise was needed to augment the data-driven algorithm. Without the collaborative qualitative data, a purely data-driven approach would not have been able to accurately capture the context around the diagnoses and problems included in the algorithm. Using a purely data-driven approach in the first algorithm iteration, we found that the algorithm was identifying patients based on most common diagnoses, such as hyperlipidemia, and problems that were not intrinsically related to palliative care. For example, if one were to attempt to identify appropriate palliative care consults from a purely data prospective, hyperlipidemia seems to be an appropriate palliative care diagnosis as it is the most common diagnosis among the patients identified in the algorithm. Through collaborating with palliative care clinicians, more precise and accurate diagnoses and problems specific to palliative care were identified. After refining the algorithm, the most common diagnoses changed from common, but not serious or life-threatening diseases such as hyperlipidemia, to more appropriate serious or life-threatening diseases such as chronic obstructive pulmonary disease.
Through understanding the clinical information, we could adjust the qualifying criteria and add additional logic, so the algorithm would alert on the most appropriate palliative care patients. Because of the changes made to the algorithm, by the third iteration, the PPV increased to 80.1%. According to recent human factors research studies, clinicians are more likely to adopt clinical decision support algorithms, such as the palliative care identification algorithm, once they reach the PPV threshold of more than 70.0%.
Despite the impactful changes to the algorithm, there is still opportunity for further refinement and optimization. Based on the chart review feedback, many of the issues identified for inappropriate triggers are due to how diagnoses, problems, symptoms, and functional status are currently documented in the electronic health record. Consideration is needed to understand the wider cultural changes regarding how diagnoses and problems are currently documented by clinicians and then consumed as data elements. In addition, there is the need to refine the oncology diagnoses included in this algorithm.
Our findings challenge the national palliative care penetration rates CAPC has recommended for inpatient hospital populations. CAPC recommends that around 8% of hospital total inpatient populations would be appropriate for palliative care consults.
National palliative care consult penetration rates range from 3.6% to 7.3%. For a facility with 300–499 beds, the national mean palliative care penetration rate is 4.3% (median 3.7%).
Results from this study suggest that palliative care penetration rates could range from 17.6% to as high as 26.4% of the total inpatient population. These findings are consistent with other studies demonstrating that current palliative care penetration rates likely underestimate the actual need for palliative services.
Further studies are needed to validate the palliative care penetration rates.
Despite results indicating the need for increased palliative care penetration rates, there are concerns about how palliative care teams can manage a large increase in the number of potential consults.
The number of patients identified by the algorithm would likely increase a hospital-based consult service census. Using this identification algorithm, the Intermountain facility would see palliative care consults increase by 4 to 8 times their current rate of 3%. A large change in the number of consults cannot be made quickly and would require dramatic changes in staffing. Unfortunately, because of the palliative care staffing shortage, recruiting more clinicians to manage an increase in consults is a long process.
American Academy of Hospice and Palliative Medicine Workforce Task Force. Estimate of current hospice and palliative medicine physician workforce shortage.
It is unlikely that palliative care teams across the country would be able to see nearly a quarter of their hospital's inpatient populations each year.
To manage the potential increase in palliative care penetration rates, several strategies can be incorporated into the electronic identification algorithm to help stratify the identified patients. One stratification method would be to incorporate a risk index to identify the patients who would, based on risk, most benefit from palliative care and from different levels of palliative care services.
Some of the algorithm-identified patients may be appropriate for specialty-level consultation at a palliative care clinic once they are discharged from the hospital. In other cases, other risk-stratified patients may be more appropriate for strengthened primary-level palliative care, in either the hospital or the outpatient setting. Regardless, more training and education will be required for these primary palliative care practitioners to ensure that patients receive the full benefits of palliative medicine.
Another method would be to stratify patients based on counts of how many serious or chronic illness diagnoses or problems are identified. With a specialized comorbidity count stratification, the algorithm could then stratify patients for those who have many serious or life-threatening diagnoses appropriate for palliative care, and other patients who only have one diagnoses, and may not need as many palliative care services. In addition, with this method, the threshold of identified patients could be modified to accommodate the available resources and capacity at individual facilities.
Strengths and Limitations
The goal of this study was to enhance the palliative care identification algorithm by bringing together the Intermountain Healthcare and Cerner development teams. This study required the clinical expertise of palliative care clinicians as well as the appropriate data-driven methods and analyses to fully understand palliative care algorithm data. This collaboration ultimately brought alignment of both patient data and clinical experience to optimize a palliative care identification algorithm. Strengths of this study include the number of chart reviews completed and the high concordance in the IRR scores. Limitations for this study include a small sample population from one health care system facility. The results may not be generalizable to a larger population.
Implications for Future Research
Through ongoing collaboration, we will evaluate and monitor the accuracy of the algorithm. Once Intermountain Healthcare turns the algorithm on in a live clinical setting where providers will be alerted in real time when a patient is identified as potentially appropriate for a palliative care consult, outcomes for patients that receive palliative care consults as a result of the algorithm trigger can be analyzed. Before turning the identification on in a live clinical setting, Intermountain will conduct proper training sessions and provide the necessary educational material. An overall assessment will be conducted to demonstrate how electronically identifying patients for palliative care can improve workflow and health care; quality of life; and cost outcomes for clinicians, patients, and the families involved.
Future development includes widening the identified population to include patients who would benefit from outpatient, community palliative care services, including pediatrics, and stratifying the identified patients to manage the available palliative care team capacity within a health care facility. In addition, refining oncology diagnoses identified in the algorithm will be an important improvement. Further iterations are needed to validate the chart reviews, algorithm PPV, and the palliative care penetration rates. Repeating this study at a larger health care system and at multiple facilities would validate the findings in a larger more generalizable population. These findings reiterate need for more research studies to understand the full scope of palliative care needs, and methods for how health care facilities can address the increasing need for palliative care services.
Disclosures and Acknowledgments
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. This project was supported in part by a collaboration between Intermountain Healthcare and Cerner Corporation. The authors thank their colleagues from Intermountain Healthcare and Cerner Corporation who provided insight and expertise that greatly assisted the project.
The authors thank Brendan Whatley for assistance with pulling data, Lameka Webster and Shannon Ruffin for support with project management, the McKay Dee Hospital inpatient and outpatient clinical team—Cindy Bauman, Nikki Ninalga, and Brandy Millward—for their assistance with chart reviews and providing their clinical expertise. The authors want to recognize Noreen Wynn and other Intermountain Medical Group leadership for their support of this project. The authors extend their appreciation to Wade Flood, Tanya Di Martino, and Justin Kimbrell for their contributions to the initial design and development of the palliative care identification trigger. Thanks also to Drs. Bharat Sutariya and Hugh Ryan from Cerner and Drs. Jeff McNally and Raj Srivastava from Intermountain for their executive leadership. The authors would also like to show their gratitude to Bob Amland, Marina Daldalian, Megan Quick, and the reviewers for their insights.
American Academy of Hospice and Palliative Medicine Workforce Task Force. Estimate of current hospice and palliative medicine physician workforce shortage.
30477-3&id=gr1.jpg){kind=link}
30477-3&id=gr2.jpg){kind=link}
30477-3&id=gr3.jpg){kind=link}
30477-3&id=gr4.jpg){kind=link}
30477-3&id=gr5.jpg){kind=link}
30477-3&id=gr6.jpg){kind=link}
30477-3&id=gr7.jpg){kind=link}