If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Centre for Population Health Sciences, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, SingaporeNTU Institute for Health Technologies (NTU HealthTech), Interdisciplinary Graduate School, Nanyang Technological University, Singapore, SingaporeHealth Services and Outcomes Research Department, National Healthcare Group, Singapore, Singapore
Centre for Population Health Sciences, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, SingaporePsychology Programme, School of Social Sciences, Nanyang Technological University, Singapore, SingaporePalliative Care Centre for Excellence in Research and Education, Singapore, Singapore
Centre for Population Health Sciences, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, SingaporeGlobal eHealth Unit, Department of Primary Care and Public Health, School of Public Health, Imperial College London, United Kingdom
One of the key outcomes of advance care planning is whether patients had received care that was consistent with their expressed goals and preferences.
The aims of this study were to illustrate the feasibility of using routinely collected health care data that include hospital procedural codes, diagnosis-related codes, health services utilization, and death registry data and to ascertain the level of concordance between care received and the stated goals.
In this retrospective cohort study, medical treatments were ascertained using a combination of hospital procedural codes and diagnosis-related codes. Places of care were obtained by reviewing the sequence of health services used, and the place of death was obtained from the national death registry. To ascertain concordance, medical treatment, places of care, and place of death were compared against the individual's preferences.
The sample includes 1731 decedents (aged 21 years and above) who completed their advance care planning documentation as part of a national program. Ninety-eight percent who wished for comfort measures met their preferences. Sixty-five percent of individuals who wished to be cared for at home received care at home. Nearly 40% of all individuals who opted to die at home achieved their wishes, whereas 76% of those who opted for home or hospital and home or hospice had their preferences fulfilled.
Administrative data offer a cost-efficient and powerful method for assessing outcomes for a large population-based national program. However, this approach is still at an early stage of development and needs to be further validated before it can be used at scale.
One of the main objectives of advance care planning (ACP) is to allow patients to maintain autonomy in relation to current and future health care decisions. In 2011, a national advance care planning program, called “Living Matters,” was launched in Singapore to meet this specific aim. The program aimed to ensure systematic conduct of end-of-life care discussions, supported by a consistent documentation of preferences. One of the first to be introduced in Asia, “Living Matters” was adapted from the Respecting Choices® program in the Gunderson Health System in Wisconsin, U.S. In the U.S., the program was associated with improvements in patient-surrogate congruence
the paucity of evaluation studies examining whether choices were respected represents a significant gap in current ACP evaluation frameworks. In a systematic review of 55 studies examining the efficacy of ACP,
have relied on similar methods. However, such data collection approaches could be time consuming and might not be scalable to population-based evaluations. The literature suggests that studies with large sample sizes have primarily examined concordance with only the place of death.
In a recent publication by Turley et al., the authors advocated for, and demonstrated the utility of, defining end-of-life care events based on medication and procedural codes that were captured routinely as part of electronic medical records documentation.
This approach allowed a systematic assessment of the level of concordance between documented preferences and actualized end-of-life care treatments.
Since its implementation, the outcomes of “Living Matters” have not been evaluated nationally. Discussions about death and dying are considered taboo with little communication about these preferences between family members. Doctors are also often requested to withhold poor prognosis from patients by their families.
Little research has been conducted in Asian settings regarding the effectiveness of ACP in promoting adherence with individual treatment preferences. To better support the assessment of ACP outcomes at the population level, administrative data collected at the national level were used to compare stated preferences against actualized outcomes.
Study Design and Population
The “Living Matters” program involves a coordinated approach to ACP whereby trained nonmedical facilitators, in collaboration with treating physicians, assist patients and their families to reflect on the patient's goals, values, and beliefs and to discuss and document their future choices about health care. Similar to the Respecting Choices® program, the Singaporean program aims to 1) increase awareness about ACP among health care professionals, and the public; 2) recruit and train ACP facilitators to facilitate conversations in health and social care organizations; and 3) establish and strengthen systems to support ACP implementation, including the development of a national ACP IT system.
Different ACP processes and documentation requirements are applicable to adults who are healthy, diagnosed with complex chronic illnesses, or diagnosed with a life-limiting/advanced illness. This study focuses on the last group. We adopted a retrospective cross-sectional design to profile the end-of-life preferences of deceased individuals and to ascertain the extent of concordance between stated preferences and end-of-life care. Individuals who had completed their ACP and died between 2011 and 2015 were included.
As part of “Living Matters,” a coordinated approach to ACP was adopted, whereby trained facilitators, who may not necessarily be medically trained, support patients and their families to discuss and document their future choices about health care. The preferences were then documented in an ACP form that specifies preferences about cardiopulmonary resuscitation during cardiopulmonary arrest, and preferences about intubation, mechanical ventilation, cardioversion, and transfer to intensive care. The patient was also asked about their preferred place of medical treatment, and care if their medical condition were to deteriorate. Finally, the preferred place of death (nursing home, acute hospital, home, inpatient hospice, no preference) was also documented. The completed form was then uploaded into the national ACP information technology system, and/or the electronic medical records of the individual hospitals.
We extracted ACP participant profile and preference data from the aforementioned databases. Other data variables were obtained from the Ministry of Health: acute hospital diagnosis, procedural, and service codes were extracted from administrative databases that captured case-mix information from all public-sector hospitals in Singapore; long-term care service usage was extracted from administrative databases that captured information for individuals who received government subsidies for services provided by dialysis centers, day care centers, home care providers, nursing homes, and inpatient hospices; and place of death data were extracted from the national death registry. To safeguard data confidentiality, a project unique identifying number was generated for each National Registration Identify Card number that identifies every Singapore resident, and this was used to link data across data sets.
The ACP document contained information about individual preferences related to the administration of cardiopulmonary resuscitation (CPR), medical intervention (full treatment, limited additional interventions, comfort measures), place of care, and place of death. Full treatment includes intubation, mechanical ventilation, and cardioversion, and transfer to intensive care units if indicated. Limited additional interventions include oral or intravenous medications as well as noninvasive ventilation support but excludes endotracheal intubation or long-term life support measures or intensive care unit admissions. Comfort measures include reasonable measures made to offer food and fluids. Medications, oxygen, and other measures may be used as needed for comfort.
For the preferred place of medical treatment or care, individuals could opt for home, hospice, nursing home, hospital, as well as a trial of treatment in their own home or nursing home or hospice before considering transfer to hospital or hospice. The documentation also allowed individuals to indicate “no preferences” or “others.” To reflect the sequential or conditional options inherent in the preferences for place of care, we mapped out 10 permutations (Supplementary Material 2).
For place of death in the event of deterioration, the options include home, the hospital, hospice, or nursing home. They could also state two or more alternatives, such as home or hospital, and home or nursing home or hospital. A “no preference” option was also available.
End-of-Life Care Events
To identify incidences of tracheostomy, intubation, mechanical ventilation, noninvasive ventilation, electrical cardioversions, and CPR, we used a combination of the International Classification of Diseases, Ninth Revision, Clinical Modification codes, diagnosis-related group codes, and the Australian Classification of Health Interventions, Version 6. First, two researchers with training in health services research created the initial list of codes, based on a combination of a literature review
and checking through the relevant code books. Second, three physician-researchers independently reviewed the list of procedural and diagnosis-related codes for final inclusion. The time frames used for collating the information were 14, 30, and 90 days before death. (Refer to Supplementary Material for the codes used.) Although the reliability of the coding has not been explored or ascertained, we expect the diagnosis-related group codes to be reliably coded since this is monitored by the Ministry of Health, as part of the annual review of health care utilization and performance.
To locate the places where care was received, we identified admissions to inpatient acute care facilities, community hospitals, nursing home, and inpatient hospices. Records of utilization of home care (medical, nursing, palliative care) and day care services were used to determine whether the patient was cared for at home. In addition, if we could identify no formal care service usage, we assumed that the patient was cared for at home. Given that an individual could access and consume different types of services at various sites before death, we first ascertained the sequence of health service usage, based on the date of admission or attendance and date of discharge, whichever is applicable for the service type, in the 14, 30, and 90 days before death.
To identify the actual place of death, the official classification of “residential home,” “nursing home and clinic,” “public and charitable institutions” (aged care facilities), “licensed sick receiving house” (inpatient hospices), and “others,” were extracted.
The four ACP preferences (CPR, full treatment, limited additional intervention, comfort measures) were mapped onto eight end-of-life care procedures or treatments (Fig. 1), which were each coded dichotomously. For each patient, concordance occurred when recorded end-of-life care treatments matched the documented preferences of full treatment, limited additional interventions, and comfort care. For example, if the patient had opted for comfort care but had received tracheostomy and/or mechanical ventilation and intubation and/or noninvasive ventilation or CPR, this would have been identified and coded as a nonconcordant case. For each patient, the date difference between the procedure administration and date of death was also computed, to ascertain concordance at 14, 30, and 90 days before death. Our mapping algorithm is, however, limited because a lack of concordance for individuals who opted for full treatment or limited additional intervention could reflect low clinical need, rather than discordance.
To compute the level of agreement for place of care, where preferences could be for a single site, or be conditional for everyone, we mapped the actual care transitions to the preferred place of care, as indicated by the 10 identified possible permutations. Concordance is achieved when the preceding-succeeding relationships were met. For example, the location(s) of care would be considered concordant with the preference to receive “a trial of treatment in their homes before considering transfer to a hospital,” if the care transition reflected that the patient had received medical, nursing, or palliative care at home, before being admitted to an acute hospital. For the place of death, concordance was determined by comparing the place of death category with the stated preference. For any preferences that included two or more options, we have considered agreement using a summative approach. For instance, there is concordance with the preference “home or hospital” if the patient had either died at home or in an acute hospital. Data on individuals who had stated “unsure,” “no preference,” or “depends on the situation for place of medical treatment or place of death” were not considered in the computation of the level of agreement.
All data analyses were carried out using Stata Version 12 (StataCorp LP, College Station, TX).
The demographics of the 1731 decedents are summarized in Table 1. More than half of the sample was aged 75 years and above. Females account for half of the sample, and individuals of Chinese ethnicity were overrepresented in this sample, compared with the national average (83% vs. 77%).
In terms of preferences, more than 90% opted for “no CPR during cardiac arrest and is not breathing or has no pulse.” Only 44 of 1731 patients opted for full treatment, with the remaining stating a preference for limited additional interventions or comfort measures. Approximately 46% would prefer to be cared for at home or to have a trial of care at home, before considering care in the hospital, and 24% outrightly preferred to be care for in the hospital. For the preferred place of death, approximately 40% expressed a distinct preference for dying at home, and about 30% expressed a preference for death in an institutional setting (hospital, nursing home, hospice). While only 5% of the sample had no preferences or were unsure about the place of care, 23% indicated they had no preference or were unsure about the place of death.
Table 1Demographics and Documented Preferences of Patients (N = 1731)
Age group (yrs)
Do not attempt
Limited additional intervention
Preferred place of medical treatment
Trial of treatment at home before transfer to hospital or hospice
Table 2 presents the level of agreement, in terms of medical treatment. The level of concordance for individuals who opted to receive comfort measures was close to 98%. The agreement between preferences for CPR, full treatment, and limited additional treatment and actual treatments was very low, but as clinical need or judgment was not ascertained in this retrospective database study, it might not reflect concordance.
Table 2Concordance With Medical Treatment, by Time Before Death (N = 1731)
The level of agreement between the preferred and actual place of treatment and care is reflected in Table 3. Of the 193 (11%) individuals who wished to be cared for at home, 57%–65% received care at home. Among the 458 (26%) patients who wished for a trial at home before admission to the hospital, close to half met their preferences. Approximately 24% (407/1731) of individuals wished to be cared for in the hospital. The percentage of concordance with hospital care ranged from 68% at 14 days before death to 90% at 90 days before death. Among the 244 (14%) patients who opted to only receive care in an inpatient hospice or to have a trial of care in the hospice before transfer to the hospital, approximately one in three patients received preference-concordant care. Close to one in two patients who opted for care in the nursing home or a trial in nursing home before admission to the hospital met their preferences, and the concordance reached 100% if the move occurred toward the time frame of 90 days before death.
Table 3Place of Care Concordance, by Time Before Death (N = 1731)
Concordance by Time Before Death (%)
Trial of treatment at home before transfer to hospital or hospice
Trial of treatment before transfer to hospital from
From Table 4, we observe that the overall concordance with place of death preferences is 50%. The rate of concordance was lower for individuals who opted for a single location, compared to those who preferred more than one alternative. Nearly 40% (680/1731) of individuals who opted to die at home achieved their wishes, whereas 76% of those who opted for home or hospital and home or hospice had their preferences fulfilled. However, a lower level (45%) of concordance was observed among those who wished to die in the hospice or nursing homes.
Table 4Place of Death Concordance
Home or hospital
Home or hospice
Hospital or hospice
Hospital or nursing home
Hospice or nursing home
Others (relative's home, health care institution, three or more options)
We presented administrative data, collected at the national level, to evaluate whether care received by an individual was in agreement with their goals and preferences. One of the strengths of our approach is its declarative nature, where the mapping of preferences to actual care and the determination of concordance are explicitly defined, rendering the process repeatable. Concordance with the goals of care has been recognized as a key ACP outcome measure by an international consensus study
; but the usual methods for ascertaining concordance through medical records review could be costly. When family members or health care professionals were surveyed after bereavement, recall bias might reduce the accuracy and reliability of the outcomes. These methods also do not support outcomes monitoring at the system level. With stronger reliance on electronic medical records, eventually, algorithms can be designed and implemented to determine individual-level concordance.
We found the level of agreement, in terms of preferences for CPR and comfort measures, to be very high among the deceased population who completed their ACP. The treatment concordance for participants who have opted for full or limited additional treatments was relatively lower. Another study found that the rates of intubation and defibrillation/electrical cardioversion among a sample of 683 inpatient decedents were 10.1% in the last 24 hours of life,
which is comparable to the 9.1% who received full treatment (intubation, mechanical ventilation, and cardioversion) in the 14 days before death in our study.
In terms of the level of agreement with place of care, there were differences depending on the time frame of analysis. Invariably, we introduced more episodes of institutional care as the time frame of analysis moved further away from the time of death. This reduces the concordance with home as one of the preferred place of care, while at the same instance, concordance increases for institutional care. There is no consensus in the literature on the time frame to consider for the computation of concordance with this set of preferences. To the best of our knowledge, no published studies have examined concordance with the preferred place of care.
Using the location of death recorded in the death certificates, we found that approximately 50% of the sample died at their preferred place of death. The percentage of individuals with a home preference dying at home (51%) is double that of the national share of deaths at home (25%).
The concordance rate in our study was lower for individuals who preferred to die in the hospice or nursing homes. This contrasts with the findings by Agar et al., who found 77% and 64% of concordance for palliative care patients who preferred to die in a hospice or aged care facility, respectively. The availability of home care and alternative care facilities within each country or geographical region could have influenced the level of concordance. In Singapore, the projected expansion of home palliative care places, from 5000 in 2014 to 6000 in 2020, could support individuals and families to fulfill the wishes to die at home.
There is a substantial research gap pertaining to the measurement and assessment of whether preferences are met. We would recommend future work in this area to improve our understanding of the importance of and the meaning placed by health care professionals, patients, and their family members on achieving concordance for each category of preferences. For instance, do they place equal weightage on achieving concordance with medical treatments and place of medical treatment or place of death.
With the anticipated expansion of the ACP program from the coverage of 10,000
there needs to be a cost-efficient and reliable method for ascertaining the effects of policy in respecting the wishes of the participants. In this study, we have illustrated that administrative data can support the assessment of population-level concordance with preferences stated in an ACP. This approach can be generalized to other health care systems, using similar coding mechanisms. With the advent of electronic medical records, text-mining techniques could be applied to enable a systematic monitoring of outcomes at the population level. However, this must be accompanied with further validation to ascertain the degree of potential misclassification, and overcoding and undercoding for life-sustaining treatments in the routine coding of data. Additional data including service codes for intravenous medications, antibiotics, and artificially administered nutrition should be included for future studies. In addition, further research is required to support the development of reliable and valid measurement tools, especially in terms of agreement with the preferred place of care.
The difference in ACP adoption rates across ethnic groups should be examined in future studies.
Potential limitations of this study include the use of administrative data to determine the incidence of life-sustaining treatments. The current method, although useful for determining the extent of concordance for comfort treatment, can be limited in determining concordance for individuals who opted for full and limited additional treatments because the use of aggressive treatment needs to be clinically indicated. We were also unable to conduct a systematic assessment of clinical relevancy in this study.
Because we have relied on documented preferences, we were unable to account for shifts in patient preferences that were not documented. Future research could compare the accuracy of relying on administrative database versus other methods of data collection, such as health care professional reports, bereaved family member surveys, and medical records review.
Routinely collected data on health care service utilization and place of death can support the assessment of concordance between end-of-life care preferences and actual treatments. Administrative data offer a cost-efficient and powerful method for assessing the outcomes for a large population-based sample, compared to traditional methods, such as medical records review and key informant interviews. Therefore, further research is required to validate this method to move toward a data-driven approach for ACP outcomes monitoring and assessments.
Disclosures and Acknowledgments
The authors would like to thank Dr. Raymond Ng, Consultant, Department of Palliative Medicine, Tan Tock Seng Hospital for contributing data that were used in the analysis. The authors would also like to acknowledge Mr. Geronimo Jimenez, NTU; Dr. Sng Ming Keat, NTU, and other colleagues from the Agency for Integrated Care and the Ministry of Health for providing project support.
Woan Shin Tan was funded by the Singapore National Medical Research Council Research and the Singapore National Healthcare Group.
This study was funded by Agency for Integrated Care Singapore, which receives public funding from the Ministry of Health of the Singaporean Government. The funder has played no role in the study design and collection, analysis, or interpretation of data.
The authors declare no conflict of interest with respect to the research, authorship, and/or publication of this article.
Supplementary Material 1Procedural and Diagnosis-Related Codes to Identify Life-Sustaining Treatments
Closed chest cardiac massage
Closed chest cardiac massage
Nonmechanical methods of resuscitation
Mechanical ventilation and intubation
Insertion of endotracheal tube
Management of continuous ventilatory support, ≤24 hours
Ventilation >95 hours W/O catastrophic CC
Other intubation of respiratory tract
Management of continuous ventilatory support, more than 24 hours and less than 96 hours
Nervous system diagnosis W ventilator support W catastrophic CC
Other continuous invasive mechanical ventilation
Management of continuous ventilatory support, 96 hours or more
Nervous system diagnosis W ventilator support W/O catastrophic CC
Continuous invasive mechanical ventilation of unspecified duration
Respiratory system diagnosis W ventilator support W catastrophic CC
Continuous invasive mechanical ventilation for less than 96 consecutive hours
Respiratory system diagnosis W ventilator support W/O catastrophic CC
Continuous invasive mechanical ventilation for 96 consecutive hours or more
Circulatory system diagnosis W ventilator support W catastrophic CC
Circulatory system diagnosis W ventilator support W/O catastrophic CC
Infectious and parasitic diseases W ventilator support
Ventilation or cranial procedures for multiple significant trauma
Injuries, poisoning, and toxic effects of drugs W ventilator support
Ventilation for burns and severe full-thickness burns
Tracheostomy W ventilation >95 hours W catastrophic CC
Trach W Vent >95 hours W/O Cat CC or Trach/Vent >95 hours W Cat CC
Other permanent tracheostomy
Tracheostomy W/O catastrophic CC
Noninvasive mechanical ventilation
Respiratory system diagnosis W noninvasive ventilation
Circulatory system diagnosis W noninvasive ventilation
ICD-9-CM = International Classification of Diseases, Ninth Revision, Clinical Modification; ACHI = Australian Classification of Health Interventions codes; DRG = diagnosis-related group.
Coding Transitions in Places of Medical Treatment/Care
Transitions in places of care were established by looking through the service records obtained from the MOH case-mix and subvention database, and intermediate- and long-term care information system. For each of the services used, the date of admission or discharge was compared with the date of death of the individual to ascertain whether the patient had used the service within 14, 30, and 90 days. A sequence of services used within each of these time frames was constructed. The table below illustrates the sequence generated for an individual with a unique identifying number “1234.”
Coding of the Match Between Preferences and Transitions in Care
In the example above, if the individual “1234” had opted for a “trial of care at home before transfer to hospital” (coded as “AD”) as his/her preferred place of care, the level of concordance would have been coded as 1 since the individual indeed transited between care at home and the hospital in the last 14 days before death (coded as “AAD”). However, if the individual “1234” had opted for hospital (coded as “B”) as the preferred place of care, the level of concordance would have been coded as 0 for nonconcordance.
Deriving Aggregate Level of Concordance
To obtain the final level of concordance number of individuals for preferred place of medical treatment/care, the number of individuals who received care at their preferred locations was divided by the number of individuals for each category of place of medical treatment/care.
A randomized, controlled trial to improve advance care planning among patients undergoing cardiac surgery.