Advertisement

Clinician Accuracy When Estimating Survival Duration: The Role of the Patient's Performance Status and Time-Based Prognostic Categories

Open AccessPublished:May 13, 2011DOI:https://doi.org/10.1016/j.jpainsymman.2011.01.012

      Abstract

      Context

      Although shown to be an independent predictor of actual survival (AS) duration, previous reports have identified significant inaccuracy in clinician estimates of survival (CES).

      Objectives

      This study aimed to both examine demographic and clinical factors potentially impacting CES accuracy and explore possible strategies for improvement in a patient population with advanced incurable disease.

      Methods

      At the time of initial assessment by a specialist palliative care team, CES for each patient was chosen from one of the following time-based categories: <24 hours, one to seven days, one to four weeks, one to three months, three to six months, three to 12 months, or >12 months. Survival estimates were then classified as an accurate (AS=CES), overestimate (AS<CES), or underestimate (AS>CES). Demographic data were analyzed using descriptive statistics, and both univariate and stepwise multivariate logistic regression analyses were used to identify any associated demographic and/or clinical factors significantly impacting accuracy.

      Results

      Within the total study population of 1835, both CES and AS data were available for 1622 patients among whom mean and median survival was 26.5 and 88 days, respectively. The remaining 213 patients (12% of the total population) remained alive at the time of analysis. Of the total study population, CES was accurate for 34% of patients and an overestimate for 51% of patients. CES of <24 hours and one to seven days were significantly more likely to be accurate than any other prognostic category (P<0.0001). Additionally, a CES of either one to four weeks or >12 months was significantly more likely to be accurate than CES of one to three months, three to six months, and six to 12 months (P<0.0001). Finally, multivariate analyses indicated CES to be significantly more likely to be accurate for males (P=0.0407) and for patients with baseline Palliative Performance Scale (PPS) ratings of either “30 and less” (P<0.0001) or “70 and greater” (P<0.0001).

      Conclusion

      In a patient population referred for specialist palliative care consultation with diverse diagnoses and a wide range of CES, time-based categorization of survival estimates along with PPS and possibly gender could be used to inform the CES process for individual patients. Intentionally incorporating these objective elements into what has historically been the subjective process of CES may lead to improvements in accuracy.

      Key Words

      Introduction

      Clinicians who care for patients with advanced incurable diseases are commonly asked to provide an opinion or estimation of survival duration. The accuracy of such estimates is an important topic as their implications can be substantial. For patients, families, and caregivers, estimated survival is often an essential element of an informed decision-making process.
      • Steinhauser K.
      • Christakis N.A.
      • Clipp E.
      • et al.
      Factors considered important at the end of life by patients, families, physicians, and other care providers.
      For the health care team, an estimate of survival may be used to determine treatment options or qualification for participation in clinical trials.
      • Glare P.
      • Sinclair C.
      Palliative medicine review: prognostication.
      Clinicians often feel uncomfortable offering opinions regarding likely survival duration.
      • Christakis N.A.
      • Iwashyna T.J.
      Attitude and self-reported practice regarding prognostication in a national sample of internists.
      Factors include a lack of training in the process of estimation, the desire to maintain and preserve patient/family hope, the inherent inability to be exact or “correct,” and the complex and diverse factors impacting actual survival (AS) duration.
      • Christakis N.A.
      • Iwashyna T.J.
      Attitude and self-reported practice regarding prognostication in a national sample of internists.
      Two main pathways inform the process of estimating survival.
      • Glare P.
      • Sinclair C.
      Palliative medicine review: prognostication.
      The first is an objective actuarial process pairing specific clinical indices with corresponding mortality data. The second method is a more subjective process in which clinicians use their clinical knowledge and experience to formulate a “best guess” regarding the likely duration of survival of an individual patient. Numerous studies have documented that health care providers are generally inaccurate when estimating patient survival, with a greater tendency to overestimate.
      • Vigano A.
      • Dorgan M.
      • Bruera E.
      • Suarez-Almazor M.
      The relative accuracy of the clinical estimation of the duration of life for patients with end of life cancer.
      • Llobera J.
      • Esteva M.
      • Rifa J.
      • et al.
      Terminal cancer: duration and prediction of survival time.
      • Glare P.
      • Virik K.
      • Jones M.
      • et al.
      A systematic review of physicians' survival prediction in terminally ill cancer patients.
      • Christakis N.
      • Lamont E.
      Extent and determinants of error in doctors' prognoses in terminally ill patients: prospective cohort study.
      • Maltoni M.
      • Pirovano M.
      • Scarpi E.
      • et al.
      Prediction of survival of patients terminally ill with cancer. Results of an Italian prospective multicentric study.
      • Oxenham D.
      • Cornbleet M.A.
      Accuracy of prediction of survival by different professional groups in a hospice.
      • Evans C.
      • McCarthy M.
      Prognostic uncertainty in terminal care: can the Karnofsky index help?.
      • Mackillop W.J.
      • Quirt C.F.
      Measuring the accuracy of prognostic judgments in oncology.
      • Glare P.
      • Sinclair C.
      • Downing M.
      • et al.
      Predicting survival in patients with advanced disease.
      • Vigano A.
      • Dorgan M.
      • Buckingham J.
      • Bruera E.
      • Suarez-Almazor M.E.
      Survival prediction in terminal cancer patients: a systematic review of the medical literature.
      • Gripp S.
      • Moeller S.
      • Bolke E.
      • et al.
      Survival prediction in terminally ill cancer patients by clinical estimates, laboratory tests, and self-rated anxiety and depression.
      • Maltoni M.
      • Caraceni A.
      • Brunelli C.
      • et al.
      Prognostic factors in advanced cancer patients: evidence-based clinical recommendations—a study by the Steering Committee of the European Association for Palliative Care.
      Efforts have been made to clarify the multiple clinical variables that correlate with AS duration.
      • Maltoni M.
      • Caraceni A.
      • Brunelli C.
      • et al.
      Prognostic factors in advanced cancer patients: evidence-based clinical recommendations—a study by the Steering Committee of the European Association for Palliative Care.
      These include patient performance scales,
      • Glare P.
      • Sinclair C.
      Palliative medicine review: prognostication.
      • Evans C.
      • McCarthy M.
      Prognostic uncertainty in terminal care: can the Karnofsky index help?.
      • Lau F.
      • Downing M.
      • Lesperance M.
      • Shaw J.
      • Kuziemsky C.
      Use of Palliative Performance Scale in end-of-life prognostication.
      • Maltoni M.
      • Pirovano M.
      • Nanni O.
      • Labianca R.
      • Amadori D.
      Prognostic factors in terminal cancer patients.
      • Pirovano M.
      • Maltoni M.
      • Nanni O.
      • et al.
      A new palliative prognostic score: a first step for the staging of terminally ill cancer patients. Italian Multicenter and Study Group on Palliative Care.
      • Morita T.
      • Tsunoda J.
      • Inoue S.
      • Chihara S.
      The Palliative Prognostic Index: a scoring system for survival prediction of terminally ill cancer patients.
      • Chuang R.-B.
      • Hu W.-Y.
      • Chiu T.-Y.
      • Chen C.-Y.
      Prediction of survival in terminal patients in Taiwan: constructing a prognostic scale.
      clinical symptoms (e.g., dyspnea, anorexia, quality of life, anxiety spiritual distress),
      • Gripp S.
      • Moeller S.
      • Bolke E.
      • et al.
      Survival prediction in terminally ill cancer patients by clinical estimates, laboratory tests, and self-rated anxiety and depression.
      • Maltoni M.
      • Caraceni A.
      • Brunelli C.
      • et al.
      Prognostic factors in advanced cancer patients: evidence-based clinical recommendations—a study by the Steering Committee of the European Association for Palliative Care.
      • Chuang R.-B.
      • Hu W.-Y.
      • Chiu T.-Y.
      • Chen C.-Y.
      Prediction of survival in terminal patients in Taiwan: constructing a prognostic scale.
      • Vigano A.
      • Donaldson N.
      • Higginson I.J.
      • et al.
      Quality of life and survival prediction on terminal cancer patients.
      • Toscani F.
      • Brunelli C.
      • Miccinesi G.
      • et al.
      Predicting survival in terminal cancer patients: clinical observation or quality-of-life evaluation?.
      clinical signs (e.g., delirium, edema),
      • Gripp S.
      • Moeller S.
      • Bolke E.
      • et al.
      Survival prediction in terminally ill cancer patients by clinical estimates, laboratory tests, and self-rated anxiety and depression.
      • Maltoni M.
      • Caraceni A.
      • Brunelli C.
      • et al.
      Prognostic factors in advanced cancer patients: evidence-based clinical recommendations—a study by the Steering Committee of the European Association for Palliative Care.
      and laboratory values (e.g., white blood cell count, C-reactive protein, albumin).
      • Gripp S.
      • Moeller S.
      • Bolke E.
      • et al.
      Survival prediction in terminally ill cancer patients by clinical estimates, laboratory tests, and self-rated anxiety and depression.
      • Maltoni M.
      • Caraceni A.
      • Brunelli C.
      • et al.
      Prognostic factors in advanced cancer patients: evidence-based clinical recommendations—a study by the Steering Committee of the European Association for Palliative Care.
      • Ripamonti C.I.
      • Farina G.
      • Garassino M.C.
      Predictive models in palliative care.
      • Kikuchi N.
      • Ohmori K.
      • Kuriyama S.
      • et al.
      Survival prediction of patients with advanced cancer: the predictive accuracy of the model based on biological markers.
      Multivariate analyses involving many of these variables have led to the development of several tools intended to simplify the process of estimating survival.
      • Pirovano M.
      • Maltoni M.
      • Nanni O.
      • et al.
      A new palliative prognostic score: a first step for the staging of terminally ill cancer patients. Italian Multicenter and Study Group on Palliative Care.
      • Lau F.
      • Cloutier-Fisher D.
      • Kuziemsky C.
      • et al.
      A systematic review of prognostic tools for estimating survival time in palliative care.
      Despite reported inaccuracies, several studies have consistently confirmed that clinician estimates of survival (CES) remain an independent predictor of AS duration.
      • Glare P.
      • Sinclair C.
      Palliative medicine review: prognostication.
      Glare et al.
      • Glare P.
      • Virik K.
      • Jones M.
      • et al.
      A systematic review of physicians' survival prediction in terminally ill cancer patients.
      suggested that CES be used as the reference standard for evaluating other methods of estimating survival and strongly recommended further research into the ways CES accuracy can be improved.
      The objectives for the present study were to examine CES accuracy for a population referred for specialist palliative care services and, with the goal of informing the process of developing strategies for improvement, to identify the demographic and clinical variables impacting CES accuracy.

      Methods

      Study Design and Setting

      This prospective cohort study involved clinicians of the Palliative Care Consult Team (PCCT) at Sunnybrook Health Sciences Centre (SHSC), a regional cancer center and academic tertiary care hospital in Toronto, Canada. The PCCT provides specialist inpatient palliative care consultation to all patient care units within the hospital and runs daily outpatient palliative care clinics in the ambulatory cancer center. CES was documented at the time of initial consultation, with outpatients assessed by physicians only and inpatients assessed and reviewed by interprofessional teams comprising both physicians and advanced practice nurses. CES was represented by choice of one time-based prognostic category felt to best represent an individual patient's likely survival duration, that is, <24 hours, one to seven days, one to four weeks, one to three months, three to six months, six to 12 months, or >12 months. Clinicians did not receive formal training in survival estimation, and accuracy data were not made available to the clinicians until after the study closed.

      Setting, Study Population, and Variables

      Data were analyzed for all patients referred to the PCCT between January 1, 2007 and June 30, 2009. Survival data were tracked for nine months after closure of the study (to March 2010). To identify clinical factors related to CES accuracy, specific variables were examined, including age, gender, Palliative Performance Scale (PPS) score at the time of initial assessment, location of consult (inpatient hospital ward vs. outpatient palliative care clinic), diagnosis, and, in the cancer cohort, the presence or absence of liver metastases. Within the study period, if patients had separate new consults in both the outpatient and inpatient setting, only the first assessment was used for the main analysis.
      In patients with a known date of death, AS was determined by calculating the time between dates of initial assessment and death. CES was determined to be “accurate” if the AS fell within the time range for the prognostic category documented at the time of initial consultation. Patients were subsequently categorized as having either “AS=CES” (survival was accurate), “AS<CES” (survival was overestimated), or “AS>CES” (survival was underestimated).
      Patients for whom a date of death could not be determined were included in the study if evidence of clinical documentation confirmed accuracy of individual survival estimates. For example, if the CES for a patient was chosen to be “>12 months” and documentation existed of a clinical interaction 12 months beyond the initial assessment, the estimate was categorized as being accurate (AS=CES), despite the absence of a death date. Similarly, if documentation existed of clinical interaction beyond the date corresponding to the maximum length of time associated with the prognostic category (e.g., if the CES chosen was “three to six months” and a clinical interaction had been documented eight months after initial assessment), the CES was categorized as being underestimated (AS>CES).
      Ethics approval for the study was obtained through the Research Ethics Board of SHSC.

      Statistical Methods

      Demographics were analyzed with descriptive statistics. Univariate logistic regression analyses were used to identify factors associated with AS=CES, AS<CES, or AS>CES using Wald Chi-squared tests and odds ratios (with 95% confidence limits). Any factor with a univariate P-value <0.1 was subsequently used in stepwise multivariate logistic regression analysis. Patient age and PPS score were each analyzed as both continuous and categorical variables. Values for mean and median AS duration were compared based on gender using the median nonparametric test from which P-values were generated. SAS version 9.2 (SAS Institute, Inc., Cary, NC) was used for the analysis.

      Results

      Study Population

      During the 30-month study period, 2269 new consultations were completed by the PCCT. Of these, 102 were excluded, as the reason for referral was to assess and manage “cancer-related treatment pain” in patients with no evidence of active disease. Data confirming status in relation to survival estimate were unable to be obtained for 332 patients, leaving a final study population of 1835 patients. Of the total study population, 1622 patients had a known date of death and 213 patients remained alive at the time of analysis.

      Descriptive Data

      A summary of demographic information is found in Table 1. At the time of referral, all patients were felt to have a diagnosis and/or extent of disease that was considered incurable. Median age of the study population was 71 years (range 20–102 years), and 51% were female. Seventy-seven percent of the study population were inpatients, and 80% had an underlying cancer diagnosis. The most common diagnoses in the noncancer population were dementia (16%), cerebral vascular accident (14%), congestive heart failure (14%), amyotrophic lateral sclerosis (8%), and end-stage renal disease (8%). Primary reasons for referral ranged from complex symptom management to routine end-of-life care.
      Table 1Demographic Information
      Characteristicsn (%)
      Unless otherwise indicated.
      Age
      n = 1835.
       Median71
       Range20–102
      Gender
      n = 1835.
       Male902 (49)
       Female933 (51)
      Care setting
      n = 1835.
       Inpatient1407 (77)
       Outpatient428 (23)
      Disease category
      n = 1835.
       Cancer1477 (80)
       Noncancer358 (20)
      Liver metastases
      Among those with a cancer diagnosis (n = 1477).
       Yes382 (26)
      PPS (n=1785)
       Median40
       Range10–100
      10–30632 (35)
      40–60681 (38)
      70–100472 (26)
       Missing, n50
      AS duration (n=1622), days
       Median26.5
       Mean88.1
       Range0–1014
      a Unless otherwise indicated.
      b n = 1835.
      c Among those with a cancer diagnosis (n = 1477).
      A PPS rating was not recorded for 50 patients within the study. Among the 1785 remaining, the median PPS rating was 40, with patient numbers relatively equally distributed among the PPS groupings of 10–30, 40–60, and 70–100. For patients with known dates of death, median and mean survival times were found to be 26.5 days and 88 days, respectively. Distribution of CES by time-based prognostic category is outlined in Fig. 1, and Table 2 shows the distribution of both CES and PPS based on diagnosis.
      Figure thumbnail gr1
      Fig. 1Distribution of CES category by time-based prognostic category (n=1835).
      Table 2Distribution of CES and PPS by Diagnosis
      Total PopulationCancer SubpopulationNoncancer Subpopulation
      CES CategoryTotal Within CES Category (n = 1835)Total Within CES Category (n = 1477)Percent Among Cancer Patients OnlyPercent Within CES CategoryTotal Within CES Category (n = 358)Percent Among Noncancer Patients OnlyPercent Within CES Category
      >12 months37934724923298
      Six to 12 months29327218932167
      Three to six months27825217912679
      One to three months4143532485611715
      One to four weeks219132960872440
      One to seven days211946451173355
      Zero to 24 hours412726614434
      PPS GroupingsTotal Within PPS Group (n = 1785)Total Within PPS Group (n = 1434)Percent Within PPS GroupPercent of Only Cancer PatientsTotal Within PPS Group (n = 351)Percent Among Noncancer Patients OnlyPercent Within PPS Group
      10–3063235825572747843
      40–6068161243906919.710
      70–1004724643398.382.31.7

      Main Results: CES Accuracy

      Table 3 outlines the distribution of CES accuracy for the demographic and clinical variables used in further analysis. Among the entire study population, overall survival estimates were found to be accurate (AS=CES) for 34%, overestimated (AS<CES) for 51%, and underestimated (AS>CES) for 15%. Of the 213 study patients for whom a date of death was unknown, a CES of “>12 months” was chosen for 105 patients, leaving the remaining 108 patients distributed among the categories one to four weeks (n=9), one to three months (n=31), three to six months (n=37), and six to 12 months (n=31).
      Table 3Distribution of CES Accuracy by Demographic Categories (n=1835)
      AS=CESAS<CESAS>CES
      n (%)
      Entire population617 (34)938 (51)280 (15)
      Gender
       Male (902)328 (36)459 (51)115 (13)
       Female (933)289 (31)479 (51)165 (18)
      Encounter type
       Inpatient (1407)466 (33)704 (50)237 (17)
       Outpatient (428)151 (35)234 (55)43 (10)
      Disease type
       Cancer (1477)470 (32)788 (53)219 (15)
       Noncancer (358)147 (41)150 (42)61 (17)
      Liver metastases
       Yes (382)112 (29)215 (56)55 (14)
       No (1095)358 (33)573 (53)164 (15)
      PPS groupings
       10–30 (632)264 (42)258 (41)110 (17)
       40–60 (681)169 (25)413 (61)99 (15)
       70–100 (472)170 (36)237 (50)65 (10)

      Other Analyses: Univariate and Multivariate

      Univariate analysis was first used to compare demographic and clinical variables between two groupings of patients: those with an “accurate estimate” (AS=CES) and those with an “inaccurate estimate” (i.e., AS<CES and AS>CES combined). Gender and PPS (both as a categorical and continuous variable) were each significantly related to CES accuracy (Table 4). Using a stepwise selection procedure of multivariate logistic regression analysis, including those variables found to have P<0.1 from the univariate analysis, gender and PPS (both categorical and continuous) remained significant (Table 5). More specifically, males and patients with PPS ratings within either the 10–30 or the 70–100 groupings were significantly more likely to have an accurate CES.
      Table 4Univariate Analysis: Accurate CES vs. Inaccurate CES (n=1835)
      VariableORCIP-value
      Age (years)0.9970.9910.39331.004
      Age categories
       <40 vs. 80+1.4390.8100.25302.556
       40–49 vs. 80+1.2830.8910.29131.847
       50–59 vs. 80+0.9070.6670.11921.233
       60–69 vs. 80+1.0610.8010.76141.406
       70–79 vs. 80+0.9860.7530.31021.290
      Gender
       M>F1.2731.0490.01471.546
      Care setting
       Outpatient vs. inpatient0.9080.7240.40761.140
      Liver metastases
       Yes vs. no0.8540.6620.22301.101
      PPS score0.9920.9880.00030.996
      PPS categories<0.001
       40–60 vs. ≤300.4600.364<0.00010.582
       70+ vs. ≤300.7850.6140.19821.003
       70+ vs. 40–601.7051.320<0.00012.203
      OR=odds ratio; CI=confidence interval.
      Values that are in bold are meant to highlight those of statistical significance.
      Table 5Multivariate Analysis: Accurate CES vs. Inaccurate CES (n=1835)
      VariableORCIP-value
      Continuous PPS
       Gender (1=M; 0=F)1.2291.0091.4970.0407
       PPS score0.9920.9880.9960.0004
      Categorical PPS
       Gender (1=M; 0=F)1.2511.0251.5270.0273
       PPS categories
      40–60 vs. ≤300.4590.3630.581<0.0001
      70+ vs. ≤300.7910.6191.0120.1709
      70+ vs. 40–601.7241.3332.227<0.0001
      OR=odds ratio; CI=confidence interval.
      Values that are in bold are meant to highlight those of statistical significance.
      Further analysis of gender-related data was completed by directly comparing demographic and clinical data for both male and female study patients. No significant differences were found between the two groups in age, PPS scores, diagnosis, or distribution of CES. Although gender differences in mean survival for the entire study population did not reach statistical significance (P=0.0589), analysis of the AS>CES subpopulation using the median nonparametric test uncovered median survival times for women to be significantly longer than those for men, and women among this subgroup were significantly more likely to outlive their CES (Table 6).
      Table 6Mean and Median AS by Gender and CES Accuracy (n=1622)
      PopulationMean Survival—Days (SD)Median Survival—Days (CI)
      MaleFemaleMaleFemale
      All patients with known date of death81
      P=0.0589.
      (139)
      95
      P=0.0589.
      (154)
      25 (21–29)29 (24–35)
      AS=CES (n=512)101 (170)111 (188)23 (15–38)18 (12–40)
      AS<CES (n=938)51 (68)61 (78)22 (19–26)25 (23–30)
      AS>CES (n=172)188 (234)229 (243)92
      P=0.0224.
      (46–144)
      147
      P=0.0224.
      (95–198)
      SD=standard deviation; CI=confidence interval.
      Values that are in bold are meant to highlight those of statistical significance.
      a P=0.0589.
      b P=0.0224.
      The relationship between time-based prognostic category and CES accuracy was further examined (Fig. 2). The likelihood of the CES being accurate was significantly higher if “<24 hours” or “one to seven days” was chosen when compared with all other categories (P<0.0001). A CES of “one to four weeks” was found to be significantly more accurate than a CES of “one to three months,” “three to six months,” or “six to 12 months” (P<0.0001). Notably, CES accuracy significantly increased for the “>12 months” category (P<0.0001) and found to be at values similar to those found for the “one to four weeks” category (P=0.4842).
      Figure thumbnail gr2
      Fig. 2Distribution of CES accuracy by time-based prognostic category (n=1835).
      Given the disproportionate number of noncancer patients within the PPS 10–30 grouping, further analysis of all patients within this PPS range was completed. Among this subpopulation (i.e., those with a low baseline performance), a significant difference in CES accuracy was not found based on underlying diagnosis (Table 7). Cancer and noncancer populations with moderate and high performance levels were not compared given the low numbers of noncancer patients in each of these groups.
      Table 7Distribution of CES Accuracy by Diagnosis Among Patients with Low Performance (n=632)
      DiagnosisAS=CESAS<CESAS > CES
      Cancer (only patients with PPS 10–30), %394418
      Noncancer (only patients with PPS 10–30), %463717
      P-value0.06030.07630.8869
      In seeking to determine a simple strategy to improve accuracy among patients for whom estimates were found to be the most inaccurate, we further explored the group whose original CES was between one and 12 months (n= 985, 20% AS=CES, 61% AS<CES, 19% AS>CES). For each individual in this cohort, we uniformly decreased the original CES category. For example, if the clinician recorded an estimated survival of “six to 12 months,” this was recoded as “three to six months.” Similarly, “three to six months” was recoded as “one to three months,” and “one to three months” as “one to four weeks.” With this artificial data set of survival estimates, accuracy was reanalyzed and, for this same n=985, AS=CES was found to be 31%, AS<CES was 38%, and AS>CES was 30%. Although these values represent slight improvements in the distribution of accurate and overestimates, further statistical analysis was not pursued because, regardless of significance, clinical utility was not thought to be possible. A similar process of recoding CES for the subgroup of patients with an initial PPS of 40–60 minimally impacted accuracy (i.e., actual data—25% AS=CES; artificial data—29% AS=CES) and again, further statistical analysis was not pursued.

      Discussion

      Patients with advanced disease tend to overestimate their own life expectancy, which has been shown to significantly impact decisions made regarding treatment preferences.
      • Eidinger R.N.
      • Schapira D.V.
      Cancer patients' insight into their treatment, prognosis, and unconventional therapies.
      • Weeks J.C.
      • Cook E.F.
      • O'Day S.J.
      • et al.
      Relationship between cancer patients' predictions of prognosis and their treatment preferences.
      If care decisions are to be informed by more accurate survival estimates, clinicians should be prepared to discuss this with patients and caregivers. Given previous reports regarding the accuracy of survival estimates made by clinicians themselves, practical strategies leading to improvements must be identified. The aims of the present study were to explore demographic and clinical factors potentially impacting CES accuracy and begin the process of formalizing improvement strategies.

      Main Findings

      For the study population, most survival estimates made by clinicians were inaccurate and, similar to patients themselves, the tendency was to overestimate. A number of factors were found to significantly impact CES accuracy. First, a direct correlation between CES accuracy and baseline performance status was identified. In particular, compared with those in the moderate range, accuracy was significantly higher for patients with either low or high PPS ratings. Using time-based categories to represent CES, the greatest inaccuracies were found with estimates between one and 12 months. Given the particularly high frequency of overestimation for patients with either baseline PPS ratings in the moderate range or an initial CES of between one and 12 months, we tested a hypothesis for patients within each of these subpopulations and analyzed artificial data created by retrospectively decreasing the time-based CES category by one. This simple strategy did not result in significant improvements in the degree of accuracy.
      In exploring other clinical and demographic factors, no impact on CES accuracy was found by either care setting (for the entire study population) or the presence or absence of metastatic disease to the liver (among the subpopulation of cancer patients). Most patients within the noncancer subpopulation were inpatients with baseline PPS ratings of 10, 20, or 30. On analysis of the subpopulation that included all patients with a baseline performance status in the low range (i.e., PPS 10–30), it was found that diagnosis (i.e., cancer vs. noncancer) did not impact CES accuracy. Finally, compared with females, the likelihood of CES accuracy was significantly greater for males. This prompted an analysis of the total subpopulation outliving their survival estimate (i.e., AS>CES). This group comprised significantly more females for whom, in addition, median survival duration was found to be significantly longer than males.

      CES Accuracy: Previous Reports

      The results of this study validate previous findings that CES accuracy is, in general, poor, with a substantial tendency to overestimate.
      • Glare P.
      • Virik K.
      • Jones M.
      • et al.
      A systematic review of physicians' survival prediction in terminally ill cancer patients.
      • Christakis N.
      • Lamont E.
      Extent and determinants of error in doctors' prognoses in terminally ill patients: prospective cohort study.
      • Maltoni M.
      • Pirovano M.
      • Scarpi E.
      • et al.
      Prediction of survival of patients terminally ill with cancer. Results of an Italian prospective multicentric study.
      • Glare P.
      • Sinclair C.
      • Downing M.
      • et al.
      Predicting survival in patients with advanced disease.
      Limitations of most previous studies addressing CES accuracy include both sample size and range of AS. The strengths of this study include the large sample size and, despite referral for palliative care consultation, the wide distribution of AS among a population with disease known to be incurable. Previously identified factors found to influence CES accuracy include the clinician's length of experience (i.e., more experience is associated with higher accuracy) and the strength of the clinician-patient relationship (i.e., stronger relationships are associated with lower accuracy).
      • Christakis N.
      • Lamont E.
      Extent and determinants of error in doctors' prognoses in terminally ill patients: prospective cohort study.
      It has been suggested that the estimation process may be consciously or subconsciously influenced by external factors such as their use in determining access to certain health care resources (i.e., hospice admission criteria).
      • Schonwetter R.S.
      • Teasdale T.A.
      • Storey P.
      • Luchi R.J.
      Estimation of survival time in terminal cancer patients: an impedance to hospice admissions?.
      • Kinzbrunner B.M.
      Ethical dilemmas in hospice and palliative care.
      Efforts were made in this study to minimize clinician-related factors by having survival estimated at the time of initial assessment and, therefore, not biased by previous knowledge of or relationship with each patient. As well, for our setting, CES was not connected to resource accessibility, eliminating this source of potential bias.
      Reviewing previous work in which time-based categories have been used to assess CES accuracy, substantial variability exists in the time intervals studied. The time-based categories chosen to represent CES for this study were intended to accurately reflect what is commonly communicated by clinicians to patients and families when addressing survival estimation, that is, “hours,” “days,” “weeks,” “months,” and “years.” Although the specific time intervals studied may appear somewhat arbitrary, they represent an endeavor to achieve an adequate balance between meaningful statistical exploration and clinical utility. From both clinical and practical perspectives, it is likely to be more helpful for patients to know if their expected survival is some number of weeks or months. The finding of significantly higher frequency of accurate estimates for patients with a CES of either four weeks or less or greater than 12 months resembles the findings of Vigano et al.,
      • Vigano A.
      • Dorgan M.
      • Bruera E.
      • Suarez-Almazor M.
      The relative accuracy of the clinical estimation of the duration of life for patients with end of life cancer.
      who reported lower CES accuracy for patients living between two and six months compared with those living either less than two or greater than six months. Although population based, their cohort was limited to cancer patients, with the inception point defined as the “time when patients entered the terminal phase,” that is, deemed by oncology as not having further disease-modifying therapies available.
      Most other studies have reported an incremental decrease in CES accuracy for patients with proportionately longer AS.
      • Glare P.
      • Virik K.
      • Jones M.
      • et al.
      A systematic review of physicians' survival prediction in terminally ill cancer patients.
      • Mackillop W.J.
      • Quirt C.F.
      Measuring the accuracy of prognostic judgments in oncology.
      Mackillop and Quirt
      • Mackillop W.J.
      • Quirt C.F.
      Measuring the accuracy of prognostic judgments in oncology.
      were the first to label this possible phenomenon by proposing that CES accuracy may be subject to a “horizon effect.” In their systematic review, Glare et al.
      • Glare P.
      • Virik K.
      • Jones M.
      • et al.
      A systematic review of physicians' survival prediction in terminally ill cancer patients.
      suggested that CES has no predictive ability for patients living longer than six months. Important to note, however, is that within this review, among the eight studies quantitatively assessed, very small numbers of patients had an AS of greater than six months. In the present study, the finding of improved accuracy for estimates greater than 12 months compared with one to three, three to six, and six to 12 months joins two key phenomena in survival estimation. Fig. 2 could be viewed as a pictorial representation of both the “horizon effect” and the clinical utility of Pattison and Romer's
      • Pattison M.
      • Romer A.
      Improving care through the end of life: launching a primary care clinic-based program.
      “surprise question” (“Would you be surprised if this patient died in the next 12 months?”). To clarify, it would seem that clinicians have a certain ability to discriminate between those who are and who are not entering the “dying phase” (i.e., death within one year).
      Previous studies exploring survival and performance status have primarily focused on the direct relationship between performance scale ratings and AS, with correlation between the two having been consistently demonstrated.
      • Evans C.
      • McCarthy M.
      Prognostic uncertainty in terminal care: can the Karnofsky index help?.
      • Glare P.
      • Sinclair C.
      • Downing M.
      • et al.
      Predicting survival in patients with advanced disease.
      • Vigano A.
      • Dorgan M.
      • Buckingham J.
      • Bruera E.
      • Suarez-Almazor M.E.
      Survival prediction in terminal cancer patients: a systematic review of the medical literature.
      • Lau F.
      • Downing M.
      • Lesperance M.
      • Shaw J.
      • Kuziemsky C.
      Use of Palliative Performance Scale in end-of-life prognostication.
      The relationship between performance status and CES accuracy was an element of the Morita et al.
      • Morita T.
      • Tsunoda J.
      • Inoue S.
      • Chihara S.
      Improved accuracy of physicians' survival prediction for terminally ill cancer patients using the Palliative Prognostic Index.
      validation study for the Palliative Prognostic Index (PPI), a tool developed to guide the process of survival estimation. Clinicians in their study estimated survival, with accuracy subsequently determined for two independent cohorts of patients admitted to their palliative care unit. For the study cohort, clinicians used the PPI, a scoring system comprising five elements, including PPS, oral intake, edema, dyspnea at rest, and delirium. Although the authors report a significant improvement in CES accuracy for the study cohort, several factors limit the study's generalizability and clinical application. All patients had a diagnosis of cancer, and “prognosis of less than six months” was a criterion for admission. As stated outright by the authors, the PPI is “most suitable for prediction of three week survival and less useful for patients with a longer prognosis.” To our knowledge, ours is the first study demonstrating PPS rating as a continuous variable significantly impacting CES accuracy and further, accuracy significantly differs among clinically relevant PPS groupings.
      Building on previous work outlining the correlation between higher CES accuracy and lower AS, we examined the potential impact of two clinical elements known to impact AS. For patients with any one of several primary cancer sites, the presence of liver metastases greatly impacts median survival times
      • Ayoub J.P.
      • Hess K.R.
      • Abbruzzese M.C.
      • et al.
      Unknown primary tumors metastatic to liver.
      • Stangl R.
      • Altendorf-Hofmann A.
      • Charnley R.M.
      • Scheele J.
      Factors influencing the natural history of colorectal liver metastases.
      • Pentheroudakis G.
      • Fountzilas G.
      • Bafaloukos D.
      • et al.
      Metastatic breast cancer with liver metastases: a registry analysis of clinicopathologic, management and outcome characteristics of 500 women.
      • Ahmed A.
      • Turner G.
      • King B.
      • et al.
      Midgut neuroendocrine tumours with liver metastases: results of the UKINETS study.
      and was chosen to represent the subset of the cancer population with disease considered “advanced.” Additionally, despite the possibilities of both, a reversible cause for admission and subsequent improvement in performance status, acute care hospitalization in general indicates an advanced and/or complex condition. It was our hypothesis that CES for a population referred for palliative care consultation would be more accurate for those admitted to hospital (vs. those referred in the ambulatory setting) and, among cancer patients, those with advanced disease, that is, documented liver metastases. Our finding that both the care setting and the presence of liver metastases do not impact CES accuracy is of clinical relevance as it may be assumed that the tendency to overestimate survival would be less in patients for whom their disease is particularly advanced and/or who have complications severe enough to warrant acute care admission.
      Christakis and Lamont
      • Christakis N.
      • Lamont E.
      Extent and determinants of error in doctors' prognoses in terminally ill patients: prospective cohort study.
      previously explored CES accuracy based on stratification by diagnosis. Although the differences were not reported as being statistically significant, in comparing cancer and noncancer cohorts, the authors identified both a greater tendency for clinicians to overestimate survival in patients with cancer and a greater tendency for clinicians to underestimate survival in patients with a noncancer diagnosis when both groups were compared. Performance status data were not reported for their study population, however, making it unclear how the two groups may have differed in baseline function. Survival estimation in the setting of noncancer diagnosis is often thought to be particularly challenging. The results of our study suggest, however, that at least for patients with a PPS of 30 or less, diagnosis may not impact the accuracy of survival estimates.
      For the present study, males and females were demographically equivalent, and overall mean and median AS for each did not differ significantly. Although gender has rarely been identified as a possible factor related to survival estimation, the CES accuracy results we reported are consistent with Christakis and Lamont's
      • Christakis N.
      • Lamont E.
      Extent and determinants of error in doctors' prognoses in terminally ill patients: prospective cohort study.
      finding that “male patients were 58% less likely to have overpessimistic than correct predictions.” Additionally, the report by Lau et al.
      • Lau F.
      • Downing M.
      • Lesperance M.
      • Shaw J.
      • Kuziemsky C.
      Use of Palliative Performance Scale in end-of-life prognostication.
      addressing a population similar to ours demonstrated that compared with males with the same initial PPS rating, females lived longer. The connection between gender, AS, and CES accuracy likely involves complexities currently not well understood.

      Future Research Directions

      Proposing and testing a strategy to address the marked inaccuracy of survival estimates for patients with either a PPS of between 40 and 60 or an estimate of between one and 12 months confirmed the simple act of secondarily decreasing the time-based category results in limited improvements. For this group of patients, then, there are presumably multiple variables impacting survival, and it is this group more than any for whom CES accuracy is likely to be improved by incorporating clinical parameters previously demonstrated to be associated with survival (e.g., anorexia-cachexia, delirium, dyspnea, and leukocytosis).
      • Maltoni M.
      • Caraceni A.
      • Brunelli C.
      • et al.
      Prognostic factors in advanced cancer patients: evidence-based clinical recommendations—a study by the Steering Committee of the European Association for Palliative Care.
      Future studies examining the impact of other clinical variables on CES accuracy may contribute to developing a concrete strategy to inform a process for the skill of estimating patient survival. Other directions for future research include seeking to achieve a better understanding of the role gender plays in AS and CES accuracy and a more detailed exploration of disease extent (e.g., examining multiple sites of metastatic disease among cancer patients).

      Limitations

      A number of limitations exist for our study. First, despite the large sample size, the generalizability of our results is limited, given that all patients were accrued from one institution and assessed by members of one specialist palliative care team. The inception point for this population was at the time of referral for palliative care consultation, suggesting a certain degree of homogeneity and thus a second limitation. Two strategies previously suggested to improve CES accuracy include assessing patients multiple times at fixed intervals
      • Oxenham D.
      • Cornbleet M.A.
      Accuracy of prediction of survival by different professional groups in a hospice.
      and having patients assessed by more than one clinician (i.e., “second opinion”).
      • Christakis N.
      • Lamont E.
      Extent and determinants of error in doctors' prognoses in terminally ill patients: prospective cohort study.
      Corresponding limitations for our study then include CES accuracy determined based on one assessment made by one clinician at one point in time. For this study, we made the assumption that the process of assigning an individual PPS rating was consistent among individual clinicians across both care settings and that these ratings were accurate. Although inter-rater agreement has been shown to be good for the PPS, our assumption does have some implications when interpreting the results. Finally, the uniformly low PPS ratings among the noncancer subpopulation reflects the limited role palliative care currently plays for this group of patients in the ambulatory clinic setting. It remains unclear how CES accuracy may compare among cancer and noncancer patients with either a moderate or a high level of function.

      Implications for Clinicians

      Despite clear evidence that most patients want specific information regarding likely survival duration and that this be both hopeful and realistic, optimal methods of communicating this information to patients and families remain unclear.
      • Glare P.
      • Sinclair C.
      Palliative medicine review: prognostication.
      Many studies stress the importance of individualizing the discussion both in terms of content and timing. In response to a call for better professional skill development in CES, we propose clinicians who intentionally incorporate both performance status and time-based prognostic categories into the process of survival estimation. If the clinician's best guess places the CES in the one to 12 month range and/or if the patient's PPS rating is between 40 and 60, there should be cause to pause and consider the very high likelihood of overestimation. The results of this study can be used to inform what may evolve to be a stepwise process of estimating survival, thus informing discussions with patients and families addressing likely survival duration. In addition, although substantial exploration is required to clarify further, clinicians should be aware of the possibility that women may survive longer than estimated, particularly if gender differences do, in fact, exist in overall AS.

      Disclosures and Acknowledgments

      The authors declare no conflicts of interest. No funding was received for this study.

      References

        • Steinhauser K.
        • Christakis N.A.
        • Clipp E.
        • et al.
        Factors considered important at the end of life by patients, families, physicians, and other care providers.
        JAMA. 2000; 284: 2476-2482
        • Glare P.
        • Sinclair C.
        Palliative medicine review: prognostication.
        J Palliat Med. 2008; 11: 84-103
        • Christakis N.A.
        • Iwashyna T.J.
        Attitude and self-reported practice regarding prognostication in a national sample of internists.
        Arch Intern Med. 1998; 158: 2389-2395
        • Vigano A.
        • Dorgan M.
        • Bruera E.
        • Suarez-Almazor M.
        The relative accuracy of the clinical estimation of the duration of life for patients with end of life cancer.
        Cancer. 1999; 86: 170-176
        • Llobera J.
        • Esteva M.
        • Rifa J.
        • et al.
        Terminal cancer: duration and prediction of survival time.
        Eur J Cancer. 2000; 36: 2036-2043
        • Glare P.
        • Virik K.
        • Jones M.
        • et al.
        A systematic review of physicians' survival prediction in terminally ill cancer patients.
        BMJ. 2003; 327: 195-200
        • Christakis N.
        • Lamont E.
        Extent and determinants of error in doctors' prognoses in terminally ill patients: prospective cohort study.
        BMJ. 2000; 320: 469-473
        • Maltoni M.
        • Pirovano M.
        • Scarpi E.
        • et al.
        Prediction of survival of patients terminally ill with cancer. Results of an Italian prospective multicentric study.
        Cancer. 1995; 75: 2613-2622
        • Oxenham D.
        • Cornbleet M.A.
        Accuracy of prediction of survival by different professional groups in a hospice.
        Palliat Med. 1998; 12: 117-118
        • Evans C.
        • McCarthy M.
        Prognostic uncertainty in terminal care: can the Karnofsky index help?.
        Lancet. 1985; 1: 1204-1206
        • Mackillop W.J.
        • Quirt C.F.
        Measuring the accuracy of prognostic judgments in oncology.
        J Clin Epidemiol. 1997; 50: 21-29
        • Glare P.
        • Sinclair C.
        • Downing M.
        • et al.
        Predicting survival in patients with advanced disease.
        Eur J Cancer. 2008; 44: 1146-1156
        • Vigano A.
        • Dorgan M.
        • Buckingham J.
        • Bruera E.
        • Suarez-Almazor M.E.
        Survival prediction in terminal cancer patients: a systematic review of the medical literature.
        Palliat Med. 2000; 14: 363-374
        • Gripp S.
        • Moeller S.
        • Bolke E.
        • et al.
        Survival prediction in terminally ill cancer patients by clinical estimates, laboratory tests, and self-rated anxiety and depression.
        J Clin Oncol. 2007; 25: 3313-3320
        • Maltoni M.
        • Caraceni A.
        • Brunelli C.
        • et al.
        Prognostic factors in advanced cancer patients: evidence-based clinical recommendations—a study by the Steering Committee of the European Association for Palliative Care.
        J Clin Oncol. 2005; 23: 6240-6248
        • Lau F.
        • Downing M.
        • Lesperance M.
        • Shaw J.
        • Kuziemsky C.
        Use of Palliative Performance Scale in end-of-life prognostication.
        J Palliat Med. 2006; 9: 1066-1075
        • Maltoni M.
        • Pirovano M.
        • Nanni O.
        • Labianca R.
        • Amadori D.
        Prognostic factors in terminal cancer patients.
        Eur J Palliat Care. 1994; 1: 122-125
        • Pirovano M.
        • Maltoni M.
        • Nanni O.
        • et al.
        A new palliative prognostic score: a first step for the staging of terminally ill cancer patients. Italian Multicenter and Study Group on Palliative Care.
        J Pain Symptom Manage. 1999; 17: 231-239
        • Morita T.
        • Tsunoda J.
        • Inoue S.
        • Chihara S.
        The Palliative Prognostic Index: a scoring system for survival prediction of terminally ill cancer patients.
        Support Care Cancer. 1999; 7: 128-133
        • Chuang R.-B.
        • Hu W.-Y.
        • Chiu T.-Y.
        • Chen C.-Y.
        Prediction of survival in terminal patients in Taiwan: constructing a prognostic scale.
        J Pain Symptom Manage. 2004; 28: 115-122
        • Vigano A.
        • Donaldson N.
        • Higginson I.J.
        • et al.
        Quality of life and survival prediction on terminal cancer patients.
        Cancer. 2004; 101: 1090-1098
        • Toscani F.
        • Brunelli C.
        • Miccinesi G.
        • et al.
        Predicting survival in terminal cancer patients: clinical observation or quality-of-life evaluation?.
        Palliat Med. 2005; 19: 220-227
        • Ripamonti C.I.
        • Farina G.
        • Garassino M.C.
        Predictive models in palliative care.
        Cancer. 2009; 115: 3128-3134
        • Kikuchi N.
        • Ohmori K.
        • Kuriyama S.
        • et al.
        Survival prediction of patients with advanced cancer: the predictive accuracy of the model based on biological markers.
        J Pain Symptom Manage. 2007; 34: 600-606
        • Lau F.
        • Cloutier-Fisher D.
        • Kuziemsky C.
        • et al.
        A systematic review of prognostic tools for estimating survival time in palliative care.
        J Palliat Care. 2007; 23: 93-112
        • Eidinger R.N.
        • Schapira D.V.
        Cancer patients' insight into their treatment, prognosis, and unconventional therapies.
        Cancer. 1984; 53: 2736-2740
        • Weeks J.C.
        • Cook E.F.
        • O'Day S.J.
        • et al.
        Relationship between cancer patients' predictions of prognosis and their treatment preferences.
        JAMA. 1998; 279: 1709-1714
        • Schonwetter R.S.
        • Teasdale T.A.
        • Storey P.
        • Luchi R.J.
        Estimation of survival time in terminal cancer patients: an impedance to hospice admissions?.
        Hosp J. 1990; 6: 65-79
        • Kinzbrunner B.M.
        Ethical dilemmas in hospice and palliative care.
        Support Care Cancer. 1995; 3: 28-36
        • Pattison M.
        • Romer A.
        Improving care through the end of life: launching a primary care clinic-based program.
        J Palliat Med. 2001; 4: 249-255
        • Morita T.
        • Tsunoda J.
        • Inoue S.
        • Chihara S.
        Improved accuracy of physicians' survival prediction for terminally ill cancer patients using the Palliative Prognostic Index.
        Palliat Med. 2001; 15: 419-424
        • Ayoub J.P.
        • Hess K.R.
        • Abbruzzese M.C.
        • et al.
        Unknown primary tumors metastatic to liver.
        J Clin Oncol. 1998; 6: 2110-2112
        • Stangl R.
        • Altendorf-Hofmann A.
        • Charnley R.M.
        • Scheele J.
        Factors influencing the natural history of colorectal liver metastases.
        Lancet. 1994; 343: 1405-1410
        • Pentheroudakis G.
        • Fountzilas G.
        • Bafaloukos D.
        • et al.
        Metastatic breast cancer with liver metastases: a registry analysis of clinicopathologic, management and outcome characteristics of 500 women.
        Breast Cancer Res Treat. 2006; 97: 237-244
        • Ahmed A.
        • Turner G.
        • King B.
        • et al.
        Midgut neuroendocrine tumours with liver metastases: results of the UKINETS study.
        Endocr Relat Cancer. 2009; 16: 885-894