Advertisement

End-of-Life Culture Change Practices in U.S. Nursing Homes in 2016/2017

Open ArchivePublished:December 19, 2018DOI:https://doi.org/10.1016/j.jpainsymman.2018.12.330

      Abstract

      Context

      The nursing home (NH) culture change (CC) movement, which emphasizes person-centered care, is particularly relevant to meeting the unique needs of residents near the end of life.

      Objectives

      We aimed to evaluate the NH-reported adoption of person-centered end-of-life culture change (EOL-CC) practices and identify NH characteristics associated with greater adoption.

      Methods

      We used NH and state policy data for 1358 NHs completing a nationally representative 2016/17 NH Culture Change Survey. An 18-point EOL-CC score was created by summarizing responses from six survey items related to practices for residents who were dying/had died. NHs were divided into quartiles reflecting their EOL-CC score, and multivariable ordered logistic regression was used to identify NH characteristics associated with having higher (quartile) scores.

      Results

      The mean EOL-CC score was 13.7 (SD = 3.0). Correlates of higher scores differed from those previously found for non-EOL-CC practices. Higher NH leadership scores and nonprofit status were consistently associated with higher EOL-CC scores. For example, a three-point leadership score increase was associated with higher odds of an NH performing in the top EOL-CC quartile (odds ratio [OR] = 2.0, 95% CI: 1.82–2.30), whereas for-profit status was associated with lower odds (OR = 0.7, 95% CI: 0.49–0.90). The availability of palliative care consults was associated with a greater likelihood of EOL-CC scores above the median (OR = 1.5, 95% CI: 1.10–1.93), but not in the top or bottom quartile.

      Conclusion

      NH-reported adoption of EOL-CC practices varies, and the presence of palliative care consults in NHs explains only some of this variation. Findings support the importance of evaluating EOL-CC practices separately from other culture change practices.

      Key Words

      Introduction

      The nursing home (NH) culture change movement attempts to transform NH care to be more person-centered by encouraging a homelike environment, facilitating resident- and family-directed care, and empowering staff to become more engaged in care practices and decision making.
      • Koren M.J.
      Person-centered care for nursing home residents: the culture-change movement.
      • Rahman A.N.
      • Schnelle J.F.
      The nursing home culture-change movement: recent past, present, and future directions for research.
      • Zimmerman S.
      • Shier V.
      • Saliba D.
      Transforming nursing home culture: evidence for practice and policy.
      Residents in NHs with culture change often live in smaller units, have control over their daily schedules, enjoy 24/7 visiting hours, and experience consistent staffing. Evidence indicates that this person-centered approach to NH care may lead to improved outcomes and quality of life for residents.
      • Zimmerman S.
      • Shier V.
      • Saliba D.
      Transforming nursing home culture: evidence for practice and policy.
      • Grabowski D.C.
      • O'Malley A.J.
      • Afendulis C.C.
      • et al.
      Culture change and nursing home quality of care.
      • Shier V.
      • Khodyakov D.
      • Cohen L.W.
      • Zimmerman S.
      • Saliba D.
      What does the evidence really say about culture change in nursing homes?.
      In addition, the implementation of NH culture change aligns with current regulatory initiatives such as the 2016 NH Medicare/Medicaid Long-Term Care regulations
      Medicare and Medicaid Programs
      Reform of requirements for long term care facilities federal register.
      and the Patient Protection and Affordable Care Act (2010),
      both of which encourage a transition toward more person-centered care.
      Although the NH culture change movement could potentially benefit all residents, it may be particularly valuable for residents near the end of life. Over 20% of Americans die in NHs
      • Temkin-Greener H.
      • Zheng N.T.
      • Xing J.
      • Mukamel D.B.
      Site of death among nursing home residents in the United States: changing patterns, 2003–2007.
      • Gruneir A.
      • Mor V.
      • Weitzen S.
      • et al.
      Where people die: a multilevel approach to understanding influences on site of death in America.
      ; however, the quality of care for these residents is often insufficient.
      • Oliver D.P.
      • Porock D.
      • Zweig S.
      End-of-life care in U.S. nursing homes: a review of the evidence.
      • Mukamel D.B.
      • Caprio T.
      • Ahn R.
      • et al.
      End-of-life quality-of-care measures for nursing homes: place of death and hospice.
      • Saliba D.
      • Kington R.
      • Buchanan J.
      • et al.
      Appropriateness of the decision to transfer nursing facility residents to the hospital.
      • Teno J.M.
      • Clarridge B.R.
      • Casey V.
      • et al.
      Family perspectives on end-of-life care at the last place of care.
      • Teresi J.
      • Abrams R.
      • Holmes D.
      • Ramirez M.
      • Eimicke J.
      Prevalence of depression and depression recognition in nursing homes.
      • Travis S.S.
      • Loving G.
      • McClanahan L.
      • Bernard M.
      Hospitalization patterns and palliation in the last year of life among residents in long-term care.
      For this vulnerable population, the alignment of care with resident values, goals, and preferences can reduce suffering and improve the quality of care at the end of life.
      Despite the potential value of person-centered care for individuals at the end of life, and the growing evidence regarding the effectiveness of NH culture change, there are no studies that differentiate general NH culture change practices from practices near or after a resident's death (i.e., end-of-life culture change (EOL-CC) practices). Although two research teams have developed comprehensive scales to measure end-of-life (EOL) and palliative care in NHs, their analyses were limited to state or regional data, and neither focused specifically on culture change practices.
      • Temkin-Greener H.
      • Zheng (Tracy) N.
      • Norton S.A.
      • et al.
      Measuring end-of-life care processes in nursing homes.
      • Thompson S.
      • Bott M.
      • Boyle D.
      • Gajewski B.
      • Tilden V.P.
      A measure of palliative care in nursing homes.
      Evidence suggests that a wide range of factors contribute to the likelihood of an NH implementing culture change practices. These factors include Medicaid reimbursement policies, local competition, NH financial resources, staffing, staff leadership opportunities, and resident case mix.
      • Grabowski D.C.
      • O'Malley A.J.
      • Afendulis C.C.
      • et al.
      Culture change and nursing home quality of care.
      • Hermer L.
      • Cornelison L.
      • Kaup M.L.
      • et al.
      The Kansas PEAK 2.0 program facilitates the diffusion of culture-change innovation to unlikely adopters.
      • Miller S.C.
      • Looze J.
      • Shield R.
      • et al.
      Culture change practice in U.S. Nursing homes: prevalence and variation by state medicaid reimbursement policies.
      • Miller S.C.
      • Schwartz M.L.
      • Lima J.C.
      • et al.
      The prevalence of culture change practice in US nursing homes: findings from a 2016/2017 nationwide survey.
      Despite the growing literature about NH culture change, we know very little about which, if any, of these factors are associated with culture change practices specific to EOL care. Temkin-Greener (2009) evaluated EOL care practices across NHs in New York state and found that NHs that emphasized EOL quality assurance, monitoring, and education had higher adoption of EOL practices, as did religious facilities and those with higher registered nurse to certified nursing assistant ratios.
      • Temkin-Greener H.
      • Zheng (Tracy) N.
      • Norton S.A.
      • et al.
      Measuring end-of-life care processes in nursing homes.
      One NH characteristic that may be particularly important to EOL-CC is the availability of specialty palliative care (PC) consulting services. For NH residents with advanced serious illness, PC offers services similar to hospice without the need for terminal diagnosis or forfeiture of curative treatments.
      • Hanson L.C.
      • Ersek M.
      Meeting palliative care needs in post-acute care settings: “to help them live until they die”.
      • Meier D.E.
      • Lim B.
      • Carlson M.D.A.
      Raising the standard: palliative care in nursing homes.
      • Unroe K.T.
      • Ersek M.
      • Cagle J.
      The IOM report on dying in America: a call to action for nursing homes.
      Much like hospice, PC focuses on improving the quality of life and managing pain for residents with chronic illness, through resident-directed care that addresses physical, emotional, social, cultural, and spiritual needs.
      • Meier D.E.
      • Lim B.
      • Carlson M.D.A.
      Raising the standard: palliative care in nursing homes.
      • Miller S.C.
      • Lima J.C.
      • Intrator O.
      • et al.
      Palliative care consultations in nursing homes and reductions in acute care use and potentially burdensome end-of-life transitions.
      While PC and culture change practice have goals that align, high-quality culture change and PC practice do not necessarily occur simultaneously.
      • Tyler D.A.
      • Shield R.R.
      • Miller S.C.
      Diffusion of palliative care in nursing homes: lessons from the culture change movement.
      Evidence links the presence of PC in NHs with improved quality of life, fewer aggressive treatments at the end of life, reduced costs, and fewer hospitalizations.
      • Hermer L.
      • Cornelison L.
      • Kaup M.L.
      • et al.
      The Kansas PEAK 2.0 program facilitates the diffusion of culture-change innovation to unlikely adopters.
      • Miller S.C.
      • Schwartz M.L.
      • Lima J.C.
      • et al.
      The prevalence of culture change practice in US nursing homes: findings from a 2016/2017 nationwide survey.
      • Hanson L.C.
      • Ersek M.
      Meeting palliative care needs in post-acute care settings: “to help them live until they die”.
      • Meier D.E.
      • Lim B.
      • Carlson M.D.A.
      Raising the standard: palliative care in nursing homes.
      • Unroe K.T.
      • Ersek M.
      • Cagle J.
      The IOM report on dying in America: a call to action for nursing homes.
      • Miller S.C.
      • Lima J.C.
      • Intrator O.
      • et al.
      Palliative care consultations in nursing homes and reductions in acute care use and potentially burdensome end-of-life transitions.
      Despite these benefits, there is no standard approach for incorporating PC into NHs. Some NHs contract with hospice programs to provide external specialty PC consults, whereas others invest in building internal specialty PC consulting programs. Little is known about the differences between these approaches, or how the availability of PC consults in NHs may be associated with person-centered EOL practices.
      This study had two objectives. The first was to evaluate the presence of person-centered EOL-CC practices in a nationally representative sample of NHs. The second was to identify NH characteristics that are associated with higher levels of EOL-CC practices. Given the potential overlap in the goals and outcomes of PC consults and EOL care, we were particularly interested in the relationship between the availability of and approach to PC consults and the presence of EOL-CC practices in NHs.

      Methods

      Study Design and Population

      This cross-sectional analysis is part of a larger longitudinal project designed to evaluate the impact of shifts in NH culture change practices over time. In the first phase of the project, administrators at 2142 NHs completed a survey about culture change practices at their facilities. The original 2009/2010 survey represented a stratified random sample of U.S. NHs, and no observable nonresponse bias was detected.
      • Clark M.
      • Rogers M.
      • Foster A.
      • et al.
      A randomized trial of the impact of survey design characteristics on response rates among nursing home providers.
      The 2009/2010 survey did not specifically address the person-centeredness of practices near or after a resident's death.
      • Miller S.C.
      • Looze J.
      • Shield R.
      • et al.
      Culture change practice in U.S. Nursing homes: prevalence and variation by state medicaid reimbursement policies.
      This study uses data from the second phase of the project conducted in 2016/2017. We administered a follow-up survey, the Nursing Home Culture Change Survey (NHCCS), to NHs whose administrators completed the original survey. An expanded set of survey items were developed based on the Holistic Approach to Transformational Change (HATCh) model, which describes interrelated domains of culture change needed for NHs to transform from institutional to individualized care.
      Quality Partners of Rhode Island
      The Holistic Approach to Transformational Change (HATCh).
      Items were developed with guidance from current literature and our expert advisory committee, and cognitive testing was conducted for all survey items. Seventy-four percent (1584) of NH administrators completed the NHCCS with no observable nonresponse bias detected.
      • Miller S.C.
      • Schwartz M.L.
      • Lima J.C.
      • et al.
      The prevalence of culture change practice in US nursing homes: findings from a 2016/2017 nationwide survey.
      Of these, 90 NHs completed a short version of the survey, leaving 1494 NHs with a fully completed questionnaire. Details regarding methodology for survey development, data collection, and overall results are reported elsewhere.
      • Miller S.C.
      • Schwartz M.L.
      • Lima J.C.
      • et al.
      The prevalence of culture change practice in US nursing homes: findings from a 2016/2017 nationwide survey.
      We excluded 35 (2.3%) NHs missing an EOL-CC score. In addition, 101 (6.8%) NHs were excluded because they were missing data for one or more of the explanatory variables included in the multivariable regression models. Our final study sample included 1358 (90.9%) NHs.

      End-of-Life Culture Change Index

      The EOL-CC index comprised six questions related to person-centered practices for residents who were dying/had died. These items were originally included in an index of resident-centered care practices; however, principal component analysis demonstrated that they represented a separate component reflective of resident-centered EOL practices. Each of the six EOL-CC questions was rated on a scale of one (never occurs) to three (almost always occurs). NHs with responses to at least four EOL-CC items were included in the study sample. Missing responses on one or two items were imputed for 132 NHs using the mean of the nonmissing EOL-CC items within the facility. Responses were summed across all items to derive the facility level EOL-CC score, with a minimum possible score of six and a maximum possible score of 18 points. A list of survey items included in the index, the weighted frequencies of responses to each item, and the points assigned for each response are available in Table 1. The EOL-CC score was developed to represent a composite variable, or index of EOL-CC practices, rather than a construct with underlying latent variables.
      • Edwards J.R.
      • Bagozzi R.P.
      On the nature and direction of relationships between constructs and measures.
      Table 1End-of-Life Culture Change Items and Scoring
      Survey ItemsWeighted Percentages of Responses [Point Value Assigned]
      How Often Does Your Facility…RarelySometimesOftenAlmost AlwaysDon't Know
      [1][1][2][3][Missing]
      Discuss a resident's spiritual needs at care planning conferences when the resident has an acute or chronic terminal illness?2.5%13.1%26.4%56.9%1.1%
      Document in the care plan of a terminally ill resident what is important to the individual at the end of life, such as the presence of family or religious or cultural practices?3.3%11.9%25.5%56.3%3.0%
      Honor in some public way (either at the facility or in the community) a resident who has died?13.8%20.7%20.0%44.6%0.9%
      Honor the resident's body in some manner upon its removal from the facility?43.7%10.6%11.1%29.9%4.8%
      Send a sympathy card to family members or significant others after a resident has died?4.4%7.8%17.0%69.5%1.4%
      Follow-up with roommate(s) or friend(s) in the facility to provide emotional support after a resident has died?4.5%11.3%26.2%56.9%1.0%

      Nursing Home Characteristics

      We identified several NH characteristics that may be associated with EOL-CC. Many of these were self-reported by NHs as part of their NHCCS. The NH leadership score
      • Miller S.C.
      • Schwartz M.L.
      • Lima J.C.
      • et al.
      The prevalence of culture change practice in US nursing homes: findings from a 2016/2017 nationwide survey.
      represents an index of 10 survey items that measure the extent of NH leadership practices to ensure that leaders model and enable culture change. Leadership questions were scored from one (rarely occurs) to three (almost always occurs) points and summed for a maximum possible score of 30.
      The availability of PC consults was derived from two survey items. NHs were asked “Does your organization have its own palliative care consulting program staffed by nurse and physician palliative care specialists?” and “Does your organization have an arrangement with an external provider for nonhospice palliative care consulting?” Based on responses to these questions, we categorized NHs into three groups: 1) no PC consults, 2) only external PC consults, and 3) any internal PC consults. The “any internal PC consults” group included NHs with only internal, but no external PC consults, as well as those with both internal and external PC consults. Because internal PC consult programs require a more significant investment of time and resources than internal programs, we felt it was most appropriate to differentiate between NHs with at least some mention of internal programs and those without.
      Additional NH, state, and region level variables were obtained from 2015 Certification and Survey Provider Enhanced Reporting data and a 2011 policy survey conducted by Brown University. Resident case-mix variables were derived from the Medicare Minimum Data Set, enrollment, and claims data and were modeled after variables found in the Long-Term Care Facts on Care in the United States (LTCfocus) data set.
      Brown University School of Public Health
      Long-term care: Facts on care in the US.
      We used a publicly available report compiled by the Medicaid and CHIP Payment and Access Commission, to determine whether states had Medicaid NH pay-for-performance (p4p) programs and whether these programs included a culture change-related quality indicator.
      Medicaid and CHIP Payment and Access Commission (MACPAC)
      States' Medicaid fee-for-Service nursing facility payment policies.
      Table 2 provides a complete list of the NH characteristics, variable definitions, data sources, and years of collection.
      Table 2Nursing Home Characteristics Included in the Generalized Ordered Logit Model
      VariableCodingData Source
      Palliative care consultsNone, external only, any internal2016/2017 NHCCS
      For profitYes/no2015 CASPER
      Bed count<80, 80–120, >1202015 CASPER
      Occupancy rateNumber of occupied beds/total number of beds (converted to five-point increments in regression model)2015 CASPER
      Part of chainYes/no2015 CASPER
      Religious affiliationYes/no2015 CASPER
      Continuing care retirement communityYes/no

      Yes indicates answered “Yes” to “Is your facility part of a continuing care retirement community?” or answered “No,” but indicated they have long-term care beds, assisted living residences, and independent living residences.
      2016/2017 NHCCS
      Any special care unitYes/no, ventilator units not included2015 CASPER
      Directors of nursing in past two yrsOne, two, or more2016/2017 NHCCS
      Administrators in past two yrsOne, two, or more2016/2017 NHCCS
      RN hours per resident dayNumber of RN hours/number of residents in the facility (standardized in regression model)2015 CASPER
      LPN hours per resident dayNumber of LPN hours/number of residents in the facility (standardized in regression model)2015 CASPER
      CNA hours per resident dayNumber of CNA hours/number of residents in the facility (standardized in regression model)2015 CASPER
      Leadership scoreTotal points achieved on the culture change index of practices related to staff opportunities for leadership and decision making. Maximum possible score is 30 points (converted to three-point increments in regression model)2016/2017 NHCCS
      State pay-for-performanceNo/yes, but without culture change measures/yes with culture change measuresMACPAC
      Medicaid and CHIP Payment and Access Commission (MACPAC)
      States' Medicaid fee-for-Service nursing facility payment policies.
      report and review of Medicaid web sites
      2009 Medicaid Reimbursement RateDollars per resident day (converted to $10 increments in regression model)2011 Medicaid Policy Survey
      County level Hirschman-Herfindahl Competition Index0–1 (categorized to four score into quartiles in regression model)2015 CASPER
      Located in a metropolitan countyYes/no2015 CASPER
      Census subregionNine subregions: West North Central, West South Central, East North Central, East South Central, Middle Atlantic, Mountain New England, Pacific, South Atlantic2015 CASPER
      Percent of residents who are blackProportion (categorized to lowest decile, below median, above median, top decile in regression model)2015 LTCfocus
      Percent of residents who are HispanicProportion (categorized to below median, above median, top decile in regression model)2015 LTCfocus
      Percent of residents with MedicareProportion with Medicare as primary payer, (categorized to lowest decile, below median, above median, top decile in regression model)2015 LTCfocus
      Percent of residents with MedicaidProportion with Medicaid as primary payer, (categorized to lowest decile, below median, above median, top decile in regression model)2015 LTCfocus
      Average age of residentsYears2015 LTCfocus
      Percent of residents receiving hospice careProportion (categorized into quartiles in regression model)2015 LTCfocus
      Percent of residents with dementiaProportion (categorized into quartiles in regression model)2015 LTCfocus
      Average RUGS NCMIAverage Resource Utilization Group Nursing Case-Mix Index (standardized in regression model)2015 LTCfocus
      Average activities of daily living scoreAverage activities of daily living score for all residents (standardized in regression model)2015 LTCfocus
      Percent of residents with high cognitive function scorePercent of residents in the facility with a cognitive function score of 4 (severe impairment) (converted to 10 percentage point increments in regression model)2015 LTCfocus
      NHCCS = Nursing Home Culture Change Survey; CASPER= Certification and Survey Provider Enhanced Reporting; RN = registered nursing; LPN = licensed practical nurse; CNA = certified nursing assistant; MACPAC = Medicaid and CHIP Payment and Access Commission; LTCfocus = Long-Term Care Facts on Care in the United States; RUGS NCMI= Resource Utilization Group Nursing Case-Mix Index.

      Statistical Analyses

      Performance on End-of-Life Culture Change Index

      Frequencies were used to describe responses to the individual survey questions included in the EOL-CC index. Frequencies were weighted using probability weights to adjust for the stratified survey design. To assess overall performance on the EOL-CC index, we calculated the weighted distribution of performance. We also evaluated the internal consistency of the items included in the index using McDonald's omega based on polychoric correlations, calculated using the Pscyh package
      • Gadermann A.M.
      • Guhn M.
      • Zumbo B.D.
      Estimating ordinal reliability for likert-type and ordinal item response data: a conceptual, empirical, and practical guide.
      • Revelle W.
      An overview of the psych package.
      in R 3.2.3.
      R Core Team
      R: A language and environment for statistical computing.
      McDonald's omega considers the maximum variance explained by the six items included in the index.

      Description of Study Population

      We summarized NH characteristics using weighted means and SDs for continuous variables and frequencies and percentages for categorical variables.

      Nursing Home Characteristics and End-of-Life Culture Change

      NHs were stratified into quartiles of performance based on their EOL-CC index scores. The first quartile represented the lowest reported adoption, and each subsequent quartile represented greater reported adoption. We used a multivariable generalized ordered logistic regression model to examine the factors associated with higher EOL-CC scores. The generalized ordered regression model considers the ordered nature of the data but relaxes the proportional odds assumption and so produces three coefficients for each variable in the model. The first reflects the log-odds of an NH scoring (i.e., performing) in the top three quartiles (compared to the first) that is associated with a one-unit increase in each of the covariates of interest; the second the log-odds of performing in the top two quartiles (compared to bottom two); and the third, the log-odds of performing in the fourth quartile of the EOL-CC index (compared to the lower three quartiles). The model was adjusted for probability weights, and standard errors were clustered by state. All analyses (except for the McDonald's Omegas) were conducted using Stata version 14 (StataCorp. 2015, College Station, TX)

      Results

      Table 1 displays the weighted frequencies of responses to the individual EOL-CC survey items. Most NHs reported that they “almost always” discuss terminally ill residents' spiritual needs at the end of life (56.9%) and send a sympathy card to family members (69.5%). However, almost half (43.7%) never honor the resident's body upon removal from the facility.
      As displayed in Table 3, the mean weighted score on the EOL-CC index was 13.7 (SD = 3.0, interquartile range = 12.0–16.0). The EOL-CC index had high internal consistency, with a total omega score of 0.9. The descriptive characteristics of the study sample are available in Table 4.
      Table 3Performance for 1358 Nursing Homes on the End-of-Life Culture Change Index (2016–2017)
      Index StatisticsWeighted Distribution
      Total possible score18Minimum6.0
      25th percentile12.0
      Weighted mean (SD)13.7 (3.0)Median14.0
      75th percentile16.0
      Omega total0.9Maximum18.0
      Table 4Weighted Descriptive Statistics for Nursing Homes in Study Sample
      Nursing Home Characteristic%/MeanStandard Error
      Nursing home structure and staffing
       Palliative care consults
      None50.0%
      External only34.1%
      Any internal15.9%
       For profit67.7%
       Bed count
      <8034.0%
      80–12028.6%
      ≥12037.5%
       Occupancy rate83.40.4
       Part of a chain56.8%
       Religious affiliation3.7%
       Continuing care retirement community18.9%
       Any special care unit (except ventilator unit)19.6%
       Directors of nursing in past two yrs
      One50.8%
      Two or more49.2%
       Administrators in past two yrs
      One55.9%
      Two or more44.1%
       RN hours/resident day0.40.0
       LPN hours/resident day0.80.0
       CNA hours/resident day2.30.0
       Leadership score (points)18.30.1
      State policy and regional characteristics
       State pay-for-performance (2014)
      No57.9%
      Yes: without culture change measures29.7%
      Yes: with culture change measures12.5%
       2009 Medicaid reimbursement rate ($/NH day)$160.3$0.8
       County Hirschman-Herfindahl Competition Index
      First quartile (≤ 0.05)27.0%
      Second quartile (0.05–0.17)25.1%
      Third quartile (0.17–0.37)24.0%
      Fourth quartile (≥0.37)23.9%
       Metropolitan county66.8%
       Census subregion
      West North Central16.0%
      West South Central11.5%
      East North Central19.4%
      East South Central5.7%
      Middle Atlantic11.2%
      Mountain4.9%
      New England7.8%
      Pacific7.0%
      South Atlantic16.5%
      Resident case mix
       Percent black residents
      Lowest decile (0%)33.7%
      Below median (0.01%–2.27%)11.2%
      Above median (2.27%–29.87%)42.5%
      Top decile (>29.87%)12.6%
       Percent Hispanic residents
      Below median (0%)54.4%
      Above median (0%–9.56%)34.6%
      Top decile (>9.56%)11.0%
       Percent with Medicare
      Lowest decile (≤2.78)10.2%
      Below median (2.78%–11.11%)40.4%
      Above median (11.11%–26.03%)39.2%
      Top decile (>26.03%)10.2%
       Percent with Medicaid
      Lowest decile (<31.33)9.1%
      Below median (31.33%–62.50%)38.9%
      Above median (62.50%–83.33%)41.1%
      Top decile (>83.33%)10.9%
       Average age (yrs)80.50.2
       Percent receiving hospice
      First quartile (<=–1.77%)24.7%
      Second quartile (1.77%–4.31%)25.7%
      Third quartile (4.31%–7.75%)24.8%
      Fourth quartile (>7.75%)24.8%
       Percent with dementia
      First quartile (<=–36.70%)24.2%
      Second quartile (36.70%–48.37%)25.2%
      Third quartile (48.37%–58.75%)25.9%
      Fourth quartile (>0.37)24.8%
       Average RUGS NCMI1.30.0
       Average ADL score16.40.1
       Percent with high CFS14.00.3
      RN = registered nurse; LPN = licensed practical nurse; CNA = certified nursing assistant; RUGS NCMI = Resource Utilization Group Nursing Case-Mix Index; ADL = activities of daily living; CFS = cognitive function score.
      Table 5 displays the results of the generalized ordered logistic regression model. NHs with PC consulting programs were more likely to perform above the median on the EOL-CC index. NHs with external PC consults had 40% (95% CI: 1.05–1.83) higher odds of performing above the median on the EOL-CC index, and those with an internal program had 50% (95% CI: 1.10–1.93) higher odds of performing above the median.
      Table 5Nursing Home Characteristics and Performance on End-of-Life Culture Change Index, Ordered Logit Model
      Nursing Home CharacteristicQuartile of Performance on EOL-CC Index
      2,3,4 vs. 1
      The outcome of the ordered logit model was quartiles of EOL-CC performance. The first column assesses the odds of performing in the top three quartiles relative to the first quartile (reference group, lowest scores). The second column assesses the odds of performing above the median vs. below the median (reference group). The third column assesses the odds of performing in the fourth (highest scores) quartile, relative to the lowest three quartiles (reference group).
      3,4 vs. 1,2
      The outcome of the ordered logit model was quartiles of EOL-CC performance. The first column assesses the odds of performing in the top three quartiles relative to the first quartile (reference group, lowest scores). The second column assesses the odds of performing above the median vs. below the median (reference group). The third column assesses the odds of performing in the fourth (highest scores) quartile, relative to the lowest three quartiles (reference group).
      4 vs. 1,2,3
      The outcome of the ordered logit model was quartiles of EOL-CC performance. The first column assesses the odds of performing in the top three quartiles relative to the first quartile (reference group, lowest scores). The second column assesses the odds of performing above the median vs. below the median (reference group). The third column assesses the odds of performing in the fourth (highest scores) quartile, relative to the lowest three quartiles (reference group).
      OR (95% CI)OR (95% CI)OR (95% CI)
      Nursing home structure and staffing
       Palliative care consults
      NoneRefRefRef
      Only external1.2 (0.90–1.64)1.4
      P < 0.05.
      (1.05–1.83)
      1.1 (0.70–1.66)
      Any internal1.4 (0.99–1.89)1.5
      P < 0.01.
      (1.10–1.93)
      1.1 (0.72–1.75)
       For profit0.6
      P < 0.01.
      (0.43–0.86)
      0.6
      P < 0.01.
      (0.44–0.82)
      0.7
      P < 0.01.
      (0.49–0.90)
       Bed count
      <80RefRefRef
      80–1201.4 (0.97–1.96)1.1 (0.78–1.52)1.2 (0.84–1.85)
      >1201.2 (0.90–1.70)1.2 (0.85–1.57)1.5 (0.94–2.41)
       Occupancy rate (five-point increase)1.0 (0.97–1.08)1.0 (0.97–1.10)1.0 (0.95–1.07)
       Part of a chain1.1 (0.83–1.39)0.9 (0.72–1.26)1.1 (0.81–1.55)
       Religious affiliation1.7 (0.85–3.31)1.0 (0.59–1.67)2.5
      P < 0.01.
      (1.37–4.54)
       Continuing care retirement community1.2 (0.83–1.80)1.5
      P < 0.05.
      (1.033–2.06)
      1.2 (0.73–1.83)
       Any special unit (except ventilator unit)1.1 (0.68–1.77)0.9 (0.61–1.30)0.9 (0.59–1.30)
       Directors of nursing in past two yrs
      OneRefRefRef
      Two or more0.9 (0.67–1.19)1.0 (0.79–1.23)0.8 (0.64–1.11)
       Administrators in past two yrs
      OneRefRefRef
      Two or More0.9 (0.65–1.28)1.0 (0.77–1.30)1.1 (0.81–1.47)
      RN hours/resident day (standardized)1.2 (0.99–1.35)0.9 (0.79–1.09)0.9 (0.72–1.12)
      LPN hours/resident day (standardized)1.1 (0.94–1.31)1.0 (0.88–1.25)1.0 (0.82–1.23)
      CNA hours/resident day (standardized)0.9 (0.76–1.01)1.1 (0.91–1.30)1.2 (0.99–1.56)
      Leadership score (three-point increase)2.3
      P < 0.01.
      (2.04–2.69)
      2.1
      P < 0.01.
      (1.86–2.45)
      2.0
      P < 0.01.
      (1.82–2.30)
      State policy and regional characteristics
       State pay-for-performance (2014)
      NoRefRefRef
      Yes: no culture change measures0.7
      P < 0.05.
      (0.52–0.99)
      0.8 (0.54–1.08)1.2 (0.82–1.72)
      Yes: with culture change measures1.0 (0.65–1.39)0.8 (0.56–1.08)1.8
      P < 0.05.
      (1.02–3.01)
       2009 Medicaid reimbursement rate ($10/day increase)1.1 (0.98–1.13)1.0 (0.96–1.09)1.0 (0.93–1.09)
       County Hirschman-Herfindahl Competition Index
      First quartile (most competition)RefRefRef
      Second quartile0.8 (0.50–1.23)0.7 (0.51–1.05)1.0 (0.65–1.41)
      Third quartile1.2 (0.78–1.97)1.2 (0.69–2.02)1.4 (0.71–2.90)
      Fourth quartile (least competition)1.3 (0.76–2.16)0.9 (0.60–1.39)1.3 (0.63–2.55)
       Metropolitan county1.2 (0.85–1.59)1.3
      P < 0.05.
      (1.03–1.73)
      1.4 (0.96–2.06)
       Census subregion
      West North CentralRefRefRef
      West South Central0.4
      P < 0.01.
      (0.23–0.70)
      0.6
      P < 0.01.
      (0.45–0.86)
      0.7 (0.31–1.63)
      East North Central0.5
      P < 0.05.
      (0.27–0.89)
      0.6
      P < 0.05.
      (0.41–0.92)
      0.7 (0.41–1.20)
      East South Central0.3
      P < 0.01.
      (0.13–0.74)
      0.4
      P < 0.05.
      (0.21–0.86)
      0.9 (0.39–2.0)
      Middle Atlantic0.4
      P < 0.01.
      (0.17–0.72)
      0.3
      P < 0.01.
      (0.18–0.63)
      0.6
      P < 0.05.
      (0.33–0.95)
      Mountain0.5
      P < 0.05.
      (0.24–0.91)
      0.7 (0.33–1.53)0.6 (0.30–1.32)
      New England0.4
      P < 0.05.
      (0.17–0.88)
      0.5
      P < 0.05.
      (0.27–0.94)
      0.7 (0.32–1.51)
      Pacific0.5
      P < 0.01.
      (0.27–0.79)
      0.3
      P < 0.01.
      (0.18–0.46)
      0.6 (0.35–1.06)
      South Atlantic0.5
      P < 0.05.
      (0.23–0.88)
      0.5
      P < 0.05.
      (0.34–0.87)
      1.2 (0.66–2.07)
      Resident case mix
       Percent black residents
      Lowest decileRefRefRef
      Below median1.3 (0.74–2.18)0.5
      P < 0.01.
      (0.39–0.78)
      0.8 (0.49–1.17)
      Above median0.9 (0.58–1.52)0.6
      P < 0.05.
      (0.43–0.95)
      0.8 (0.48–1.26)
      Top decile0.9 (0.47–1.80)0.6 (0.30–1.12)0.6 (0.26–1.52)
       Percent Hispanic residents
      Below medianRefRefRef
      Above median0.9 (0.68–1.08)1.0 (0.81–1.27)0.9 (0.68–1.29)
      Top decile0.6
      P < 0.05.
      (0.43–0.91)
      0.7 (0.47–1.16)0.7 (0.43–1.26)
       Percent with Medicare
      Lowest decileRefRefRef
      Below median0.9 (0.57–1.56)1.0 (0.63–1.54)1.6 (0.81–3.09)
      Above median0.9 (0.53–1.64)1.1 (0.66–1.74)1.5 (0.73–3.09)
      Top decile1.6 (0.80–3.12)1.5 (0.79–2.76)1.8 (0.73–4.61)
       Percent with Medicaid
      Lowest decileRefRefRef
      Below median1.1 (0.65–1.94)1.7
      P < 0.05.
      (1.07–2.76)
      1.9 (0.91–4.15)
      Above decile1.2 (0.58–2.28)2.0
      P < 0.05.
      (1.03–3.69)
      2.2 (0.91–5.13)
      Top decile1.5 (0.65–3.58)2.4
      P < 0.05.
      (1.06–5.24)
      2.2 (0.84–5.91)
       Percent receiving hospice
      First quartile (fewest hospice residents)RefRefRef
      Second quartile1.6
      P < 0.05.
      (1.09–2.31)
      1.0 (0.70–1.37)0.9 (0.55–1.52)
      Third quartile1.4 (0.97–2.04)1.0 (0.73–1.51)0.9 (0.54–1.45)
      Fourth quartile (most hospice residents)2.0
      P < 0.05.
      (1.35–2.92)
      1.3 (0.96–1.76)1.1 (0.64–1.82)
       Percent with dementia
      First quartile (fewest dementia residents)RefRefRef
      Second quartile1.6
      P < 0.05.
      (1.09–2.28)
      1.1 (0.73–1.77)1.4 (0.82–2.27)
      Third quartile2.1
      P < 0.01.
      (1.47–2.90)
      1.5 (0.96–2.21)1.7
      P < 0.05.
      (1.04–2.85)
      Fourth quartile (most dementia residents)2.2
      P < 0.01.
      (1.33–3.59)
      1.2 (0.81–1.66)1.2 (0.63–2.09)
      Average RUGS NCMI (standardized)1.2 (0.93–1.45)1.3
      P < 0.01.
      (1.07–1.47)
      0.9 (0.62–1.06)
      Average ADL Score (standardized)0.8
      P < 0.05.
      (0.62–0.96)
      0.9 (0.76–1.14)1.1 (0.83–1.36)
      Percent high CFS (10% increase)1.0 (0.82–1.4)1.0 (0.86–1.11)0.9 (0.76–1.09)
      RN = registered nurse; LPN = licensed practical nurse; CNA = certified nursing assistant; RUGS NCMI = Resource Utilization Group Nursing Case-Mix Index; ADL = activities of daily living; CFS = cognitive function score.
      a The outcome of the ordered logit model was quartiles of EOL-CC performance. The first column assesses the odds of performing in the top three quartiles relative to the first quartile (reference group, lowest scores). The second column assesses the odds of performing above the median vs. below the median (reference group). The third column assesses the odds of performing in the fourth (highest scores) quartile, relative to the lowest three quartiles (reference group).
      b P < 0.05.
      c P < 0.01.
      There was a significant and consistent association between the leadership score and EOL-CC score. For example, a three-point increase in NH leadership score was associated with an NH having 2.3 (95% CI: 2.04–2.69) greater odds of performing in the top three quartiles (relative to the lowest quartile), and 2.1 (95% CI: 1.86–2.45) greater odds of performing above the median.
      A higher percent of patients receiving hospice care and a higher percent with dementia were both associated with an increased likelihood of performing in the top three quartiles of the EOL-CC index. NHS with the “most dementia residents” had 2.2 greater odds (96% CI: 1.33–3.59) of performing in the top three quartiles, relative to those with the “fewest dementia residents.” However, this association was not consistent for NHs with scores above the median or in the highest quartile.
      Compared to nonprofit facilities, for-profit NHs had 40% lower odds of performing in the top three quartiles, 40% lower odds of performing above the median, and 30% lower odds of performing in the top quartile. The region in which NHs were located was also associated with their EOL-CC index scores (Table 5). We did not observe a relationship between state Medicaid policies (i.e., NH reimbursement rates or pay-for-performance) and EOL-CC scores.

      Discussion

      Using a nationally representative survey of U.S. NHs, we found ample room for NHs to increase their reported adoption of person-centered EOL care practices. Although most NHs have adopted some components of person-centered EOL care practices, uptake of these practices varies across NHs. Using our EOL-CC index, which has strong internal reliability (0.9), the average NH scored 13.7 out of a possible 6–18 points.
      Only two variables were consistently associated with performance on the EOL-CC index across all quartiles of performance: leadership score and profit status. NHs with higher leadership scores (i.e., those with greater reported adoption of leadership practices that model and enable culture change) had higher scores on the EOL-CC index. In our other work, we observed a similar relationship between the NH leadership score and the culture change domains of physical environment, person-centered care, staff empowerment, and family and community engagement.
      • Miller S.C.
      • Schwartz M.L.
      • Lima J.C.
      • et al.
      The prevalence of culture change practice in US nursing homes: findings from a 2016/2017 nationwide survey.
      This current finding further emphasizes the importance of high-quality NH leadership in facilitating the adoption of NH culture change for all residents. In addition, similar to findings in other studies, we found that for-profit NHs consistently reported adopting fewer person-centered EOL care practices than not-for-profit NHs.
      • Grabowski D.C.
      • O'Malley A.J.
      • Afendulis C.C.
      • et al.
      Culture change and nursing home quality of care.
      • Hermer L.
      • Cornelison L.
      • Kaup M.L.
      • et al.
      The Kansas PEAK 2.0 program facilitates the diffusion of culture-change innovation to unlikely adopters.
      Several NH characteristics were associated with increased odds of performance for some but not all of the EOL-CC index quartile comparisons. These inconsistent findings are more difficult to interpret but are important to consider. We observed a relationship between the use of PC consults and performance above the median on the EOL-CC index, particularly for NHs with internal PC consulting programs. Because developing an internal PC program requires a substantial investment of resources, it is not surprising that these NHs also invest in person-centered EOL practices. Although this finding indicates an association between PC consulting services and the person-centeredness of EOL practices, it is important to note that we did not observe statistically significant differences when we assessed the odds of performing in the highest or lowest quartiles. Thus, although the presence of PC consults may help differentiate between above and below average facilities in terms of EOL-CC, it is not an indicator of particularly high or low adoption of person-centered practices for residents who are dying/have died. In addition, we do not know if the presence of PC consults leads to the adoption of EOL culture change, or if a focus on person-centered EOL practices leads NHs to consider investing in PC consults. Further analysis regarding the relationship between PC consults, person-centered EOL practices and outcomes will help inform both NHs and policymakers as they consider investing resources in PC programs and culture change.
      Similarly, we found that increases in both the percent of residents receiving hospice care and the percent with dementia were associated with an increased likelihood of performing above the lowest quartile on the EOL-CC index, but not for performance above the median or in the top quartile. Again, we cannot determine the direction of these relationships.
      Prior studies indicate a relationship between Medicaid policies, such as reimbursement rates or pay-for-performance, and NH culture change adoption.
      • Hermer L.
      • Cornelison L.
      • Kaup M.L.
      • et al.
      The Kansas PEAK 2.0 program facilitates the diffusion of culture-change innovation to unlikely adopters.
      • Miller S.C.
      • Looze J.
      • Shield R.
      • et al.
      Culture change practice in U.S. Nursing homes: prevalence and variation by state medicaid reimbursement policies.
      We found neither to be associated with the EOL-CC score. Although increasingly more states are rewarding NHs for implementing culture change practices, the majority of these programs are not focused on specific populations, such as on residents near the end of life. Although other culture change practices can certainly benefit dying residents, person-centered practices that focus on residents near or after death should also be considered. Additional research is needed to rigorously measure EOL-CC practices and evaluate their impact on outcomes and quality of life for residents with advanced serious illness.
      There are some limitations to this study. We lost approximately 7% of our study sample due to missing NH characteristics. The majority (4.7%) of excluded NHs were missing data for the leadership score. Upon closer examination, we found that larger nursing homes were more likely to be missing leadership scores. Thus, it is possible that the generalizability of the relationship between leadership and EOL-CC scores is somewhat limited for the largest NHs. We did not observe any other statistically significant differences between the characteristics of the excluded and included study samples. The mean weighted performance on the EOL-CC index for the excluded NHs was identical to that of the study sample.
      There are a few concerns regarding the use of survey data to address our research questions. We are aware that many NHs in our survey experienced high levels of administrator turnover, which may have impacted their ability to answer questions accurately. However, we used tailored follow-up protocols to allow new administrators to become sufficiently knowledgeable about the facility before requesting survey completion. In addition, although we did cognitive testing on all our survey items, it is possible that some administrators interpreted questions differently than we intended. Furthermore, NH administrators may have felt some pressure to respond to questions in specific ways, thus resulting in social desirability bias. In conducting cognitive testing of our survey items, we found that responses from administrators demonstrated less social desirability bias when compared to responses provided by directors of nursing,
      • Shield R.
      • Tyler D.
      • Berridge C.
      • Clark M.A.
      • Miller S.C.
      Innovative use of cognitive interviewing for nursing home research.
      and prior research suggests that administrators are credible sources regarding NH culture change practices.
      • Tyler D.A.
      • Shield R.R.
      • Miller S.C.
      Diffusion of palliative care in nursing homes: lessons from the culture change movement.
      • Shield R.R.
      • Looze J.
      • Tyler D.
      • Lepore M.
      • Miller S.C.
      Why and how do nursing homes implement culture change practices? Insights from qualitative interviews in a mixed methods study.
      • Tyler D.A.
      • Shield R.R.
      • Rosenthal M.
      • et al.
      How valid are the responses to nursing home survey questions? Some issues and concerns.
      Finally, we cannot determine the consistency across NHs regarding practice implementation. For example, in some NHs, discussing a resident's spiritual and religious needs may represent simply asking residents if they would like to speak to a chaplain, whereas in other NHs, it may represent a lengthy discussion. Despite these limitations, our survey had a high response rate (73.9%), no detectable response bias, and included a nationally representative sample of U.S. NHs.
      In conclusion, our findings indicate that although many NHs have adopted person-centered EOL practices, there is wide variation in practice across the U.S. Although the presence of PC consulting services in NHs is associated with a greater presence of person-centered EOL practices, our findings suggest there is still unexplained variation. In addition, although some factors may be associated with both EOL-CC and other culture change practices, EOL practices are unique, and NHs that adopt person-centered care practices specific to residents who are dying may be different from those that invest in other culture change practices. To better inform policy change, rigorous research is needed to specifically assess the implementation and effectiveness of person-centered EOL practices for the vulnerable population of persons dying in NHs (and their families).

      Disclosures and Acknowledgments

      This work was supported by the National Institutes on Aging (grant number NIA R01 AG048940-01A1). Schwartz received funding from the National Institutes of Aging (NIA) during the conduct of this study. Dr. Lima received funding from the NIA during the conduct of this study. Dr. Lima also received funding from NIA outside of the submitted work in 2016. Dr. Clark received funding from the NIA during the conduct of this study. Dr. Miller received funding from the NIA during the conduct of this study.

      Appendix A. Supplementary Data

      References

        • Koren M.J.
        Person-centered care for nursing home residents: the culture-change movement.
        Health Aff (Millwood). 2010; 29: 312-317
        • Rahman A.N.
        • Schnelle J.F.
        The nursing home culture-change movement: recent past, present, and future directions for research.
        Gerontologist. 2008; 48: 142-148
        • Zimmerman S.
        • Shier V.
        • Saliba D.
        Transforming nursing home culture: evidence for practice and policy.
        Gerontologist. 2014; 54: S1-S5
        • Grabowski D.C.
        • O'Malley A.J.
        • Afendulis C.C.
        • et al.
        Culture change and nursing home quality of care.
        The Gerontologist. 2014; 54: S35-S45
        • Shier V.
        • Khodyakov D.
        • Cohen L.W.
        • Zimmerman S.
        • Saliba D.
        What does the evidence really say about culture change in nursing homes?.
        Gerontologist. 2014; 54: S6-S16
        • Medicare and Medicaid Programs
        Reform of requirements for long term care facilities federal register.
        Sect. 2016; 81
      1. 111th Congress. The Patient Protection and Affordable Care Act (PPACA). 2010: 111-148
        • Temkin-Greener H.
        • Zheng N.T.
        • Xing J.
        • Mukamel D.B.
        Site of death among nursing home residents in the United States: changing patterns, 2003–2007.
        J Am Med Dir Assoc. 2013; 14: 741-748
        • Gruneir A.
        • Mor V.
        • Weitzen S.
        • et al.
        Where people die: a multilevel approach to understanding influences on site of death in America.
        Med Care Res Rev. 2007; 64: 351-378
        • Oliver D.P.
        • Porock D.
        • Zweig S.
        End-of-life care in U.S. nursing homes: a review of the evidence.
        J Am Med Dir Assoc. 2005; 6: S21-S30
        • Mukamel D.B.
        • Caprio T.
        • Ahn R.
        • et al.
        End-of-life quality-of-care measures for nursing homes: place of death and hospice.
        J Palliat Med. 2012; 15: 438-446
        • Saliba D.
        • Kington R.
        • Buchanan J.
        • et al.
        Appropriateness of the decision to transfer nursing facility residents to the hospital.
        J Am Geriatr Soc. 2000; 48: 154-163
        • Teno J.M.
        • Clarridge B.R.
        • Casey V.
        • et al.
        Family perspectives on end-of-life care at the last place of care.
        JAMA. 2004; 291: 88-93
        • Teresi J.
        • Abrams R.
        • Holmes D.
        • Ramirez M.
        • Eimicke J.
        Prevalence of depression and depression recognition in nursing homes.
        Soc Psychiatry Psychiatr Epidemiol. 2001; 36: 613-620
        • Travis S.S.
        • Loving G.
        • McClanahan L.
        • Bernard M.
        Hospitalization patterns and palliation in the last year of life among residents in long-term care.
        Gerontologist. 2001; 41: 153-160
      2. Committee on approaching death: addressing key end of life issues, Institute of Medicine. Dying in America: Improving quality and honoring individual preferences near the end of life. National Academies Press (US), Washington (DC)2015
        • Temkin-Greener H.
        • Zheng (Tracy) N.
        • Norton S.A.
        • et al.
        Measuring end-of-life care processes in nursing homes.
        Gerontologist. 2009; 49: 803-815
        • Thompson S.
        • Bott M.
        • Boyle D.
        • Gajewski B.
        • Tilden V.P.
        A measure of palliative care in nursing homes.
        J Pain Symptom Manage. 2011; 41: 57-67
        • Hermer L.
        • Cornelison L.
        • Kaup M.L.
        • et al.
        The Kansas PEAK 2.0 program facilitates the diffusion of culture-change innovation to unlikely adopters.
        Gerontologist. 2018; 58: 530-539
        • Miller S.C.
        • Looze J.
        • Shield R.
        • et al.
        Culture change practice in U.S. Nursing homes: prevalence and variation by state medicaid reimbursement policies.
        Gerontologist. 2014; 54: 434-445
        • Miller S.C.
        • Schwartz M.L.
        • Lima J.C.
        • et al.
        The prevalence of culture change practice in US nursing homes: findings from a 2016/2017 nationwide survey.
        Med Care. 2018; 56: 985-993
        • Hanson L.C.
        • Ersek M.
        Meeting palliative care needs in post-acute care settings: “to help them live until they die”.
        JAMA. 2006; 295: 681-686
        • Meier D.E.
        • Lim B.
        • Carlson M.D.A.
        Raising the standard: palliative care in nursing homes.
        Health Aff (Millwood). 2010; 29: 136-140
        • Unroe K.T.
        • Ersek M.
        • Cagle J.
        The IOM report on dying in America: a call to action for nursing homes.
        J Am Med Directors Assoc. 2015; 16: 90-92
        • Miller S.C.
        • Lima J.C.
        • Intrator O.
        • et al.
        Palliative care consultations in nursing homes and reductions in acute care use and potentially burdensome end-of-life transitions.
        J Am Geriatr Soc. 2016; 64: 2280-2287
        • Tyler D.A.
        • Shield R.R.
        • Miller S.C.
        Diffusion of palliative care in nursing homes: lessons from the culture change movement.
        J Pain Symptom Manage. 2015; 49: 846-852
        • Clark M.
        • Rogers M.
        • Foster A.
        • et al.
        A randomized trial of the impact of survey design characteristics on response rates among nursing home providers.
        Eval Health Prof. 2011; 34: 464-486
        • Quality Partners of Rhode Island
        The Holistic Approach to Transformational Change (HATCh).
        2005: 1-12 (Providence, RI)
        • Edwards J.R.
        • Bagozzi R.P.
        On the nature and direction of relationships between constructs and measures.
        Psychol Methods. 2000; 5: 155-174
        • Brown University School of Public Health
        Long-term care: Facts on care in the US.
        (Available from:)
        http://ltcfocus.org/
        Date accessed: June 5, 2018
        • Medicaid and CHIP Payment and Access Commission (MACPAC)
        States' Medicaid fee-for-Service nursing facility payment policies.
        (Available from:)
        • Gadermann A.M.
        • Guhn M.
        • Zumbo B.D.
        Estimating ordinal reliability for likert-type and ordinal item response data: a conceptual, empirical, and practical guide.
        Pract Assess Res Eval. 2012; 17
        • Revelle W.
        An overview of the psych package.
        (Available from:)
        http://www.personalitytheory.org/r/book/overview.pdf
        Date: 2011
        Date accessed: June 5, 2018
        • R Core Team
        R: A language and environment for statistical computing.
        (Available from:)
        https://www.r-project.org/
        Date: 2015
        Date accessed: June 5, 2018
        • Shield R.
        • Tyler D.
        • Berridge C.
        • Clark M.A.
        • Miller S.C.
        Innovative use of cognitive interviewing for nursing home research.
        J Am Med Dir Assoc. 2018; 19: 1133-1135
        • Shield R.R.
        • Looze J.
        • Tyler D.
        • Lepore M.
        • Miller S.C.
        Why and how do nursing homes implement culture change practices? Insights from qualitative interviews in a mixed methods study.
        J Appl Gerontol. 2014; 33: 737-763
        • Tyler D.A.
        • Shield R.R.
        • Rosenthal M.
        • et al.
        How valid are the responses to nursing home survey questions? Some issues and concerns.
        Gerontologist. 2011; 51: 201-211