key: cord-0274139-b89u21sg
authors: Mak, J. K. L.; Hagg, S.; Eriksdotter, M.; Annetorp, M.; Kuja-Halkola, R.; Kananen, L.; Bostrom, A.-M.; Kivipelto, M.; Metzner, C.; Back Jerlardtz, V.; Engstrom, M.; Johnson, P.; Lundberg, L. G.; Akesson, E.; Suhl Oberg, C.; Olsson, M.; Cederholm, T.; Jylhava, J.; Religa, D.
title: Development of an electronic frailty index for hospitalized geriatric patients in Sweden
date: 2021-11-28
journal: nan
DOI: 10.1101/2021.11.26.21266912
sha: 29e6375219efe64af9c49096f81ab54fb3655e4c
doc_id: 274139
cord_uid: b89u21sg

Background: Frailty assessment in the Swedish health system relies on the Clinical Frailty Scale (CFS), but it requires training, in-person evaluation, and is often missing in medical records. We aimed to develop an electronic frailty index (eFI) from routinely collected electronic health records (EHRs) and assess its predictive ability for adverse outcomes in geriatric patients. Methods: EHRs were extracted for 18,225 geriatric patients with unplanned admissions between 1/3/2020 and 17/6/2021 from nine geriatric clinics in Stockholm, Sweden. A 48-item eFI was constructed using diagnostic codes, functioning and other health indicators, and laboratory data. The CFS, Hospital Frailty Risk Score, and Charlson Comorbidity Index were used for comparative assessment of the eFI. We modelled in-hospital mortality and 30-day readmission using logistic regression; 30-day and 6-month mortality using Cox regression; and length of stay using linear regression. Results: 13,188 patients were included in analyses (mean age 83.1 years). A 10% increment in the eFI was associated with higher risks of in-hospital (odds ratio: 5.34; 95% confidence interval: 4.20-6.82), 30-day (hazard ratio [HR]: 3.28; 2.91-3.69), and 6-month mortality (HR: 2.70; 2.52-2.90) adjusted for age and sex. Of the frailty and comorbidity measures, the eFI had the best predictive accuracy for in-hospital mortality, yielding an area under receiver operating characteristic curve of 0.813. Higher eFI also predicted a longer length of stay, but had a rather poor discrimination for 30-day readmission. Conclusions: An EHR-based eFI has good predictive accuracy for adverse outcomes, suggesting that it can be used in risk stratification in geriatric patients.

With the ageing of the global population, it becomes increasingly important to identify and support older adults who are at the greatest risk of adverse outcomes. Frailty, characterized by reduced physiological reserve and increased vulnerability to stressors (1) , is a measure that captures such risk. Frailty has consistently been linked to negative health outcomes, such as mortality (2, 3) , hospitalization (4) , and increased healthcare costs (5, 6) . Despite the lack of a universal consensus on how to best assess frailty, the most widely used and validated frailty models are the physical frailty phenotype (characterized by weight loss, weakness, slowness, inactivity, and exhaustion) (7) and the frailty index (FI; multidimensional deficit accumulation) (8) . Nevertheless, these measures are time-and resource-consuming, which limits their incorporation into routine clinical practice (9) . One of the most frequently adopted frailty measure in clinical settings is the Clinical Frailty Scale (CFS) (10), which is a quick and simple screening tool and often has high accuracy and feasibility (11). However, as an instrument that requires in-person assessment, it may be subject to interrater error and may not always be available at hospital admission (12) (13) (14) .

Automated frailty scores, based on routinely collected electronic health records (EHRs) or administrative claims data, have recently been developed in certain countries such as the UK (15) , the US (16, 17) , and Canada (18) . One of the first models was the electronic frailty index (eFI) proposed by Clegg et al. (15) , who created it using the UK primary care Read codes on the basis of the Rockwood deficit accumulation approach (8) .

its ability to predict mortality and prolonged length of stay in hospitalized older patients (23) (24) (25) . However, its composition of ICD-10 diagnoses makes it more similar to a comorbidity measure, possibly missing out other frailty aspects such as functioning (22) .

To date, frailty is not yet routinely assessed in all older adults in Sweden, but the CFS has started to be implemented in Stockholm, at least in geriatric clinics. Routine frailty screening can help identifying patients who would benefit most from a Comprehensive Geriatric Assessment (CGA) (26) . To reduce the burden of bedside frailty assessment and aid in risk stratification, there is also an increasing need to analyze whether an eFI can be adapted to Swedish context. The objective of this study was therefore to develop an eFI using routinely collected EHRs in geriatric clinics in Stockholm, Sweden. For validation, we compared its predictive ability to other validated frailty and comorbidity measures, the CFS, HFRS, and the Charlson Comorbidity Index (CCI) (27) , considering mortality, readmission and the length of stay as outcomes.

We conducted a retrospective cohort study using electronic medical records in the Stockholm Region. Between 1 March 2020 and 17 June 2021, patients with unplanned admissions to nine geriatric clinics treated for any causes, except COVID-19, were included. COVID-19 patients (defined based on ICD-10 codes of U07.1, U07.2, U08.9, U09.9, or U10.9) often have different characteristics and prognosis and were analyzed in a separate paper. Exclusion criteria included admissions without discharge information or with a length of stay <24h. For patients with multiple admissions during the study period, the first available admission was considered in the analyses. In total, 18,225 non-COVID-19 geriatric patients were included, of whom 13,188 (72.4%) had sufficient data for calculation of the eFI. A flowchart of sample selection is presented in Supplementary Figure 1 . This study was All rights reserved. No reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in 

Following the eFI model developed by Pajewski et al. (17) , we selected a total of 48 items for the construction of the eFI. The items consisted of deficits in three categories: (i) disease diagnoses, extracted from ICD-10 codes assigned during the current admission (29 items), (ii) functional data and other health indicators, such as falls, neuropsychological problems, and oral health (10 items), and (iii) laboratory measures (9 items (8), we calculated the eFI as the sum of deficit items divided by the total number, providing that the patient had ≥30 non-missing items (8) . For instance, a patient with 10 deficit points out of 45 non-missing items would receive an eFI of 10/45=0.22. As 60% (29 out of 48) of the eFI items were based on ICD-10 codes (which were non-missing for all participants), we required that at least half of the functioning and/or laboratory-based deficit items were non-missing in each individual. This was to avoid the lack of such items that are necessary in reaching the multidimensional definition of frailty, and prevent the eFI from having an overrepresentation of comorbidity items. Based on the distribution of the eFI in our sample and by adopting a previously used cut-off of 0.25 (28,29), we categorized patients into four groups: fit (eFI ≤0.15), mild frailty (0.15> eFI ≥0.2), moderate frailty (0.2> eFI ≥0.25), and severe frailty (eFI >0.25).

was categorized in three groups: CFS 1-3, CFS 4-5, CFS 6-9. Both the HFRS and CCI were calculated based on ICD-10 codes. The HFRS is based on 109 frailty-associated and differently weighted ICD-10 codes, and categorizes the individuals into low (<5), intermediate (5) (6) (7) (8) (9) (10) (11) (12) (13) (14) (15) and high risk (>15) frailty groups (22) . The CCI was computed using an algorithm adapted for Swedish settings (30) , and was considered as a continuous variable in all analyses.

Primary outcomes were in-hospital, 30-day, and 6-month all-cause mortality, calculated from the date of admission. Dates of death were extracted from the Population System in Sweden. Secondary outcomes were 30-day readmission to any of the nine included clinics, and the length of stay. Only patients discharged to home were included in the 30-day readmission analysis, i.e., excluding those who were transferred to another unit or care facility after the stay at the geriatric unit.

Patients' characteristics were summarized and compared by sex using t-tests or Mann-Whitney U tests for continuous variables, and Pearson χ 2 tests for categorical variables.

Spearman's correlations (ρ) were calculated between continuous frailty and comorbidity measures.

We fitted multivariable regression models to compare the eFI to the CFS, HFRS, and CCI in predicting different outcomes, adjusting for age and sex in all models. In-hospital mortality and 30-day readmission were modelled using logistic regression, with odds ratios (ORs) and 95% confidence intervals (CI) presented. The predictive accuracy (discrimination) of the logistic regression models was examined using area under the receiver operating characteristic curve (AUC). Hazard ratios (HRs) and 95% CI for 30-day and 6-month mortality were estimated using Cox proportional-hazards models. Harrell's C-statistics were All rights reserved. No reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in The copyright holder for this this version posted November 28, 2021. ; https://doi.org/10.1101/2021.11.26.21266912 doi: medRxiv preprint calculated for assessing predictive accuracy of Cox models, with 95% CI calculated through 1,000 bootstrapping resampling. Linear regression models were performed for the length of stay, and coefficients of determination (R 2 ) were calculated to assess the proportion of variation explained by the independent variables.

As a sensitivity analysis, we stratified the analysis of in-hospital mortality by the nine admitting clinics to assess the variation by hospitals.

All analyses were performed using R version 4.0.5.

Among the 13,188 patients who had sufficient data for calculation of the eFI, the mean age was 83.1 years and 60.2% were women; the in-hospital mortality rate was 1.4% and the median length of stay was 6.7 days ( Table 1) . The patients with and without sufficient data for calculation of the eFI were similar with regards to age, sex, frailty level according to CFS and HFRS, and in-hospital death rate (Supplementary Table 2 ).

The eFI was approximately normally distributed, with a median of 0.181 (interquartile range: 0.143-0.222; range: 0-0.432) (Supplementary Figure 2) . The proportions of patients classified as fit, mildly frail, moderately frail, and severely frail were 29.3%, 33.1%, 24.4%, and 13.2%, respectively. Although women were on average older than men (mean age: 84.1 vs. 81.6), men had significantly higher frailty scores than women when defined by the eFI (severe frailty: 14.6% vs. 12.3%) and the HFRS (intermediate or high risk: 28.0% vs. 24 .1%), but not by the CFS ( Table 1) Associations with in-hospital, 30-day, and 6-month mortality All rights reserved. No reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in The copyright holder for this this version posted November 28, 2021. ; https://doi.org/10.1101/2021.11.26.21266912 doi: medRxiv preprint After adjusting for age and sex, the eFI was strongly associated with in-hospital mortality (OR per 10% increase: 5.34, 95% CI: 4.20-6.82) ( Table 2) . The CFS, HFRS, and CCI also had positive associations with in-hospital mortality; however, among all the measures, the eFI had the greatest predictive accuracy for in-hospital mortality when added to a model with age and sex, yielding an AUC of 0.813 (95% CI: 0.769-0.857) ( Figure 1A) . As a sensitivity analysis, we stratified the sample by the nine clinics and found robust predictive performance of the eFI for in-hospital mortality across the clinics, with AUCs ranging from 0.745-0.871 (Supplementary Table 4 ). The associations were also essentially unchanged when included the admitting clinics as a covariate in the models, or stratified by sex and age (data not shown).

We observed a gradient of an increasing risk of mortality in the higher eFI categories compared to the lowest category (fit) in six months from admission (Figure 2) . Similar to inhospital mortality, a 10% rise in the eFI score was significantly associated with elevated risks of 30-day (HR: 3.28, 95% CI: 2.91-3.69) and 6-month mortality (HR: 2.70, 95% CI: 2.52-2.90) after adjusting for age and sex ( Table 2 ). The eFI models had the highest Harrell's Cstatistics (0.733 for 30-day mortality & 0.707 for 6-month mortality) in comparison to the other frailty and comorbidity measures, indicating that the eFI had good predictive accuracy for 30-day and 6-month mortality in the Cox models ( Figure 1B) .

Despite the significant associations between the eFI and 30-day readmission (OR per 10% increase: 1.50, 95% CI: 1.35-1.68), and likewise between the CFS, HFRS and CCI, and 30day readmission, the AUCs for these models were <0.6, indicating poor discrimination for readmission (Supplementary Table 5 ).

There was an approximately linear relationship between the eFI and the length of stay, regardless of age and sex (Figure 3) . Similarly, linear regression analysis showed that after adjusting for age and sex, a 10% increment in the eFI was on average associated with All rights reserved. No reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in Table 6 ).

Although it is well-known that frailty predicts adverse outcomes, traditional frailty measurements often require in-person assessment and may not always be feasible or available in clinical settings. There is a growing interest of deriving frailty measures from routinely collected health data to facilitate frailty screening in clinical practice. Several such eFI models have been developed for primary care, and inpatient and outpatient settings using diagnostic codes, health service codes, and clinical information (31) . Here we adapted the US eFI model (17) to Swedish EHR data, i.e. retrieved from geriatric clinics in the Stockholm region. We found that the eFI outperformed the currently available frailty and comorbidity scales (i.e., CFS, HFRS, and CCI) in predicting in-hospital mortality, yielding an AUC of 0.813 when added to a model with age and sex. It also had better accuracy for 30day and 6-month mortality than CFS and HFRS and was significantly associated with longer length of stay. Nevertheless, all the frailty scales and the CCI had a poor predictive accuracy for 30-day readmission.

Based on data availability and following the model developed by Pajewski et al. (17) , we included ICD-10 codes, functional assessment scales and other health indicators, and laboratory measurements in the eFI. It had a moderate correlation with a clinical frailty measure (i.e., CFS, ρ=0.420), but a weaker correlation with a comorbidity measure (i.e., CCI, ρ=0.368) and a frailty measure based on ICD-10 codes (i.e., HFRS, ρ=0.289). These findings suggest that the eFI captures frailty rather than multimorbidity. This is also on par with previous studies showing a moderate correlation for the eFI with the CFS (32), but a low correlation with the HFRS (33).

Although a data-driven or machine learning approach can be used in claims-based frailty indices (31), we applied the conventional deficit accumulation model (8) -a All rights reserved. No reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in The copyright holder for this this version posted November 28, 2021. ; https://doi.org/10.1101/2021.11.26.21266912 doi: medRxiv preprint generalizable approach to frailty, in which a wide range of deficits e.g., signs, symptoms, diseases, functional limitations, laboratory measures, can be included as long as it includes at least 30 age-related deficits (8, 34) . We noticed, however, some differences in the characteristics of our eFI compared to a survey-based Rockwood FI. First, instead of a rightskewed distribution characteristic to the Rockwood FI, our eFI was approximately normally distributed, possibly due to a homogenous sample of less healthy hospitalized older adults (35, 36) , and inclusion of laboratory tests (representing subclinical and cellular deficits) (37, 38) . Second, our eFI had a relatively low maximum value of 0.432. One possible reason is that for assessing the diagnosis-based ICD-code items (29 out of 48), we used only those codes that were recorded during the current visit, possibly leading to underreporting of certain diseases. Third, we observed slightly, but significantly higher eFI scores in men than in women. Although FI scores are generally higher in women in community populations (39), no sex difference (36, 40) or higher scores in men (18) have been found in hospitalized patients. The sex differences may also be influenced by whether self-reported (41) and laboratory deficits (37) are included. Indeed, more studies are needed to understand the mechanisms regarding sex differences in frailty (39).

Our finding of a robust predictive ability of the eFI for mortality and the length of stay is consistent with previous studies (18) (19) (20) (21) . Importantly, our eFI performed better than the CFS in predicting adverse outcomes, hence providing important insights on its potential use in risk stratification in geriatric patients to ease personnel burden in hospital settings.

However, all the frailty and comorbidity measures had a poor predictive accuracy for 30-day readmission. This may partly be explained by the potential misclassification of readmission, since we did not have data on patients who were admitted to other clinics than the nine geriatric clinics. Besides, a relatively low AUC or c-statistic for frailty in predicting readmission (around 0.5-0.6) is also frequently reported in the literature (19, 22, 23, 33, 42, 43) .

Hospital readmission is often described as a complex outcome not merely related to the health status of patients, but also social factors such as access to care, social support, and drug abuse (44, 45) , which are factors not captured in the frailty measures. We may All rights reserved. No reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in The copyright holder for this this version posted November 28, 2021. ; https://doi.org/10.1101/2021.11.26.21266912 doi: medRxiv preprint speculate that many of the more frail individuals resided in elderly care homes where care facilities are better, have a less pronounced tendency for readmission to hospital, compared to less frail individuals living in their own homes.

This study has a relatively large sample size from multiple hospitals, allowing us to assess the predictive performance of the eFI across the nine clinics. We also had data on other validated frailty and comorbidity measures for validation and comparative assessment of the eFI. Nevertheless, some limitations should be considered. As mentioned above, the use of ICD-10 codes from a single admission may have caused an underreporting of the diseases, and the same applies to the HFRS and CCI. As we only analyzed patients in geriatric clinics, our results cannot be applied to older patients in other units. It would be of interest to investigate whether the eFI performs equally well in other older in-patient groups.

Finally, the eFI could not be calculated for 27.6% of our sample due to missing data on functioning and laboratory measures. Our rate of missing data is nevertheless comparable to the ~30% missing data in the US eFI by Pajewski et al. (17) . Instead of the patients' demographic characteristics and health status being associated with the rate of missing data in the US eFI, our missing data were more related to variations in the data collection practice between hospitals. The COVID-19 pandemic might have affected the data collection routines for the functional and other health assessments, leading to more missing data than usual.

Future work may reveal which eFI items are the most decisive for predicting adverse outcomes, as well as which alternative available items from the EHRs could be included to complement the current eFI. Furthermore, it may be relevant to use other outcome measures than mortality in coming research efforts, although measures related to physical function and cognition are inherent in the FI. Crucial is also to show whether the eFI is able to identify patients that will benefit from CGA (26) , and that will respond to treatment tailored from the outcome of the CGA. When implementing these results, it is also essential to include patients' perspectives and to understand their feelings on frailty (46, 47) .

To conclude, we developed an eFI based on routinely collected EHRs for Swedish geriatric patients, and showed that it correlates with the CFS and has a good predictive All rights reserved. No reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in The copyright holder for this this version posted November 28, 2021. ; https://doi.org/10.1101/2021.11.26.21266912 doi: medRxiv preprint accuracy for short-and long-term mortality and the length of stay. This work provides evidence that an eFI is feasible for risk stratification among geriatric patients in Stockholm, Sweden, and may as well be applicable to countries with similar health system.

We would like to thank all patients, caregivers and staff who contributed to this study. This perpetuity.

preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in

The copyright holder for this this version posted November 28, 2021. ; https://doi.org/10.1101/2021.11.26.21266912 doi: medRxiv preprint perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in The copyright holder for this this version posted November 28, 2021. ; https://doi.org/10.1101/2021.11.26.21266912 doi: medRxiv preprint All rights reserved. No reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in The copyright holder for this this version posted November 28, 2021. ; https://doi.org/10.1101/2021.11.26.21266912 doi: medRxiv preprint

Frailty: review and meta-analysis

Frailty and risk of adverse outcomes in hospitalized older adults: a comparison of different frailty measures

The relationship of frailty and hospitalization among older people: evidence from a meta-analysis

Frailty and healthcare costs-longitudinal results of a prospective cohort study

Use of health resources and healthcare costs associated with frailty: The FRADEA study

Frailty in older adults: evidence for a phenotype

A standard procedure for creating a frailty index

Frailty measurement in research and clinical practice: A review

Agreement and predictive value of the Rockwood Clinical Frailty Scale at emergency department triage

Interrater reliability of the Clinical Frailty Scale by geriatrician and intensivist in patients admitted to the intensive care unit

Reliability of frailty assessment in the critically ill: a multicentre prospective observational study

Development and validation of an electronic frailty index using routine primary care electronic health record data

Measuring frailty in Medicare data: development and validation of a claims-based frailty index

Frailty screening using the electronic health record within a Medicare accountable care organization

Derivation and validation of a generalizable preoperative frailty index using population-based health administrative data

Automated frailty screening at-scale for -15

Development of a frailty index from routine hospital data in perioperative and critical care

Development and validation of a Hospital Frailty Risk Score focusing on older people in acute care settings using electronic hospital records: an observational study

External validation of the Hospital Frailty Risk Score in France

Does identifying frailty from ICD-10 coded data on hospital admission improve prediction of adverse outcomes in older surgical patients? A population-based study

Hospital frailty risk score and adverse health outcomes: evidence from longitudinal record linkage cardiac data

Costs and effects of comprehensive geriatric assessment in primary care for older adults with high risk for hospitalisation

A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation

Adaptation of the Charlson comorbidity index for register-based research in Sweden

Measuring frailty in health care databases for clinical care and research

Convergent validity of the electronic frailty index

A comparison of two national frailty scoring systems

The degree of frailty as a translational measure of health in aging

Predictors of adverse outcomes on an acute geriatric rehabilitation ward

The male-female health-survival paradox in hospitalised older adults

Standard laboratory tests to identify older adults at increased risk of death

A frailty index based on mortality and readmission in older adults in acute care wards: a prospective study

Measuring frailty using self-report and test-based health measures

Frailty status at admission to hospital predicts multiple adverse outcomes

The predictive properties of frailty-rating scales in the acute medical unit

Risk prediction models for hospital readmission: a systematic review

Hospital readmission and social risk factors identified from physician notes

Frailty: mind the gap

How community-dwelling Mild frailty

None.

Continuous (per point increase) 1