key: cord-0736345-busxgjfo
authors: Webb, B. J.; Levin, N. M.; Grisel, N.; Brown, S. M.; Peltan, I. D.; Spivak, E. S.; Shah, M.; Stenehjem, E.; Bledsoe, J.
title: Simple Scoring Tool to Estimate Risk of Hospitalization and Mortality in Ambulatory and Emergency Department Patients with COVID-19
date: 2021-02-23
journal: nan
DOI: 10.1101/2021.02.22.21252171
sha: d76acfb881835d2936053af85e9b3efefcae433e
doc_id: 736345
cord_uid: busxgjfo

Background: Accurate methods of identifying patients with COVID-19 who are at high risk of poor outcomes has become especially important with the advent of limited-availability therapies such as monoclonal antibodies. Here we describe development and validation of a simple but accurate scoring tool to classify risk of hospitalization and mortality. Methods: All consecutive patients testing positive for SARS-CoV-2 from March 25-October 1, 2020 within the Intermountain Healthcare system were included. The cohort was randomly divided into 70% derivation and 30% validation cohorts. A multivariable logistic regression model was fitted for 14-day hospitalization. The optimal model was then adapted to a simple, probabilistic score and applied to the validation cohort and evaluated for prediction of hospitalization and 28-day mortality. Results: 22,816 patients were included; mean age was 40 years, 50.1% were female and 44% identified as non-white race or Hispanic/Latinx ethnicity. 6.2% required hospitalization and 0.4% died. Criteria in the simple model included: age (0.5 points per decade); high-risk comorbidities (2 points each): diabetes mellitus, severe immunocompromised status and obesity (body mass index[≥]30); non-white race/Hispanic or Latinx ethnicity (2 points), and 1 point each for: male sex, dyspnea, hypertension, coronary artery disease, cardiac arrythmia, congestive heart failure, chronic kidney disease, chronic pulmonary disease, chronic liver disease, cerebrovascular disease, and chronic neurologic disease. In the derivation cohort (n=16,030) area under the receiver-operator characteristic curve (AUROC) was 0.82 (95% CI 0.81-0.84) for hospitalization and 0.91 (0.83-0.94) for 28-day mortality; in the validation cohort (n=6,786) AUROC for hospitalization was 0.8 (CI 0.78-0.82) and for mortality 0.8 (CI 0.69-0.9). Conclusion: A prediction score based on widely available patient attributes accurately risk stratifies patients with COVID-19 at the time of testing. Applications include patient selection for therapies targeted at preventing disease progression in non-hospitalized patients, including monoclonal antibodies. External validation in independent healthcare environments is needed.

A relatively conserved set of clinical and demographic factors are now recognized to correlate with an increased risk for severe disease requiring hospitalization, mechanical ventilation and death. (2) (3) (4) Accurate methods of risk stratifying ambulatory patients at the point of test positivity have many possible applications, including prioritizing patients at highest risk of hospitalization for early treatments aimed to prevent progression to severe disease, such as monoclonal antibodies, which are both limited in availability and also more likely to be effective in high-risk groups. Several risk classification models have been proposed. (3, (5) (6) (7) (8) (9) We describe development and validation of a simple scoring model to predict hospitalization and mortality in a large cohort of ED and ambulatory patients with COVID-19.

Intermountain Healthcare is an integrated healthcare system that provides care to more than 1.5 million patients each year in Utah and bordering communities. As part of a systemwide COVID-19 response, Intermountain provides SARS-CoV-2 testing at 32 urgent care facilities, 23 emergency departments, and 16 community drive-up testing sites. During the study period, only polymerase chain reaction (PCR) assays were performed (Thermofisher, Waltham, MA;

Cepheid, Sunnyvale, CA; Quidel, San Diego, CA, BioFire, Salt Lake City, UT; Roche, Basel, Switzerland). All testing required an order entered in the electronic health record (EHR) (Cerner, Kansas City, KS) by the ordering clinician through a structured form that requires the clinician to input the patient's clinical symptoms and epidemiological features. These data are stored in the Intermountain Prospective Observational COVID-19 (IPOC) database, and the enterprise data . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ;

warehouse. This analysis was approved by the Institutional Review Board at Intermountain Healthcare under #1051342.

We queried the IPOC database for consecutive adult patients with positive SARS-CoV-2 tests from March 25-October 1, 2020. Symptom data were extracted from the electronic test order form while demographic and co-morbidity data were obtained from the IPOC database and data warehouse using the Charlson and Elixhauser definitions (10,11). We defined immunosuppression as: recipient of a solid organ or hematopoietic stem cell transplant, on chemotherapy, biologic or other immunosuppressive agents targeting B or T cell activity, chronic corticosteroids at a prednisone-equivalent dose of 20mg per day or greater for more than 30 days, human immunodeficiency virus complicated by acquired immunodeficiency syndrome (AIDS), heritable immunodeficiency. We defined obesity as body mass index (BMI) of greater than or equal to 30 (12) . Symptom and demographic data were complete; comorbidity data were complete insofar as patients had prior encounters in the integrated health system. Mortality data was captured via an existing linkage to state death records.

We used a random number generator to divide the cohort into a 70% derivation cohort and 30% validation cohort. In the derivation cohort data, we fitted a multivariable logistic regression model for hospitalization within 14 days of testing, using clinical and demographic features. Predictors were prespecified before model development based on: clinical features that would be available at the time of testing for all ambulatory and emergency department patients regardless of testing venue (a criterion that precludes, for instance, laboratory data), biological plausibility of association with severity, and reproducibility in other studies in existing COVID-19 literature. We intentionally did not fit a model for mortality, but instead planned a priori to validate the ultimate model against that outcome. Model discrimination was evaluated using the area under the receiver-operator characteristic curve (AUROC) and model fit by evaluating R 2 using the Nagelkerke method (13, 14) .

We included patients who tested in the ambulatory setting as well as patients who tested positive in the emergency department to ensure that the score would be applicable in both environments. However, we recognized that some patients testing positive in the emergency department (ED) are then subsequently admitted. The decision to admit or not is not immediately known to emergency medicine providers who may still wish to use the score to stratify risk to aid in clinical decision making and selection of therapies. However, because patients who are admitted from the ED may have different characteristics than those tested in the ambulatory setting, we planned a priori to perform a sensitivity analysis by repeating the regression above after restricting the cohort to patients who were not admitted to the hospitalization at the time of their test.

We then adapted the original logistic regression model into a simple scoring tool by converting exponentiated β coefficients into weighted point assignments for each variable. We evaluated the test performance characteristics of this simplified clinical prediction tool in the derivation and validation cohorts using AUROC and by calculating the sensitivity, specificity, negative and positive predictive values across the range of scoring thresholds.

From March 25 through October 1, 2020, 22,816 patients had a positive PCR test for SARS-CoV-2. The mean age was 40 years (see Table 1 ); 11,424 (50.1%) patients were female and 8753 (43.9%) identified as a member of a community of color (either non-white race or Hispanic or Latinx ethnicity). Patients had on average one significant medical comorbidity. 1419 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10.1101/2021.02.22.21252171 doi: medRxiv preprint (6.2%) of patients were admitted; of these, 799 (3.6%) tested positive in the emergency department during the encounter that culminated in admission. Overall 93 patients (0.4%) died within 28 days of their positive SARS-CoV-2 assay. Demographic and clinical features were very similar between derivation (n=16,030) and validation (n=6786) cohorts.

In the derivation cohort, the primary multivariable model (see Table 3 Table 4 ) with the exception that the magnitude of risk of dyspnea was less in the ambulatory-only cohort (OR 2.1 vs 3.5), and the odds of immunosuppressed patients without palliative goals of care being admitted were greater (OR 7.0 vs 3.9).

Criteria included in the probabilistic, simplified clinical prediction score are displayed in Table 5 . Because cumulative comorbidity count was significantly associated with poor outcomes, we included comorbidities in the simplified tool that were not statistically significant individually in the expanded logistic regression model. In the derivation cohort, the AUROC for . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10.1101/2021.02.22.21252171 doi: medRxiv preprint the simplified clinical prediction score for 14-day hospitalization was 0.82 (95% CI: 0.81-0.84) and 0.8 (95% CI: 0.78-0.82) in the validation cohort. AUROC for 28-day all-cause mortality in the derivation cohort was 0.91 (95% CI: 0.83-0.94) and in the validation cohort 0.80 (95% CI: 0.69-0.9). The scoring threshold that optimized sensitivity and specificity (by Youden's index) was 6 with test characteristics of 71.1% and 76.2% respectively ( Table 6 )(15).

Given recent straining hospital volumes and the emergence of promising but limitedavailability outpatient therapies for COVID-19, methods are needed to identify patients with COVID-19 at highest risk of progression to severe disease, hospitalization and death. Here we describe a simple scoring model capable of accurately risk stratifying ambulatory and emergency department patients for COVID-19 for subsequent hospitalization and mortality.

One of the primary strengths of this model is the simplified and easily calculable score using features that are widely accessible. In particular, our score does not require laboratory studies, which are unavailable in the majority of ambulatory patients testing positive for SARS-CoV-2. While preserving discriminative value, this simple scoring system has potential to facilitate more widespread clinical application in settings lacking robust integration of informatics. The model was derived and validated in a very large and diverse population in the western United States and is based on risk factors for severe disease that are largely conserved across global populations, including age (12) , male sex, overall comorbid burden (13) , and shortness of breath at the time of risk stratification. These factors align closely with those included in models derived in other locations and populations (3, (5) (6) (7) (8) (9) 16, 17) . As a result, we expect that this tool will be generalizable.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10.1101/2021.02.22.21252171 doi: medRxiv preprint ultimately adopted because of the simplicity, widely accessible clinical features and validation in a large, representative local population (20). By regularly adjusting the eligibility criteria based on the risk score threshold that best calibrates current infusion capacity to the number of new cases in high-risk strata, this risk-targeted drug allocation strategy has provided an equitable and flexible means of drug delivery in the context of still-uncertain efficacy and limited resources.

Limitations of our study include the retrospective, observational design, and the possibility that comorbidity data may have been unavailable or out of date for some patients in the cohort who receive the majority of their medical care outside our integrated healthcare system. Although the large study population and inclusion of widely recognize features improves the likelihood of generalizability, this will need to be confirmed through external validation.

In this large retrospective cohort study, we identified simple risk factors that can easily be calculated at the bedside without laboratory values to risk stratify COVID-positive individuals for risk of hospitalization and death. Applications include guiding allocation of therapies that are limited in availability. External validation is needed to confirm generalizability in diverse and geographically independent population.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10.1101/2021.02.22.21252171 doi: medRxiv preprint Conflicts of Interest: IP reports salary support through a grant from the National Institutes of Health (U.S.A). SB reports salary support from the U.S. NIH, Centers for Disease Control and the Department of Defense; he also reports receiving support for chairing a data and safety monitoring board for a respiratory failure trial sponsored by Hamilton, effort paid to Intermountain for steering committee work for Faron Pharmaceuticals and Sedana Pharmaceuticals for ARDS work, support from Janssen for Influenza research, and royalties for books on religion and ethics from Oxford University Press/Brigham Young University. BW reports partial salary support from a U.S. Federal grant from AHRQ. ES receives partial salary support through grants from the Centers for Disease Control. is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10.1101/2021.02.22.21252171 doi: medRxiv preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10.1101/2021.02.22.21252171 doi: medRxiv preprint

A Novel Coronavirus from Patients with Pneumonia in China

Clinical Characteristics of 138 Hospitalized Patients With 2019 Novel Coronavirus-Infected Pneumonia in Wuhan, China

Physiological and socioeconomic characteristics predict COVID-19 mortality and resource utilization in Brazil

Infection fatality risk for SARS-CoV-2 in community dwelling population of Spain: nationwide seroepidemiological study

SARS2 simplified scores to estimate risk of hospitalization and death among patients with COVID-19. medRxiv

Development and validation of a model for individualized prediction of hospitalization risk in 4,536 patients with COVID-19

Derivation and Validation of Clinical Prediction Rules for COVID-19 Mortality in Ontario, Canada. Open Forum Infect Dis

Personalized predictive models for symptomatic COVID-19 patients using basic preconditions: Hospitalizations, mortality, and the need for an ICU or ventilator

A Novel Scoring System for Prediction of Disease Severity in COVID-19. Front Cell Infect Microbiol

Defining Adult Overweight and Obesity | Overweight & Obesity | CDC

A note on a general definition of the coefficient of determination

Limited-Dependent and Qualitative Variables in Econometrics

Index for rating diagnostic tests

Predictors of in-hospital COVID-19 mortality: A comprehensive systematic review and meta-analysis exploring differences by age, sex and health conditions

Predictive Accuracy of COVID-19 World Health Organization (WHO) Severity Classification and Comparison with a Bayesian-Method-Based Severity Score (EPI-SCORE). Pathogens

NQF: Risk Adjustment for Socioeconomic Status or Other Sociodemographic Factors

Centers for Disease Control and Prevention