key: cord-0879071-dfpozgad authors: Garry, Elizabeth M.; Weckstein, Andrew R.; Quinto, Kenneth; Bradley, Marie C.; Lasky, Tamar; Chakravarty, Aloka; Leonard, Sandy; Vititoe, Sarah E.; Easthausen, Imaani J.; Rassen, Jeremy A.; Gatto, Nicolle M. title: Categorization of COVID‐19 severity to determine mortality risk date: 2022-05-09 journal: Pharmacoepidemiol Drug Saf DOI: 10.1002/pds.5436 sha: 318a76afe9850cc1ecf02397f2a45ee950cb351e doc_id: 879071 cord_uid: dfpozgad PURPOSE: Algorithms for classification of inpatient COVID‐19 severity are necessary for confounding control in studies using real‐world data. METHODS: Using Healthverity chargemaster and claims data, we selected patients hospitalized with COVID‐19 between April 2020 and February 2021, and classified them by severity at admission using an algorithm we developed based on respiratory support requirements (supplemental oxygen or non‐invasive ventilation, O2/NIV, invasive mechanical ventilation, IMV, or NEITHER). To evaluate the utility of the algorithm, patients were followed from admission until death, discharge, or a 28‐day maximum to report mortality risks and rates overall and by stratified by severity. Trends for heterogeneity in mortality risk and rate across severity classifications were evaluated using Cochran‐Armitage and Logrank trend tests, respectively. RESULTS: Among 118 117 patients, the algorithm categorized patients in increasing severity as NEITHER (36.7%), O2/NIV (54.3%), and IMV (9.0%). Associated mortality risk (and 95% CI) was 11.8% (11.6–12.0%) overall and increased with severity [3.4% (3.2–3.5%), 11.5% (11.3–11.8%), 47.3% (46.3–48.2%); p < 0.001]. Mortality rate per 1000 person‐days (and 95% CI) was 15.1 (14.9–15.4) overall and increased with severity [5.7 (5.4–6.0), 14.5 (14.2–14.9), 32.7 (31.8–33.6); p < 0.001]. CONCLUSION: As expected, we observed a positive association between the algorithm‐defined severity on admission and 28‐day mortality risk and rate. Although performance remains to be validated, this provides some assurance that this algorithm may be used for confounding control or stratification in treatment effect studies. • The positive association between the algorithm-defined severity and mortality provides some assurance that this algorithm may be used for confounding control or stratification in treatment effect studies. Algorithms for classification of inpatient COVID-19 severity are necessary to conduct studies using real-world data. We developed an algorithm to classify disease severity in hospitalized COVID-19 patients based on respiratory support requirements (supplemental oxygen or noninvasive ventilation, O2/NIV, invasive mechanical ventilation, IMV, or NEITHER). Using a cohort study, we evaluated the utility of the algorithm by determining if patients classified as having greater disease severity at admission are at higher risk for inpatient mortality. Among the 118,117 patients hospitalized with COVID-19 from HealthVerity claims and chargemaster data between April 2020 and February 2021, the algorithm categorized patients in increasing severity as NEITHER (36.7%), O2/NIV (54.3%), and IMV (9.0%). Associated mortality risk was 11.8% overall and increased with severity (3.4% NEITHER, 11.5% O2/NIV, and 47.3% IMV). Mortality rate per 1000 person-days was 15.1 overall and increased with severity (5.7 NEITHER, 14.5 O2/NIV, and 32.7 IMV). This provides some assurance that this algorithm may be used for confounding control or stratification in real world data studies. In more severe cases when organs start to fail, additional support may be added to IMV to help the heart and lungs pump oxygen into the blood (extracorporeal membrane oxygenation [ECMO] ), help the kidneys with filtration (renal replacement therapy), or improve blood and oxygen delivery to vital organs (vasopressors). As indicators of COVID-19 severity, O2, NIV, and IMV respiratory support requirements at admission may be critical measures of risk, prognosis, and severe outcomes, such as death. The U.S. Food and Drug Administration (FDA) has also issued guidance recommending that patients be classified according to baseline disease severity in all clinical trials aimed to determine the effectiveness of new COVID-19 treatments and prevention. 2 Independent of treatment, we expected an increased COVID-19 severity level to be associated with an increased mortality risk. The World Health Organization (WHO) developed a Clinical Progression Scale to classify COVID-19 severity. 3 While this scale was developed for determining patient outcomes, it may potentially be used to determine severity at the time of admission. However, the scale relies heavily on the availability of clinical information that may not always be available in real-world data (RWD) --health care data routinely collected, such as claims and billing activities or electronic health records (EHR). 4 The FDA Sentinel Initiative also developed a practical severity score using RWD to classify patient severity ranging from asymptomatic to critical. 5 However, the categorization utilizes data from the entire hospitalization and relies on day-level diagnoses that are often unavailable or under-recorded within the coded variables of inpatient data sources. Therefore, we developed an inpatient RWD algorithm, called the mWHO score to classify COVID-19 severity (respiratory support requiring O2/NIV, IMV, or neither) that is a modified version of the WHO Scale and influenced by the FDA Sentinel score. This study (which is a result of a research collaboration agreement between Aetion and FDA to use RWD to advance the understanding of the natural history and treatment of COVID-19) estimates mortality risk and incidence rate in a cohort of patients hospitalized for COVID-19 stratified by mWHO subgroups at admission. The aim was to establish the utility of the mWHO algorithm for use in confounding control or subgroup characterization in treatment effectiveness studies using RWD by demonstrating that patients with greater algorithmdefined COVID-19 disease severity at admission are at higher risk for mortality. The WHO Clinical Progression Scale scores a range of COVID-19 severity outcomes from uninfected (score of 0) to dead (score of 10; see Table 1 ). 3 We developed a modified version, referred to here as the mWHO score, that restricts to severity levels applicable to hospitalized COVID-19 patients (WHO original scores of 4-9), collapsed into three mutually exclusive categories for neither O2/NIV nor IMV, O2 or NIV without IMV, and any IMV with or without additional support (NEITHER, O2/NIV, and IMV, respectively; see supplemental Appendix B for additional algorithm detail). These categories correspond to the FDA Sentinel's Moderate, Severe, and Critical categories, respectively, leaving out the Asymptomatic and Mild categories that apply only to non-hospitalized patients. 5 The algorithm includes procedure codes, diagnosis codes, and free-text search terms to query the chargemaster data indicating the occurrence of respiratory support procedures. Diagnoses indicating a clinical need for O2/NIV (hypoxia or hypoxemia) or IMV (acute respiratory distress syndrome) were also added to the algorithm to increase the sensitivity of reporting these procedures. We identified a study cohort of patients hospitalized with an admission date between 01-April-2020 and 25-February-2021, available chargemaster data (necessary for capture of inpatient medication use), confirmed COVID-19 (ICD-10 diagnosis of U07.1 or a positive NAAT SARS-CoV-2 laboratory result) on the admission date or within 21 days prior, and at least one medical encounter during the 183-day baseline period (see supplemental Appendix C for study diagram). We excluded patients with missing sex, age, or geographic region and patients with any record of a COVID-19 vaccine on or prior to admission. We then stratified the cohort into subgroups according to their greatest mWHO COVID-19 severity level at admission (NEITHER, O2/NIV, and IMV). The day prior to admission was included to minimize potential misclassification from situations in which patients received O2 or IMV in other medical settings (e.g., emergency room or ambulance), or cases where the billing date for a procedure was captured at a later calendar date than the procedure was performed due to a lag in reporting. The day after admission was included when categorizing COVID-19 severity at admission to account for patients who may be admitted late in the evening who do not have any record of respiratory support until the next morning. This study reports descriptive statistics for patient characteristics determined a priori, including demographic characteristics upon admission, and comorbidities (individual, the combined comorbidity score, 13 and the frailty index 14 ) , and medication use during the 183-day baseline period. Follow-up to ascertain 28-day mortality, defined as an inpatient chargemaster encounter with a discharge status of "expired", began on the admission date and continued until occurrence of the outcome, hospital discharge, or 28 days following admission. Mortality risk was calculated as the total number of qualifying mortality endpoints divided by the total number of patients in the cohort at any point in time over the entire study period and was reported per 100 patients with corresponding 95% confidence intervals (CI). Mortality incidence rate was calculated as the total number of qualifying mortality endpoints divided by the total follow-up of all patients in the cohort and was reported per 1000 person-days with corresponding 95% CIs. We also plotted cumulative incidence curves to confirm expected divergence among the mWHO subgroups. Descriptive statistics and the mortality risk and incidence rates are reported among the overall cohort and for each mWHO subgroup. We conducted three sensitivity analyses to evaluate the robustness of our findings. The first analysis evaluated mortality risk and incidence rate for inpatient severity at admission, based on the greatest level of the following groups that more closely resembled those of the WHO Progression Scale: NEITHER, O2, NIV, IMV without additional organ support (vasopressors, dialysis, or ECMO), or IMV with additional organ support (see additional detail in supplemental Appendix D, Table D .1). 3 The second analysis aimed to understand mortality trends over time by plotting the mortality rate each month of the study period. The third analysis aimed to confirm the utility of the mWHO algorithm during the early months of the COVID-19 pandemic, when the disease landscape was still rapidly evolving, by evaluating mortality risks and incidence rates among a subset of patients admitted between 01-April-2020 and 31-August-2020. Analyses were conducted using the Aetion Evidence Platform ® (2021), a software for RWD analysis, validated for a range of studies. 15 Trends for heterogeneity in mortality risk and incidence rate across subgroups were evaluated using Cochran-Armitage and log rank trend tests, respectively, via the DescTools 16 and survminer 17 packages in R (v4.0.3). The study was approved under exemption by the New England Institutional Review Board. subgroups was also observed via cumulative incidence plots that held over the entire follow-up ( Figure 2 ). In the first sensitivity analysis, using the five more granular severity subgroups that were like the WHO Progression Scale (supplemental Appendix D, In the second sensitivity analysis to examine mortality trends over time, we observed monthly mortality incidence rates that were generally stable over the study period, with a slight decline among patients Comprised dexamethasone, methylprednisolone, prednisone, and hydrocortisone. d Patients could have initiated a potential COVID-19 therapy prior to the index admission due to use for an indication other than COVID-19, transfer from another hospital outside of the chargemaster network, transfer from the emergency department, or an inpatient or outpatient visit prior to qualifying for index admission.This category comprised the following listed in the NIH guidelines (2021) 18 other than corticosteroids: hydroxychloroquine/ chloroquine, remdesivir, lopinavir/ritonavir and other HIV protease inhibitors, ivermectin, IL-6 inhibitors (sarilumab, tocilizumab, siltuximab), JAK inhibitors (baricitinib, ruxolitinib, tofacitinib, upadacitinib), BTK inhibitors (acalabrutinib, ibrutinib, zanubrutinib), interferons (alfa-2b or beta-1a), COVID-19 monoclonal antibody treatments (casirivimab/imdevimab, bamlanivimab, etesevimab), convalescent plasma, ivermectin, mesenchymal stem cells, fluvoxamine, and colchicine. in the NEITHER and O2/NIV subgroups and a slight incline among patients in the IMV subgroup (supplemental Appendix D, Figure D.1) . In the third sensitivity analysis, the increasing trend for mortality risk and incidence rate stratified by the algorithm-defined mWHO subgroups remained among patients hospitalized in the early months of the COVID-19 pandemic (supplemental Appendix D, Table D. 3). The mWHO algorithm was developed to categorize inpatient COVID-19 severity based on respiratory support requirements (O2/NIV, IMV, or NEITHER). To operationalize this concept, we included diagnoses that indicate a clinical requirement for O2/NIV or IMV in addition to procedure-based encounters. While the mWHO algorithm has not been validated against medical records, we note agreement with expectations among factors known to be plausibly associated with both COVID-19 severity and mortality, such as age, coronary artery disease, and chronic pulmonary disease within the three mWHO subgroups. Further agreement with expectations was observed via a positive association between the mWHO algorithm-defined severity level at admission and 28-day mortality risk and rate. A lack of overlap in 95% CIs of the risks or rates across subgroups with increasing severity and the heterogeneity tests further substantiate the increasing trend observed. This finding was expected given that the algorithm was designed to differentiate the degree of severity and therefore the associated risk of adverse outcomes, such as death. Sensitivity analyses also confirmed similar increasing trends in mortality risks and rates, providing further assurance that our algorithm was operating as anticipated. Strengths and limitations of the HealthVerity administrative data were taken into consideration. First, although the open claims data has the benefit of near-real-time capture, it may be less complete for the most recent calendar dates. However, we truncated our study period to end 60 days prior to the last date of data available to minimize this concern (protocol available on clinicaltrials.gov, NCT04926571). 19 When determining the cohort selection criteria, we required at least one In addition, we explored weekly mortality events over time as they compared to two external national benchmarks, the Centers for Disease Control 22 and data sourced from State and local health agencies, 23 and the similar trends we observed minimized potential concern for misclassification of mortality data in our study. As with most RWD sources, data indicating cause of death or the occurrence of death outside of the hospital is not available in the data. Further, we were unable to assess the distribution of deaths with COVID-19 as the immediate cause of death and deaths occurring after discharge were considered outside the scope of the research question. The performance of the mWHO algorithm to identify each severity level on hospital admission remains to be validated against a "gold standard" such as manual review of medical records to quantify specificity and sensitivity. Further revisions to the algorithm may be considered moving forward. Although this study evaluates its utility to define COVID-19 severity at admission and in only one administrative data source, the algorithm has the potential for use more broadly in other RWD sources using the code lists provided, and may be expanded to utilize additional data available during the hospitalization (after admission) as appropriate (e.g., ascertaining study outcomes). This algorithm is an important addition to the COVID-19 and pulmonary RWD literature. The positive association observed between the algorithm-defined severity level at hospital admission and 28-day inpatient mortality risk and incidence rate provide some assurance that this algorithm may be useful for severity level confounding control or subgroup characterization in treatment effectiveness studies using RWD. Coronavirus Disease (COVID-19) Dashboard. World Health Organization COVID-19: Developing Drugs and Biological Products for Treatment or Prevention Guidance for Industry World Health Organization (WHO) Working Group on the Clinical Characterisation and Management of COVID-19 infection. A minimal common outcome measure set for COVID-19 clinical research Framework for FDA's Real-World Evidence Program COVID-19 natural history master protocol. Sentinel Initiative, version 3.0 Risk for newly diagnosed diabetes >30 days after SARS-CoV-2 infection among persons aged <18 years -United States Use of dialysis, tracheostomy, and extracorporeal membrane oxygenation among 240,392 patients hospitalized with COVID-19 in the United States Evaluation of COVID-19 vaccine breakthrough infections among immunocompromised patients fully vaccinated with BNT162b2 Comparative hostcoronavirus protein interaction networks reveal pan-viral disease mechanisms Association of SARS-CoV-2 seropositive antibody test with risk of future infection Diagnosis-wide analysis of COVID-19 complications: an exposurecrossover study COVID-19 evidence accelerator: a parallel analysis to describe the use of hydroxychloroquine with or without azithromycin among hospitalized COVID-19 patients A combined comorbidity score predicted mortality in elderly patients better than existing scores Measuring frailty in Medicare data: development and validation of a claims-based frailty index Transparency and reproducibility of observational cohort studies using large healthcare databases DescTools: Tools for Descriptive Statistics National Institute of Health (NIH). Coronavirus Disease 2019 (COVID-19) Treatment Guidelines Effect of dexamethasone on inpatient mortality among hospitalized COVID-19 patients A COVID-19-ready public health surveillance system: the Food and Drug Administration's sentinel system Risk factors associated with in-hospital mortality in a US National Sample of patients with COVID-19 In-hospital mortality among hospital confirmed COVID-19 encounters by week from selected hospitals Death Data Sourced from State and Local Health Agencies The authors thank the following additional members of the COVID- Additional supporting information may be found in the online version of the article at the publisher's website.