key: cord-1011571-o3yantj0 authors: Dugdale, Caitlin M; Rubins, David M; Lee, Hang; McCluskey, Suzanne M; Ryan, Edward T; Kotton, Camille N; Hurtado, Rocio M; Ciaranello, Andrea L; Barshak, Miriam B; McEvoy, Dustin S; Nelson, Sandra B; Basgoz, Nesli; Lazarus, Jacob E; Ivers, Louise C; Reedy, Jennifer L; Hysell, Kristen M; Lemieux, Jacob E; Heller, Howard M; Dutta, Sayon; Albin, John S; Brown, Tyler S; Miller, Amy L; Calderwood, Stephen B; Walensky, Rochelle P; Zachary, Kimon C; Hooper, David C; Hyle, Emily P; Shenoy, Erica S title: COVID-19 Diagnostic Clinical Decision Support: a Pre-Post Implementation Study of CORAL (COvid Risk cALculator) date: 2021-02-10 journal: Clin Infect Dis DOI: 10.1093/cid/ciab111 sha: 51ebf3517594c6601fb587b2e174b7340ca09964 doc_id: 1011571 cord_uid: o3yantj0 BACKGROUND: Isolation of hospitalized persons under investigation (PUIs) for COVID-19 reduces nosocomial transmission risk. Efficient PUI evaluation is needed to preserve scarce healthcare resources. We describe the development, implementation, and outcomes of an inpatient diagnostic algorithm and clinical decision support system (CDSS) to evaluate PUIs. METHODS: We conducted a pre-post study of CORAL (COvid Risk cALculator), a CDSS that guides frontline clinicians through a risk-stratified COVID-19 diagnostic workup, removes transmission-based precautions when workup is complete and negative, and triages complex cases to Infectious Diseases (ID) physician review. Pre-CORAL, ID physicians reviewed all PUI records to guide workup and precautions. Post-CORAL, frontline clinicians evaluated PUIs directly using CORAL. We compared pre- and post-CORAL frequency of repeat SARS-CoV-2 nucleic acid amplification tests (NAATs), time from NAAT result to PUI status discontinuation, total duration of PUI status, and ID physician work-hours, using linear and logistic regression, adjusted for COVID-19 incidence. RESULTS: Fewer PUIs underwent repeat testing after an initial negative NAAT post-CORAL than pre-CORAL (54% vs. 67%; aOR 0.53, 95% CI: 0.44-0.63, p<0.01). CORAL significantly reduced average time to PUI status discontinuation (adjusted difference: -7.4 [SE 0.8] hours/patient; p<0.01), total duration of PUI status (adjusted difference: -19.5 [SE 1.9] hours/patient; p<0.01), and average ID physician work-hours (adjusted difference: -57.4 [SE 2.0] hours/day; p<0.01). No patients had a positive NAAT within 7 days after discontinuation of precautions via CORAL. CONCLUSIONS: CORAL is an efficient and effective CDSS to guide frontline clinicians through the diagnostic evaluation of PUIs and safe discontinuation of precautions. to prevent nosocomial transmission of SARS-CoV-2. Once COVID-19 infection has been excluded, prompt discontinuation of transmission-based precautions is critical to preserve hospital bed capacity, conserve personal protective equipment (PPE), and reduce adverse impacts associated with isolation [1] [2] [3] . SARS-CoV-2 nucleic acid amplification tests (NAATs) are the gold standard for COVID -19 diagnosis, but they have imperfect sensitivity dependent upon specimen source, collection technique, and symptom duration [4] . The Infectious Diseases Society of America recommends a repeat NAAT 24-48 hours after the first negative test among symptomatic patients when clinical suspicion for COVID-19 remains intermediate or high [4] . Several reports describe COVID-19 diagnosed after two negative nasopharyngeal (NP) NAATs [5, 6] . However, repeating a NAAT extends the duration of isolation and may delay treatment, and standardized frameworks to categorize patients by clinical suspicion have not been developed. Among hospitalized PUIs, the stakes for accurate diagnoses are high given the potential for nosocomial transmission [7] [8] [9] . Many medical centers have turned to Infectious Diseases (ID) physicians, infection preventionists, and hospital epidemiologists to guide COVID-19 evaluations and make case-by-case determinations regarding discontinuation of transmissionbased precautions [10] [11] [12] . These specialists are in limited supply [13] [14] [15] [16] [17] and may not be available for 24/7 evaluation [13] . Approaches to increase the efficiency and accessibility of COVID-19 diagnostic evaluation are needed. A c c e p t e d M a n u s c r i p t 6 Clinical decision support systems (CDSS) improve adherence to guideline-based care [18] [19] [20] . SARS-CoV-2 testing criteria, diagnostics, and literature on evaluation of PUIs have evolved rapidly; CDSS can standardize evaluations through adoption of best practices. We created the COvid Risk cALculator (CORAL) CDSS, using a modified Delphi method informed by >4,500 person-hours of ID physician-led, case-by-case evaluation of PUIs. We describe the development and implementation of CORAL and outcomes from its use in over 2,000 inpatient encounters. We conducted a pre-post intervention study of COVID- 19 To be eligible for CORAL, patients must be adult (19 years old) PUIs with >1 negative SARS-CoV-2 NAATs and have chest imaging performed within the past 3 days. Classification of CT chest findings are based on Radiologic Society of North America standardized categories, which were widely used at MGH during the study period [21] . There are 4 possible CORAL pathways depending upon the number of negative NAATs and available imaging: 1 NAAT + chest X-ray (CXR), 1 NAAT + chest CT, >2 NAATs + CXR, and >2 NAATs + CT chest (Supplementary Clinicians load CORAL into a SmartForm (i.e., advanced note template) and then select answers, which generates a standardized note for the chart (Supplementary Figure 4) . A c c e p t e d M a n u s c r i p t 9 Responses are scored once CORAL is completed; scores are not visible to the clinician. If the score is below a preset threshold, CORAL discontinues CoV-Risk (Supplementary Figure 5 ). If the score is above the preset threshold, CORAL displays orders for recommended diagnostics. If additional workup cannot be obtained and clinical suspicion for COVID-19 remains high, CORAL prompts ID physician review. We evaluated pre-CORAL outcomes among hospitalized patients with new CoV-Risk status from March 18 th to May 19 th . Allowing for a one week transition period for a hospital-wide CORAL education campaign, we evaluated post-CORAL outcomes from May 27 th to July 28 th . During both periods, we examined the proportion of PUIs with an initial negative NAAT who underwent a repeat NAAT and who required additional diagnostics beyond two negative NAATs. We evaluated the time from the first (if only one test was performed) or second negative NAAT result to CoV-Risk discontinuation, total duration of CoV-Risk during the hospitalization, and average ID physician person-hours dedicated to PUI evaluations. Post-CORAL, we described provider-documented patient characteristics from the first use of CORAL and CORAL recommendations. After CoV-Risk discontinuation, repeat SARS-CoV-2 NAAT was recommended for patients with new or worsening symptoms concerning for COVID-19. We performed chart review for patients We conducted unadjusted and adjusted analyses for each outcome measure, comparing pre-CORAL values to post-CORAL values. The pre-CORAL period included both the passive PUI evaluation and active PUI evaluation periods, which were analyzed together. We used Chisquare tests to compare unadjusted pre-and post-CORAL proportions of patients with repeat testing after an initial negative NAAT and additional workup after two negative NAATs. We used logistic regression to adjust these comparisons for daily COVID-19 incidence among hospitalized PUIs as a surrogate for pre-test probability. COVID-19 incidence was defined as the proportion of daily new COVID-19 diagnoses among all newly admitted PUIs. For continuous outcomes, we used t-tests to compare unadjusted mean time from NAAT resultreturn to CoV-Risk discontinuation and total duration of CoV-Risk among PUIs. Pre-and post-CORAL means for these continuous outcomes were compared using a generalized linear model adjusting for daily COVID-19 incidence. On average, the proportion of new COVID-19 diagnoses among all newly admitted PUIs was We report the epidemiology, symptoms, and imaging findings, and additional diagnostics recommended for post-CORAL PUIs (Table 1) ; these structured EHR data were not available pre-CORAL. A c c e p t e d M a n u s c r i p t 12 In the unadjusted analysis, repeat NAAT after one initial negative test was less frequent post-CORAL compared with pre-CORAL (54% vs. 67%, OR 0.59, 95% CI: 0.52-0.66, p<0.01; Table 2 ) and persisted after adjusting for COVID-19 incidence (aOR 0.53, 95% CI: 0.44-0.63, p<0.01). Additional diagnostics for COVID-19 beyond two negative NAATs were also less frequent post-CORAL than pre-CORAL (19% vs. 30%) in both unadjusted (OR 0.53, 95% CI: 0.45-0.63, p<0.01) and adjusted (aOR 0.42, 95% CI: 0.33-0.54, p<0.01) analyses. These declines in repeat testing and additional diagnostics beyond two NAATs were not evident during the transition from active to passive ID physician review (April 24 th through May 19 th ); repeat testing and additional diagnostics were actually more frequent during this period despite decreasing COVID-19 incidence, which then led to lower odds ratios following adjustment for COVID-19 incidence ( Figure 3A ). Among patients who had CoV-Risk discontinued while in hospital, the mean time from negative NAAT result-return to CoV-Risk discontinuation was 5. Table 2 ). In contrast, the mean time from negative NAAT result-return to CoV-Risk discontinuation and mean total duration of CoV-Risk increased during the pre-CORAL period when transitioning from active to passive ID physician review and then declined in the post-CORAL period ( Figure 3B ). In 360/2,189 (16%) pre-CORAL encounters, patients underwent repeat NAAT within 14 days of ERI discontinuation due to new symptoms or required testing for discharge settings; 4/360 (1.1%) were diagnosed with COVID-19, all within 7 days of ERI discontinuation. Among 2,232 encounters after CoV-Risk discontinuation with CORAL, 433 (19%) patients had cliniciandirected repeat testing within 14 days of ERI discontinuation. One patient (1/433, 0.2%) had a positive NP NAAT 9 days after initial negative NAAT and ERI discontinuation by CORAL, as well as a positive total SARS-CoV-2 antibody test 11 days after admission. Details of the five cases are provided (Supplementary Table 5 ); no onward nosocomial transmission from these individuals to other patients was identified. We describe the development and implementation of CORAL, a novel real-time diagnostic algorithm and CDSS to guide front-line clinicians through the evaluation of PUIs and assist with safe and efficient discontinuation of transmission-based precautions. As a structured diagnostic algorithm, CORAL decreased the use of repeat SARS-CoV-2 NAAT after a first negative NAAT and the use of additional COVID-19 diagnostics beyond two negative NAATs. CORAL reduced A c c e p t e d M a n u s c r i p t 14 the duration of transmission-based precautions for PUIs compared to individualized ID physician review, supporting efforts to conserve PPE, preserve hospital capacity, and advance patient care. Prior to CORAL implementation, individualized inpatient PUI evaluations were very resourceintensive [10] . Alternatives to case-by-case ID physician review such as indefinite continuation of ERI precautions, or algorithmic discontinuation of PUI status after two NAATs, were neither feasible nor an efficient use of laboratory or other resources. Sustaining individualized ID physician PUI evaluations required a considerable proportion of the ID workforce, at a time when demand for ID consultation was also high [24] . CORAL solved this critical problem, because it guided diagnostic evaluation and discontinuation of the PUI status 24 hours/day. CORAL was quickly adopted by clinicians, resulting in a significant decrease in ID physician person-power required and serving important roles in laboratory stewardship, PPE conservation, and improved bed allocation. Overall COVID-19 incidence declined prior to CORAL implementation, as public health interventions were implemented state-wide [25] . Declining incidence may have contributed to decreased need for repeat NAAT and additional testing beyond two negative NAATs, as pretest probability of COVID-19 declined. However, the difference in these key outcomes pre-CORAL and post-CORAL was sustained after adjusting for COVID-19 incidence. Furthermore, in the late pre-CORAL period, we also observed paradoxical increases in these outcomes, as well as time from final result-return to CoV-Risk discontinuation and overall CoV-Risk duration, despite declining COVID-19 incidence. These increases corresponded to a shift from a system of active PUI to passive PUI evaluation, which makes it more likely that the decreased testing A c c e p t e d M a n u s c r i p t 15 utilization and CoV-Risk duration were directly attributable to the standardization of and 24/7 access to diagnostic guidance through the CORAL CDSS rather than declining COVID-19 incidence. Implementation of CDSS has benefits beyond direct clinical guidance. The CDSS captures structured data entered by clinicians, which can be used to refine the scoring algorithm. Second, as frontline clinicians must answer structured questions to complete CORAL, it also serves as an educational tool to teach clinicians about the diagnostic workup of COVID-19 [4] . Additionally, the platform is easily extendable and could be modified to follow local ID expert and stakeholder consensus, provide differential guidance based on population prevalence, or incorporate emerging laboratory testing modalities, such as SARS-CoV-2 serologies, which may also help assess patient transmissibility [26, 27] . Several other COVID-19 diagnostic risk scores based on combinations of patient epidemiology, laboratory values, imaging findings, symptoms, and time from symptom onset have been validated [28] [29] [30] [31] . For example, the Corona-score algorithm relies on age, sex, chest imaging, and laboratory values; it demonstrates 82-96% sensitivity to predict COVID-19 infection among patients presenting to the emergency department [30, 32] . While other scoring systems have demonstrated reasonable sensitivity for COVID-19 infection compared with a single NAAT, CORAL remains unique as the only algorithm, to our knowledge, that has been incorporated into routine, prospective clinician workflow via an EHR CDSS. CORAL guides clinicians through additional diagnostic recommendations after a negative test, which other published tools do not incorporate [28] [29] [30] [31] [32] . Finally, since CORAL is embedded into Epic™, it could be adopted by This study has several limitations. COVID-19 incidence declined before CORAL was implemented; however, differences in outcomes pre-and post-CORAL were sustained after adjusting for COVID-19 incidence to capture pre-test probability. Delays in test TAT may have contributed to longer duration of CoV-Risk in the early pre-CORAL period, but CoV-Risk duration was lower post-CORAL compared with the late pre-CORAL. Our evaluation of patients diagnosed with COVID-19 in the 14 days after CoV-Risk discontinuation is limited by cliniciandirected testing of patients, and may have been influenced by the lower COVID-19 incidence in the post-CORAL period. CORAL requires recent imaging and accurate history taking by frontline clinicians. Due to changes in PUI definitions over time, patients in the pre-CORAL period were more likely to be at higher risk for COVID-19, given that testing was initially focused on symptomatic individuals; however, testing of all admitted patients regardless of symptoms started mid-way through the pre-CORAL period. Finally, this analysis was conducted at a singlecenter with specific testing protocols and guidelines for discontinuation of transmission-based precautions. However, other facilities have similar protocols and guidelines [34, 35] , and CORAL has since been implemented in eight other affiliated facilities and used over 19,000 times. Impact of isolation on hospitalised patients who are infectious: systematic review with meta-analysis The effect of hospital isolation precautions on patient outcomes and cost of care: A multi-site, retrospective, propensity score-matched cohort study Adverse outcomes associated with Contact Precautions: a review of the literature Infectious Diseases Society of America Guidelines on the Diagnosis of COVID-19. Infectious Diseases Society of America Testing for SARS-CoV-2: Can we stop at two? Clinical, laboratory, and radiologic characteristics of patients with initial false-negative SARS-CoV-2 nucleic acid amplification test results Nosocomial outbreak of SARS-CoV-2 infection in a haematological unit -High mortality rate in infected patients with haematologic malignancies Nosocomial spread of COVID-19: Lessons learned from an audit on a stroke/neurology ward in a UK district general hospital Nosocomial transmission of COVID-19: a retrospective study of 66 hospital-acquired cases in a London teaching hospital Outcomes from an infectious disease physician-guided evaluation of hospitalized persons under investigation for coronavirus disease 2019 (COVID-19) at a large US academic medical center Utility of repeat testing for COVID-19: Laboratory stewardship when the stakes are high Utility of retesting for diagnosis of SARS-CoV-2/COVID-19 in hospitalized patients: Impact of the interval between tests Where Is the ID in COVID-19? Charting the future of Infectious Disease: Anticipating and addressing the supply and demand mismatch Infection prevention staffing and resources in U.S. acute care hospitals: Results from the APIC MegaSurvey Necessary infrastructure of infection prevention and healthcare epidemiology programs: A review A systematic approach to quantifying infection prevention staffing and coverage needs Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success Can the use of digital algorithms improve quality care? An example from Afghanistan Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review Radiological Society of North America expert consensus statement on reporting chest CT findings related to COVID-19 Lost in definitions: Reducing duplication and clarifying definitions of knowledge and decision support tools. A RANDmodified Delphi consensus study UCLA appropriateness method and a novel group awareness and consensus methodology for consensus measurement: a systematic literature review Automatic ID consultation for inpatients with COVID-19: point, counterpoint, and a single-center experience Massachusetts Department of Health. COVID-19 information for local boards of health The role of antibody testing for SARS-CoV-2: Is there one? Review of current advances in serologic testing for COVID-19 Derivation and validation of a scoring system to assess pre-test probability of being COVID-19 positive A novel risk score to predict diagnosis with coronavirus disease 2019 (COVID-19) in suspected patients: A retrospective, multicenter, and observational study Rapid identification of SARS-CoV-2-infected patients at the emergency department using routine testing A predictive model and scoring system combining clinical and CT characteristics for the diagnosis of COVID-19 Validation of the Corona-Score for rapid identification of SARS-CoV-2 infections in patients seeking emergency department care in the United States A novel cohorting and isolation strategy for suspected COVID-19 cases during a pandemic Clinical sensitivity of Severe Acute Respiratory Syndrome Coronavirus 2 nucleic acid amplification tests for diagnosing Coronavirus Disease A c c e p t e d M a n u s c r i p t 19 A c c e p t e d M a n u s c r i p t 21 A c c e p t e d M a n u s c r i p t 22 A c c e p t e d M a n u s c r i p t 31 Figure 3