key: cord-0925101-fpguf07n
authors: Ejima, K.; Maki, K.; Golzarri-Arroyo, L.; Allison, D. B.
title: Utility of COVID-19 Decision Rules Related to Consecutive Decline in Positivity or Hospitalizations: A Data-driven Simulation Study
date: 2020-12-16
journal: nan
DOI: 10.1101/2020.12.14.20248190
sha: 047a97a07cfe5b74690bec28728c3630842bfbdf
doc_id: 925101
cord_uid: fpguf07n

The White House issued Guidelines for Opening Up America Again to help state and local officials when reopening their economies. These included a 'downward trajectory of positive tests as a percent of total tests within a 14-day period.' To examine this rule, we computed the probability of observing continuous decline in positivity when true positivity is in decline using data-driven simulation. Data for COVID-19 positivity reported in New York state from April 14 to May 5, 2020, where a clear reduction was observed, were used. First, a logistic regression model was fitted to the data, considering the fitted values as true positivity. Second, we created observed positivity by randomly selecting 25,000 people per day from a population with those true positivity for 14 days. The simulation was repeated 1,000 times to compute the probability of observing a consecutive decline. As sensitivity analyses, we performed the simulation with different daily numbers of tests (10 to 30,000) and length of observation (7 and 21 days). We further used daily hospitalizations as another metric, using data from the state of Indiana. With 25,000 daily tests, the probability of a consecutive decline in positivity for 14 days was 99.9% (95% CI: 99.7% to 100%). The probability dropped with smaller numbers of tests and longer lengths of consecutive observation, because there is more chance of observing an increase in positivity with smaller numbers of tests and longer observation. The probability of consecutive decline in hospitalizations was ~0.0% regardless of the length of consecutive observation due to large variance. These results suggest that continuous declines in sample COVID-19 test positivity and hospitalizations may not be observed with sufficient probability, even when population probabilities truly decline. Criteria based on consecutive declines in metrics are unlikely to be useful for making decisions about relaxing COVID-19 mitigation efforts.

different daily numbers of tests (10 to 30,000) and length of observation (7 and 21 days). We CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted December 16, 2020. 35 As a reaction to the COVID-19 pandemic in the US, lockdown orders restricting people's 36 movement were issued by state governments to mitigate the transmission risk in March 2020. 37 After the lockdown orders continued over several months, federal and state governments started 38 seeking an exit strategy to reopen educational and economic activities suspended during the 39 lockdown orders. The White House issued a guideline for reopening of economies, "Opening up 40 America Again"(1), which included several numerical criteria. One of them was a "downward 41 trajectory of positive tests as a percent of total tests within a 14-day period." They also proposed 42 criteria based on consecutive decline in metrics related to COVID-19, such as COVID-like 43 syndromic cases. State guidelines were devised that were based on the White House 44 recommendations. For example, the state of Indiana proposed that a decision to reopen will be 45 made when "the number of hospitalized COVID-19 patients statewide has decreased for 14 46 days."(2) and in Wisconsin, "the three phases by the state would begin when there are 14 47 consecutive days of decreasing COVID-19 cases."(3) However, these criteria were proposed 48 without solid scientific evidence. 49 Beyond the above criteria, another metric used in policymaking has been the effective 50 reproduction number, i.e., the average number of secondary cases produced by a single primary 51 case (4). The effective reproduction number is used in assessing whether the epidemic is under 52 control and supported policy-making for COVID-19 and other infectious diseases (5). There are 53 quite a number of approaches in estimating the effective reproduction number proposed with 54 different assumptions (4, 6, 7) and different types of data (8, 9) . If the effective reproduction 55 number is below 1, this suggests the epidemic is shrinking. However, the number itself cannot tell 56 us why the number is below 1. For example, the effective reproduction number could be reduced 57 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted December 16, 2020. ; https://doi.org/10.1101/2020.12.14.20248190 doi: medRxiv preprint 4 because the disease spread substantially and the remaining susceptible population is insufficient 58 to maintain transmission, or because herd immunity has been achieved by mass vaccination. If so, 59 the risk of resurgence of epidemic is minimal at least for a while until (if) people lose immunity 60 (obtained through either infection or vaccination) or susceptible children accumulate, which has 61 been observed for measles outbreaks (10). However, there is another possibility that individual 62 and societal protective behaviors, including lockdown orders, temporally reduced the effective 63 reproduction number. For example, the mitigated mobility was associated with the growth of the is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted December 16, 2020. ; https://doi.org/10.1101/2020.12.14.20248190 doi: medRxiv preprint Herein, we examine this hypothesis by the use of simple simulations that preserve the properties 82 of the empirically observed data (i.e., data-driven simulation). To provide some context to inform 83 the potential utility of rules involving consecutive runs of a decreasing outcome metric, we is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted December 16, 2020. ; https://doi.org/10.1101/2020.12.14.20248190 doi: medRxiv preprint 6 2. Hospitalization data from Indiana state 104 The data for daily number of new hospitalizations (both ICU and non-ICU) in Indiana has been 105 summarized by the Regenstrief Institute (13). We extracted the daily data from April 7 to May 9, 106 2020. The data are depicted in Figure 1D . The number of hospitalizations fluctuate and dropped 107 during weekends. We fitted a linear model to the data. We denoted the observed number of 108 hospitalizations on day t as H r (t), and the corresponding conditional expected value estimated by 109 the regression model on day t as H m (t) (Figure 1D) , both of which are used in the simulation. 

As sensitivity analyses, we performed the same simulation with different length of observation (7 124 and 21 days) and different number of tests (10, 100, 1,000, 10,000, 50,000 tests per day).

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this this version posted December 16, 2020. ; https://doi.org/10.1101/2020.12.14.20248190 doi: medRxiv preprint 7 Similarly, we computed the probability of a consecutive decline in the daily number of is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this this version posted December 16, 2020. ; https://doi.org/10.1101/2020.12.14.20248190 doi: medRxiv preprint (Table 1) , observing consecutive decline in positivity for 14 days with 5,000 daily tests is 168 less than 60%, suggesting the criterion may not be able to capture the actual decline in positivity 169 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted December 16, 2020. ; https://doi.org/10.1101/2020.12.14.20248190 doi: medRxiv preprint 9 with this number of tests. Further, in small or rural counties or locales, the number of testing 170 should be much smaller, and the probability of consecutive declines becomes very low, despite 171 the presence of a true downward trend. The same probability (i.e., consecutive decline) using the 172 daily hospitalization data from Indiana state was extremely low (zero for practical purposes) 173 regardless of the length of observation. This is due to the large variance in hospitalizations. 174 Overall, our computations suggest that criteria for decision-making based on lengthy consecutive 175 runs of monotonic changes are unlikely to be useful, or at least need to be carefully examined on a 176 case-by-case basis considering realistic settings and factors that influence the magnitude of 177 observational error, such as number of tests. 178 The novelty and strength of this study is that we used real data from New York and Indiana states 179 from the current pandemic to realistically assess the criteria used for decision-making. Because 180 the results are dependent on the data and the criteria used, we suggest using the same framework 181 for each case needing to be assessed. That said, the criterion (consecutive decline) did not seem 182 realistically achievable based on our study, partially because positivity and hospitalizations did 183 not decrease fast enough and the variance of observational error of hospitalization was large. The 184 criterion might work for other metrics or other diseases. 185 A limitation of this study is that we assumed logistic or linear models for the outcome metrics. is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted December 16, 2020. ; https://doi.org/10.1101/2020.12.14.20248190 doi: medRxiv preprint 10 Compared with the criterion based on consecutive decline metrics, the effective reproduction 194 number is more robust against such measurement error because its computation is dependent on 195 the longitudinal case reports weighted by an infectivity function, rather than the number of cases 196 reported on a single time point. Further, the fluctuation of the effective reproduction number has 197 been observed and acknowledged (7, 18). Therefore, researchers are advised to assume the 198 effective reproduction number is constant for a short period to simplify interpretation. There is a 199 long history for conceptualization and computation of the effective reproduction number, whereas 200 consecutive decline has not been examined until our study, as far as we know. 216 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted December 16, 2020. ; https://doi.org/10.1101/2020.12.14.20248190 doi: medRxiv preprint 

The state of Indiana. Our Principles To Get Back On Track

Estimating in real time the efficacy of 231 measures to control emerging communicable diseases

Key questions for modelling COVID-19 exit strategies

Different Epidemic Curves for Severe Acute Respiratory Syndrome 235

Reveal Similar Impacts of Control Measures

A New Framework and Software to 238

Estimate Time-Varying Reproduction Numbers During Epidemics

The Effective Reproduction Number as a Prelude to Statistical 241

Estimation of Time-Dependent Epidemic Trends

Mathematical and Statistical Estimation Approaches in Epidemiology

Birth-death skyline plot reveals 245 temporal changes of epidemic spread in HIV and hepatitis C virus (HCV)

Travelling waves and spatial hierarchies in measles 247 epidemics

Association between 249 mobility patterns and COVID-19 transmission in the USA: a mathematical modelling study

NYSDOH COVID-19 Tracker [Available from: 13. Regenstrief Institute. Regenstrief COVID-19 Dashboard

Early dynamics 258 of transmission and control of COVID-19: a mathematical modelling study. The Lancet Infectious 259 Diseases

The effect of 261 control strategies to reduce social mixing on outcomes of the COVID-19 epidemic in Wuhan, 262 China: a modelling study

A mathematical model reveals the influence of population 264 heterogeneity on herd immunity to SARS-CoV-2

Transmission in Households During the 1918 Pandemic

It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprintThe copyright holder for this this version posted December 16, 2020. ; https://doi.org/10.1101/2020.12.14. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this this version posted December 16, 2020. ; https://doi.org/10.1101/2020.12.14.20248190 doi: medRxiv preprint 14 Tables 286 Table 1 CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprintThe copyright holder for this this version posted December 16, 2020. ; https://doi.org/10.1101/2020.12.14.20248190 doi: medRxiv preprint