key: cord-1039568-ddy1l4h8
authors: Ge, Q.; Hu, Z.; Li, S.; Lin, W.; Jin, L.; Xiong, M.
title: A Noel Intervention Recurrent autoencoder for real time forecasting and non-pharmaceutical intervention selection to curb the spread of Covid-19 in the world
date: 2020-05-08
journal: nan
DOI: 10.1101/2020.05.05.20091827
sha: d26ea8a5df844c6332046abb856b6cf47b4ac5ed
doc_id: 1039568
cord_uid: ddy1l4h8

ABSTRACT Objective: Develop the AI and casual inference-inspired methods for forecasting and evaluating the effects of public health interventions on curbing the spread of Covid-19. Methods: We developed recurrent neural network (RNN) for modeling the transmission dynamics of the epidemics and Counterfactual-RNN (CRNN) for evaluating and exploring public health intervention strategies to slow down the spread of Covid-19 worldwide. We applied the developed methods to real-time forecasting the confirmed cases of Covid-19 across the world. The data were collected from January 22 to April 18, 2020 by John Hopkins Coronavirus Resource Center (https://coronavirus.jhu.edu/MAP.HTML). Results: The average errors of 1-step to 10-step forecasting were 2.9%. In the absence of a COVID-19 vaccine, we evaluated the potential effects of a number of public health measures. We found that the estimated peak number of new cases and cumulative cases, and the maximum number of cumulative cases worldwide with one week later additional intervention were reduced to 103,872, 2,104,800, and 2,271,648, respectively. The estimated total peak number of new cases and cumulative cases would be the same as the above and the maximum number of cumulative cases would be 3,864,872 in the world with 3 week later additional intervention. Duration time of the Covid-19 spread would be increased from 91 days to 123 days. Our estimation results showed that we were in the eve of stopping the spread of COVID-19 worldwide. However, we observed that transmission would quickly rebound if interventions were relaxed. Conclusions: The accuracy of the AI-based methods for forecasting the trajectory of Covid-19 was high. The AI and causal inference-inspired methods are a powerful tool for helping public health planning and policymaking. We concluded that the spread of COVID-19 would be stopped very soon.

As of April 28, 2020, global confirmed cases of Covid-19 passed 3,116,398 and has spread to 210 countries, causing fear globally (Anastassopoulou et al. 2020) . The serious public health threat of Covid-19 has never been seen for more than one century. The government officers and people around the world are desperately trying to slow the spread of Covid-19 (Irfan, 2020) . We must change our policies to deal with increased mobility of citizens and immediately implement the public health interventions to stop the spread of Covid-19 across the world. How computer modeling of Covid-19's transmission dynamics could help governments to quickly and strongly move slow down the spread of Covid-19?

Widely used statistical and computer methods for modeling of Covid-19 simulate the transmission dynamics of epidemics to understand their underlying mechanisms, forecast the trajectory of epidemics, and assess the potential impact of a number of public health measures on curbing the spread speed of Covid-19 (Li et al., 2020 , Zhao et al., 2020 , Kucharski et al., 2020 , Tuite et al., 2020 , Hellewell et al., 2020 , Li et al. 2020 . Although these epidemiological models are useful for estimating the dynamics of transmission, and evaluating the impact of intervention strategies, they have some serious limitations (Funk et al., 2018 , Johansson et al., 2019 . First, the epidemiological models consist of ordinary differential equations that have many unknown parameters, and depend on many assumptions. It is difficult to translate public interventions to these parameters. Most analyses used hypothesized parameters, which often lead to fitting data very poor. Health officers desperately want to track the trajectory of epidemics and accurately estimate the peak time and umber of cases, duration, and ending time and number of cases of Covid-19 for their health policy plan. However, the forecasting results of using the classical epidemiological models such as Susceptible-Exposed-Infectious-Removed (SEIR) models are highly unreliable. Second, the successful application of public health intervention planning highly depends on the model parameter identifiability.

However, overall, the parameters in the complex compartmental dynamic models are unidentifiable (Roosa and Chowell, 2019, Roda et al., 2020) . The values of parameters cannot be uniquely determined from the real data (Gábor et al., 2017) . The variances of the estimators of these parameters are very high.

To overcome limitations of the epidemiological model approach, and assist public health planning and policy making, we formulated the real-time forecasting and evaluating multiple public health intervention problem into forecasting treatment response problem. We viewed the interventions as treatments where multiple interventions were administered at different time points. The number of new cases were taken as treatment responses. The ability to accurately estimate effects of public health interventions over time would allow health officers to determine what intervention strategies should be used and the optimal time at which to implement them (Lim et al., 2018) . Recurrent Intervention Network (RIN) (Lim et al., 2018) where a recurrent neural network architecture for forecasting a nation's response (number of new cases) to a series of planned interventions were used to forecast and evaluate multiple public health interventions for Covid-19 worldwide. Potential outcomes of RIN were trajectory of the spread of Covid-19. university dormitories. To quantify comprehensive intervention strategies, an intervention variable was used as an input variable for each block of RIN. We cluster all the countries in the world into several groups. For each group, a value (weight) was assigned to each group such that the average prediction error of CRN was minimum. The RIN is taken as a general framework for investigating how Covid-19 evolves under different intervention plans, how individual nation responds to intervention over time, but also which are optimal timings for assigning interventions. Therefore, this approach will provide new tools to improve public health planning and policy making.

The RIN was applied to the surveillance data of confirmed and new Covid-19 cases in the world up to April 7, 2020. Data on the number of confirmed, new and death cases of Covid-19 from January 22, 2020 to April 7, 2020 were obtained from John Hopkins Coronavirus Resource Center (https://coronavirus.jhu.edu/MAP.HTML).

The RIN uses sequence-to-sequence multi-input/output recurrent neural network (RNN) architectures to model health intervention plan and make multi-step prediction of the response trajectory of Covid-19 over time with multiple interventions (Lim et al., 2018) . The RNN can learn the complex dynamics within the temporal ordering of input time series of Covid-19 and use an internal memory to remember. The health intervention plan has multiple intervention regimens. As shown in Figure 1 , the RIN determines the intervention response (similar to counterfactual outputs) for a given set of planned interventions and evaluates the impact of . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 8, 2020 . . https://doi.org/10.1101 /2020 different intervention strategies and their implementation times on the curbing the spread of Covid-19 and provides timely selection of optimal sequence of intervention strategies.

The RIN is a RNN autoencoder. It consists of two RNNs: the encoder RNN (standard RNN or long short-term memory (LSTM) is used as encoder) and the decoder RNN (standard RNN or LSTM is used as the decoder). The RNN encoder models input time series ( The state in the hidden layer at the time was denoted by ℎ . The output layer had the output . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . variable +1 . A nonlinear activation function was exponential linear unit (ELU) (Clevert et al., 2015) which was defined as

where > 0.

The input data were divided into several batches with length of 7 days. Each batch was used to train the RNN encoder which forecasted standard one-step-ahead intervention response ̂+ 1 as close to the observed intervention response +1 as possible via the nonlinear mapping (Supplementary A)

The mean-squared error was used as loss function for training the RNN encoder. consisted of {ℎ , ( +1 , +2 , … , + −1 , + )}. The mean square errors were still used as the . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. .

The trained RIN was used to forecast the future number of cumulative cases of Covid-19 for each country. The recursive multiple-step forecasting involved using a one-step model multiple times where the prediction for the preceding time step and intervention strategy were used as an input for making a prediction on the following time step. For example, for forecasting the number of new confirmed cases for the one more next day, the predicted number of new cases in one-step forecasting would be used as an observational input in order to predict day 2. Repeat the above process to obtain the two-step forecasting. The summation of the final forecasted number of new confirmed cases for each country was taken as the prediction of the total number of new confirmed cases of Covid-19 worldwide.

The analysis is based on surveillance data of confirmed cumulative and new COVID-19 cases worldwide as of April 7, 2020. Data on the number of cumulative and new cases and COVID-19attributed deaths across 184 countries from January 22, 2020 to April 7 were obtained from John Hopkins Coronavirus Resource Center (https://coronavirus.jhu.edu/MAP.HTML).

log 2 was used to transform the original data: ̃= log 2 ( + 1). A value of 1 was assigned to the intervention variable for China. A value of zero was assigned to the intervention variable if the country did not implement intervention. Evenly spaced numbers over the interval [0, 1] were assigned to the intervention variable for other countries. The values of intervention variable were . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . determined such that they minimized the prediction error via training process. The data were divided into batches. To train the RIN, the countries and starting day in each selected country were randomly selected. The length of each batch was 7. The values of each time series in the batch were divided by their mean values to normalize the data.

Accurate prediction of the spread of Covid-19 is important for health intervention plan for the future. To demonstrate that the RIN is an accurate forecasting method, the RIN was applied to confirmed accumulated cases of COVID-19 across 184 countries. Figure 3 plotted reported and one-step ahead predicted time-case curves of Covid-19 where blue dotted curve was the number of reported cumulative cases after the analysis completion. To further reliably evaluate the forecasting accuracy, we reported 10-step ahead forecasted numbers of cumulative cases and errors of Covid-19 of 8 countries in Table S1 starting with April 8, 2020. The forecasting errors were quite mall.

Inputting a sequence of hypothesized public health intervention strategies, the outputs of the RIN decoder were counterfactual numbers of cases of Covid-19 to respond to the intervention strategies. Interventions were measured by number in the interval [0, 1], where 1 indicated the strictest comprehensive public health intervention, 0 indicated no intervention and the values between 0 and 1 indicated the various less strict interventions. To intuitively illustrate the impact of the measure of intervention on the spread of Covid-19, we presented Figure S1 . Figure S1 plotted counterfactual numbers of cases of Covid-19 over time worldwide to respond the interventions with values 0.3, 0.5, 0.7 and 1. We observed that if the measure of intervention was . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . https://doi.org/10.1101/2020.05.05.20091827 doi: medRxiv preprint 1, the number of cases was dramatically deceased to zero. However, when the measure of intervention was 0.3, the number of cases exponentially increased. The measure of intervention had big effect on the spread of Covid-19.

The number of cases of Coid-19 was a function of the past history and the measure of intervention. Forecasting also depended on the measure of intervention. In Table S1 , we also listed the measures of the interventions which provided information on the degrees of current interventions in the country. The measure of interventions in the most countries was 0.75.

However, the current measure of interventions in UK was 0.5, the smallest in 8 countries. These results showed that the RIN for forecasting the trajectory of Covid-19 was accurate and reliable.

Similar to causal inference, the RIN can be used to evaluate the impact of a sequence of multiple intervention strategies on the curbing the spread of Covid-19 if the interventions were viewed as treatments.

To investigate how Covid-19 pandemic surges around the world, we presented Figure 4 that showed the forecasted number of cumulative cases of Covid-19 worldwide over time, assuming that the current intention remains unchanged. We observed that the number of cumulative cases of Covid-19 exponentially grown and would reach extremely high number 199,554,596 on July 6, 2020 if none of additional comprehensive public health intervention was implemented.

Similarly, Figure 5 and Figure S2 plotted time-case curves of Covid-19 of eight countries: Italy, Spain, Iran, Germany, USA, France, Belgium and UK, and worldwide with unchanged intervention strategies in the future, respectively. We also observed exponentially growth of the numbers of cases of Coid-19 for every country without additional intervention.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. .

As Covid-19 Accelerates and exponentially grows, how to slow down the spread of Covid-19 is an urgent task for every country around world. To demonstrate that when the additional intervention was implemented, the number of new cases of Covid-19 would decrease, we (Table S1 ). These countries were closing to stopping the spread of Covid-19 if additional interventions such as wearing face masks were implemented.

Next we investigated how various intervention strategies reduced the peak time and cumulative case numbers, and the final total number of cases. Table S2 showed the forecasted results of COVID-19 in 12 countries under three sequences of interventions (Scenarios 1-3) . We can see that under all three scenarios, the peak times were before April 15, 2020 and the spread of COVID-19 would be sopped before the end of May, 2020.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . https://doi.org/10. 1101 /2020 As an alternative to the epidemiologic transmission models, we formulated the real-time forecasting and evaluating multiple public health intervention problem into a novel causal inference problem. We viewed the interventions as treatments where multiple interventions were administered at different time points. The number of new cases were taken as treatment responses. The RIN uses sequence-to-sequence multi-input/output recurrent neural network as a tool for modeling the real-time trajectory of the transmission dynamics of Covid-19, health intervention planning and making multi-step prediction of the response trajectory of Covid-19 over time with multiple interventions. The RNN can learn the complex dynamics within the temporal ordering of input time series of Covid-19 and use an internal memory to remember the hidden features.

This AI and causal inference-inspired approach allows us to address three important questions. The first question is the prediction accuracy. Unlike other dynamic systems where the parameters in the systems and control variables are, in general, independent, the epidemic systems have intervention and system dependent parameters. We designed the intervention variable that quantified comprehensive intervention strategies and had close relationships with the parameters in the epidemic systems. Therefore, the RINs could take the parameters in the epidemic dynamic systems as input control variables that can be estimated in the RIN training.

The RIN models were closer to real epidemic dynamic systems than the epidemiological models. Therefore, our results showed that the RIN substantially improved the accuracies of prediction and subsequently multiple-step forecasting.

The second question is how important is the intervention time. Since interventions are complicated and are difficult to quantify, we designed four intervention scenarios to represent the degrees and delays of interventions. Since the proposed methods combine the real data and . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . https://doi.org/10.1101/2020.05.05.20091827 doi: medRxiv preprint models, they allowed us to evaluate the consequences of multiple intervention strategies, while maintaining the analysis as close to the real data as possible. The RIN investigated the impact of multiple public intervention plans and intervention measures on the size, duration and time of the virus outbreak and recommended the appropriate intervention times.

We estimated the duration, peak time and ending time, peak number of new cases and cumulative cases , and maximum number of cumulative cases of COVID-19 under four intervention scenarios for 184 countries in the world. We observed that the number of cumulative cases of Covid-19 would exponentially grow and reach extremely high number 199,554,596 on July 6, 2020 if none of additional comprehensive public health intervention was implemented. However, we also found that top 12 countries with the largest number of the lab confirmed cumulative cases of COVID-19 were closing to stopping the spread of Covid-19 if additional interventions such as wearing face masks were implemented. We can see that under all three scenarios, the peak times were before April 15, 2020 and the spread of COVID-19 would be sopped before the end of May, 2020.

Non to declare . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . https://doi.org/10.1101/2020.05.05.20091827 doi: medRxiv preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . https://doi.org/10.1101/2020.05.05.20091827 doi: medRxiv preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . https://doi.org/10.1101/2020.05.05.20091827 doi: medRxiv preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020 . . https://doi.org/10.1101 /2020 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020 . . https://doi.org/10.1101 /2020 Table S1   Table S2 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020 . . https://doi.org/10.1101 /2020 Let ℎ = [ℎ 1 , … , ℎ ] be a dimensional hidden state vector. The data is inputted into the input layer. The linear transformation ℎ of the data is then sent to the hidden layer, where ℎ is a × ( + 2) dimensional matrix. The hidden layer receives information from the input layer and hidden layer at the previous time point. The state is determined by the following nonlinear transformation of its received information: ℎ = ℎ ( ℎℎ ℎ −1 + ℎ + ℎ ) , . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. . https://doi.org/10. 1101 /2020 where ℎℎ is a × dimensional weight matrix that connect the previous state to the current state, and ℎ = [ ℎ 1 , … , ℎ ] is a dimensional bias vector that corrects the bias, and ℎ is a element-wise nonlinear activation function and is often defined as the following "tanh" function:

The neurons in hidden layer are connected to the output layer via a dimensional weight vector ℎ . The output ̂+ 1 is determined by

where is an activation function and is the bias vector of the output neurons. 

The next step, the input gate layer decides which values we'll update:

= ( + ℎ ℎ −1 + ), . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 8, 2020. 

The old cell state is updated by

where ° denotes element-wise multiplication.

Output gate decides what parts of the cell state we're going to output:

Since the output is continuous, the mean square errors are used as the loss function for the encoder:

For the decoder, we assumed that observations were batched into shorter sequences of up to .

The loss function for the decoder is defined as

Algorithm for RIN training is summarized as follows (Lim et al., 2018) .

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 8, 2020 . . https://doi.org/10.1101 /2020 Input: training and validation data { 1 , … , , 1 , … , , 1 , … , } .

Output: Weights in the RNN encoder and RNN decoder.

Step 1 : Estimate parameters in encoder .

Fit the encoder and estimate by optimizing in equation (A1):

← min ( ) .

Step 2 Step 3: Estimate parameters in RNN decoder

Fit the decoder and estimate by optimizing in equation (A2):

← min ( ) ..

Potential outcome framework is our basic model to evaluate the impact of the public health interventions on the spread of Covid-19. The potential outcome framework is often referred to the Neyman-Rubin model (Rubin 1974) . Potential outcomes consist of actual (or observed) and

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 8, 2020. . https://doi.org/10.1101/2020.05.05.20091827 doi: medRxiv preprint counterfactual (hypothesized) outcomes. We are interested in number of cases of Covid-19 under some specific intervention. We observed the number of cases of Covid-19 (actual observation) without intervention. However, we want to know what number of cases of unobserved) would be if an intervention was implemented. To evaluate the effect of intervention, we should compare the difference between the observed actual number of cases of Covid-19 and the counterfactual number of cases of Covid-19. Our aim is to learn the counterfactual outcomes of Coid-19 under a sequence of public health intervention options and evaluate the impact of the intervention strategies on the spread of Covid-19.

Let be a set of covariates and be an intervention at time . can be a binary variable.

= 1 ( = 0) indicates that intervention is (not) implemented. can also be continuous variable taking values in the interval [0, 1]. If is a continuous variable, the value of represents the intensity of intervention. = 1 indicates that the intervention is the most strict and comprehensive public health intervention. Let +1 = ( ) be the potential outcome under intervention and be observed only when = . The potential outcome framework assumes the existence of the hypothetical outcome with some interventions which is not observed in the data. The hypothetical outcome under hypothetical intervention is called counterfactual outcome.

The set { , , +1 } forms a potential framework for causal inference.

Data-based analysis, modelling and forecasting of the COVID-19 outbreak

The math behind why we need social distancing, starting right now

Early transmission dynamics in Wuhan, China, of novel Coronavirus-infected pneumonia

Nowcasting and forecasting the potential domestic and international spread of the 2019-nCoV outbreak originating in Wuhan, China: a modelling study

Estimating the unreported number of novel Coronavirus (2019-nCoV) cases in China in the first half of January 2020: A data-driven modelling analysis of the early outbreak

Analysis and projections of transmission dynamics of nCoV in Wuhan

Reporting, epidemic growth, and reproduction numbers for the 2019 novel coronavirus (2019-nCoV) epidemic

Centre for the Mathematical Modelling of Infectious Diseases COVID-19 Working Group, Funk S1, Eggo RM2. Feasibility of controlling COVID-19 outbreaks by isolation of cases and contacts. Lancet Glob Health

Forecasting errors of worldwide and eight countries

Country Estimated Reported Error