key: cord-1002603-ncb2hx5f
authors: Alassafi, Madini O.; Jarrah, Mutasem; Alotaibi, Reem
title: Time Series Predicting of COVID-19 based on Deep Learning
date: 2021-10-19
journal: Neurocomputing
DOI: 10.1016/j.neucom.2021.10.035
sha: b60bf3016786148894f6fd9db10a9634937b999b
doc_id: 1002603
cord_uid: ncb2hx5f

COVID-19 was declared a global pandemic by the World Health Organisation (WHO) on 11th March 2020. Many researchers have, in the past, attempted to predict a COVID outbreak and its effect. Some have regarded time-series variables as primary factors which can affect the onset of infectious diseases like influenza and severe acute respiratory syndrome (SARS). In this study, we have used public datasets provided by the European Centre for Disease Prevention and Control for developing a prediction model for the spread of the COVID-19 outbreak to and throughout Malaysia, Morocco and Saudi Arabia. We have made use of certain effective deep learning (DL) models for this purpose. We assessed some specific major features for predicting the trend of the existing COVID-19 outbreak in these three countries. In this study, we also proposed a DL approach that includes recurrent neural network (RNN) and long short-term memory (LSTM) networks for predicting the probable numbers of COVID-19 cases. The LSTM models showed a 98.58% precision accuracy while the RNN models showed a 93.45% precision accuracy. Also, this study compared the number of coronavirus cases and the number of resulting deaths in Malaysia, Morocco and Saudi Arabia. Thereafter, we predicted the number of confirmed COVID-19 cases and deaths for a subsequent seven days. In this study, we presented their predictions using the data that was available up to December 3rd, 2020.

The COVID-19 virus has been responsible for ≥113 million confirmed cases and ≥2. The WHO declared this disease as a pandemic during the initial phases of its transmission, indicating that it is a very severe and deadly disease [3] . It is noted that the coronavirus significantly affects the health of people and even causes death, either directly or through exacerbating pre-existing health problems. As a large proportion of people have been affected by the COVID-19 pandemic throughout the world, and there is no cure available for the disease, it becomes important to estimate the number of potential cases that may occur using available data.

Many researchers, including data scientists, have been working intensely to determine ways to eradicate this disease. Data scientists can effectively contribute to the research by designing prediction models that highlight the probable activities of this virus, which can further help in accurately predicting the spread of this virus. Hence, deep learning (DL) models are regarded as accurate tools which can help in developing prediction models. Though many neural networks (NNs) have been described in the past, the recurrent neural network (RNN) and the long short-term memory (LSTM) are investigated in the forecasting of COVID-19 as they can use temporal data [4] .

In this study, RNN and LSTM deep-learning networks have been used. These networks were selected as they could analyse the time series data and accurately predict future trends [4] .

These two models showed considerable success in forecasting temporal data among other traditional methods. Firstly, RNNs have been used for processing the time series and the sequential data, which are also helpful in modelling sequence data. Derived from feedforward networks, RNNs exhibit similar behaviour to how the human brain functions. Simply put, RNNs produce predictive results in sequential data that other algorithms cannot. Then, LSTMs, which have a sophisticated gated memory unit designed to handle the vanishing gradient problems in simple RNNs limiting the efficiency, have been used [5, 6, 7, 8, 9, 10] .

The main contribution of this paper is that it proposes DL prediction models which can present the best results related to the prediction of confirmed positive COVID-19 cases and cases of death attributed to COVID-19 in Malaysia, Morocco and Saudi Arabia using past and current data. This study uses the Rectified Linear Unit (ReLU) activation function existing in LSTMs [11] ; along with tanh and sigmoid activation functions presented in the RNN models [12] . It further predicts the number of coronavirus cases and the deaths directly resulting from this disease using NNs. These NNs used the existing datasets that contained all available data related to the COVID-19 pandemic in countries such as Saudi Arabia, Morocco and Malaysia. This paper is structured as follows: after this introduction, the second section covers the main objectives. Section 3 covers related works on predicting COVID-19. This is followed by an explanation of the data and the research methodology in the fourth and fifth section. The experiment set-up and the analysis of the results are introduced in Sections 6 and 7, respectively. Section 8 discusses the results and, finally, the conclusion is given in Section 9.

This research paper aims to fulfil the following objectives:

1. To compare and assess the performance of two NN prediction models, i.e., RNN and LSTM, for understanding which model shows a better performance while predicting the number of positive COVID-19 cases, COVID-recovered cases, and the level of mortality caused by the disease.

2. To use an effective activation function that helps in achieving the best acceptable performance.

3. To estimate the number of potential coronavirus cases for a subsequent seven days.

Dechter stated that the concept of DL was a complement of machine learning (ML) [9] .

Deep learning was seen to be a subset of machine learning and even artificial intelligence (AI).

The AI technique allows computers to imitate human behaviour, while ML displays similar behaviour after using data-driven algorithms. On the other hand, deep learning is regarded as the component of machine learning which is influenced by the human brain structure. This framework is called an artificial neural network (ANN). While training the model using ML, we need to determine all features which are considered by the model while differentiating between two objects. On the other hand, in DL, these features were derived by the neural networks without requiring any human interference. This degree of independence was only achieved after using a large data volume for training the machines.

For predicting the number of people who would succumb to the COVID-19 virus in the subsequent ten days, we made use of the RNN and LSTM algorithms. In an earlier study, we thoroughly reviewed the prevailing COVID-19 daily cases that used LSTM, RNN, and Gated Recurrent Unit (GRU) and successfully predicted the approximate number of deaths over the next ten days [13] . As an alternative, Zeroual, A. et al., [14] used a technique using RNN, LSTM, Bidirectional LSTM (Bi-LSTM), and GRUs that required the daily number of confirmed COVID-19 cases as the input value. This approach considered the predicted number of new contaminated and recovered cases. In another study, the researchers [15] analysed the confirmed and fatal COVID-19 cases to predict the next month's number of cases and deaths using RNN, Stacked LSTM, Bi-LSTM and Convolutional LSTM. However, the researchers in [16] proposed an effective COVID-19 prediction model based on LSTM for the data for daily confirmed cases in both national and provincial levels in Iran. The model was able to predict the cases for a subsequent 21 days and it performed better than the other methods. Abbasimehr and Paki [17] presented a forecast of the pandemic based on LSTM and CNN with the Bayesian optimization algorithm. The model's effectiveness was evaluated using symmetric mean absolute percentage error (SMAPE) which were 0.25 and 2.59 for the short-term (ten days ahead) and for long-term forecasting, respectively. A different forecasting model for COVID-19 was also proposed by Wang et al. using LSTM for predicting cases over the next 30 days [18] .

A few machine learning algorithms were also suggested by the Shahid, F., et al., such as autoregressive integrated moving average (ARIMA), SVR and deep learning algorithms such as LSTM and Bi-LSTM to be used for forecasting cases and deaths associated with COVID-19 where the accuracy accomplished for each model revealed that Bi-LSTM generated lowest MAE and RMSE values of 0.0070 and 0.0077, respectively. [19] . The LSTM network and fully connected layer proposed by Kai-chaoMiao, et al. the network framework consists of an LSTM network and fully connected layer. In order to make the proposed LSTM framework work, the meteorological element observation data returned hourly is transferred into time series data [20] . Kai and et. al used deep supervised learning using self-adaptive auxiliary loss for COVID-19 diagnosis from imbalanced CT images [21] . Table 1 below summarises the related work.

This research used daily confirmed cases and deaths data from Saudi Arabia, Morocco and Malaysia, which were available from the European Centre for Disease Prevention and Control 1 .

Firstly, this research used data from March 15, 2020, as the date of the first reported case until December 3, 2020. It contained of 816 records and its total size was 4.5 MB. The data for that period was used as follows: 80% for training and 20% for testing [22] of models to find the appropriate parameters. After training, the next step was testing. The pattern of daily confirmed cases for Saudi Arabia, Malaysia, and Morocco -COVID-19 are presented in Fig.1 and Fig. 2 shows the pattern of daily deaths cases for the three countries. 

Deep neural networks are seen to be an effective technique, which could be used for automatically learning the arbitrary complex mappings from inputs to outputs. These processes support multiple inputs and outputs. Furthermore, these processes are robust to non-linear, multivariate or noisy inputs and multi-step forecasts [23] . Fig. 3 summarises the general research methodology used in this paper to provide COVID-19 predictions using deep neural networks. The special features of this study are also described below:

1. All COVID-19 datasets were considered for training and testing the predictive models [24] .

2. To predict the confirmed cases and deaths related to COVID-19, we have used two types of DL-NN, i.e., RNNs and LSTMs. We estimated and compared the performances of the prediction models using the above NNs.

3. We determined the performances of the RNNs and the LSTMs networks, using three types of activation functions, i.e., Relu, tanh, and sigmoid.

4. The drawbacks of using a simple RNN have been presented below. 

RNNs have been used for processing the time series and the sequential data [5] . The RNN structure included three layers, i.e., an input layer (x), output layer (y), and a hidden layer (h). The advanced feed-forward NNs are generally called RNNs since the information in the simple feedforward NNs moves in one direction, i.e., it is transmitted from the input to the hidden layer and finally to the output layer. This information never moves in the reverse direction [6] .

However, some ties in the RNNs point backwards and show that the information can move in both the forward and the backward directions [7] . These parameters are shared by the RNNs using different time-steps. They consist of loops within the layers, which indicate that a neuron can receive inputs and generate outputs for transmitting the outputs again back to itself. Thus, the RNNs use two forms of input, i.e., existing input which is indicated by xt and another input which is described as yt-1 and is generated from the outputs of the earlier time-steps [1] .

In this study, we have used RNNs to model time series data associated with the COVID-19 time series for removing all noise. In Step 2, the data is categorised into two parts, and the data normalisation technique is used. Here, we carried out data normalisation for adjusting the numerical values within the dataset based on a standard scale, while ensuring that the differences within the value range were not distorted. Thereafter, Step 3 reshapes the 1-D array in a matrix. This matrix can again be converted to form an array. Finally, it is important to initialise the network, i.e., properly set the outputs for the neurons which are initially hidden. The RNN is initialised with the 0-state value, which is also known as the steady state. With regards to the dynamic system recognition, all the above initialisation steps indicate that the system that requires modelling is in a steady state.

RNNs cause the disappearance of the vanishing gradient point error [8] , which leads to the development of a novel model called the Long Short-Term Memory (LSTM) that can handle this issue. These NNs can record the information for a longer period of time. The LSTM networks were first developed in 1997 by Horchreiter and Schmidhuber. They possess a chain-like structure having multiple repeating modules. It was noted that this RNN learning technique could effectively tolerate the avoidance of the vanishing gradient and explosion of the gradient errors by using the LSTMs-RNN process [9] . However, even this special RNN group could resolve these issues [10] .

After we collected the weak results generated from the model that was based on the simple RNN technique, we implemented the LSTM-based prediction model for predicting the COVID-19 cases in the three countries. LSTMs possess certain cell states which can either selectively forget or remember things. Three gates are applicable for a cell state. The first forget gate removes all information from a cell state that it does not need. The second input gate adds vital information to a cell state. The third output gate helps in selecting all vital information and generating output [25] . The cell can store values from a random time interval, while the three gates control the complete information flow into and out of the cell. Fig. 5 describes the LSTM gates [26] while Table 2 presents the formulae used for every component at the time step, t. //Training the model

We evaluated the results of the above experiments using different metrics, i.e., accuracy, mean squared error (MSE), mean absolute error (MAE), mean absolute percentage error (MAPE) and root mean squared error (RMSE) [27] . Accuracy helps in calculating how often the prediction was similar to the actual label. MAE and MSE help in computing the mean absolute error value and mean squared error that is noted between the y_true and y_pred, respectively. Lastly, MAPE is defined as the measure of the prediction accuracy of the forecasting technique used in statistics, such as trend estimation. It is also used as the loss function for regression problems in ML processes.

In this work, we have investigated the best parameter settings such as number of epochs, batch size, and neurons to achieve good prediction result for cases of and death from COVID-19 over a subsequent seven-say period. These parameters are described as follows:

• Epochs: The number of epochs is a factor that defines the number of times that the learning method will function through the whole training dataset. The number of epochs is the number of full passes through the dataset of the training.

• Batch size: The size of batch is a factor that defines the number of samples to work with prior to updating the variables of the internal model. The batch size refers to the number of samples processed ahead of the updating of the model.

• Neurons: The number of neurons impacts the network's learning capacity. In general, the greater the number of neurons, the greater the learning of the problem structure at the expense of a longer learning period. Greater capacity for learning also results in potential problem of overfitting the data used for training [28] . Table 3 shows 

In this section, we have presented the experimental results that were generated after the implementation and analysis of RNN and LSTM with EMA proposed prediction models. RNN and LSTM have been adopted for predicting the two parameters associated with COVID-19 cases in three countries. We determined the number of a) Confirmed COVID cases, and the b) Resultant COVID-related deaths.

We initially implemented the simple RNN model, however, it was soon discarded as it performed poorly. Thereafter, we used the LSTM-based prediction model for predicting the COVID-19 cases derived from the datasets. The LSTMs consist of cell states and actively forget or remember information. The three gates which worked in the cell state included the forget gate, input and output gates. Hence, we used these gates for developing three layers, i.e., the LSTM layer, the Dropout layer and the Dense layer for developing the LSTM model. Here, we implemented two different steps compared to simple RNN. Initially, we set a fixed random seed for reproducibility and used a rectified linear activation function (ReLU). Using the ReLU, we estimated the Keras metrics for the LSTM and derived the best results. Furthermore, we also decreased the vanishing gradient point error. Table 4 presents the results of the RNN-and the LSTM-based prediction models. Fig. 7- to Fig. 18 depict the graphical representation of the results. The results indicated that the LSTMbased prediction model showed a 98.53% accuracy, which was 5.13% better compared to the accuracy displayed by the RNN model. • Gradient vanishing and exploding problems.

• Training RNN is a very difficult task.

• It cannot process very long sequences if using tanh or relu as an activation function [29] .

Also, we found that the proposed LSTM model achieved better results, and this was due to the presence of three types of memory, the first -the Input Gate: decides which values from the input to update the memory state (take the input from tanh and input weight and apply the Sigmoid activation, then the output 0 or 1 

Radiology perspective of coronavirus disease 2019 (COVID-19): lessons from severe acute respiratory syndrome and Middle East respiratory syndrome

Implementation of SimpleRNN and LSTMs based prediction model for coronavirus disease (Covid-19)

Overview of transnational recommendations for COVID-19 transmission control in dental care settings

Comparative analysis of recurrent neural network architectures for reservoir inflow forecasting

Using LSTM for the Prediction of Disruption in ADITYA Tokamak

Time-series well performance prediction based on Long Short-Term Memory (LSTM) neural network model

A review on the long short-term memory model

A Deep Learning Based Hybrid Framework for Day-Ahead Electricity Price Forecasting

Long short term memory recurrent neural network (LSTM-RNN) based workload forecasting model for cloud datacenters

Time series forecasting of petroleum production using deep LSTM recurrent networks

A Framework for Predicting Network Security Situation Based on the Improved LSTM

The impact of activation functions applying to recurrent neural network on Intrusion Detection

Deep learning methods for forecasting COVID-19 time-Series data: A Comparative study

Time series forecasting of Covid-19 using deep learning models: India-USA comparative case study

Covid-19 infection forecasting based on deep learning in iran

Prediction of COVID-19 confirmed cases combining deep learning methods and Bayesian optimization

Time series prediction for the epidemic trends of COVID-19 using the improved LSTM deep learning method: Case studies in Russia, Peru and Iran

Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM

The Prediction of COVID-19 Using LSTM Algorithms

Application of LSTM for short term fog forecasting based on meteorological elements

Deep supervised learning using self-adaptive auxiliary loss for COVID-19 diagnosis from imbalanced CT images

Machine learning recommends affordable new Ti alloy with bone-like modulus', Materials Today

Systematic Generalization in Neural Networks-based Multivariate Time Series Forecasting Models

European Centre for Disease Prevention and Control

Deep learning via LSTM models for COVID-19 infection forecasting in India

Book Short-Term Metro Passenger Flow Prediction Based on Random Forest and LSTM

Foreign Currency Exchange Rate Prediction Using Bidirectional Long Short Term Memory': 'The Big Data-Driven Digital Economy: Artificial and Computational Intelligence

Question-Answer System on Episodic Data Using Recurrent Neural Networks (RNN)': 'Data Management

Book Beyond exploding and vanishing gradients: analysing RNN training using attractors and smoothness

Alassafi received his B.S. degree in

Reem Alotaibi: Conceptualization, Formal analysis, Investigation, Visualization Methodology, Software, Writing and editing original draft preparation

This research work was funded by institutional Fund Project under grant no. (IFPHI-157-612-2020). Therefore, the authors greatly acknowledge technical and financial support from the Ministry of Education and King Abdulaziz University, DSR, Jeddah, Saudi Arabia.