key: cord-0921952-d5ynhhns
authors: Shankar, Subramanian; Mohakuda, Sourya Sourabh; Kumar, Ankit; Nazneen, P.S.; Yadav, Arun Kumar; Chatterjee, Kaushik; Chatterjee, Kaustuv
title: Systematic review of predictive mathematical models of COVID-19 epidemic
date: 2021-07-26
journal: Med J Armed Forces India
DOI: 10.1016/j.mjafi.2021.05.005
sha: 33573546dff3d73fdb01d1e1e978983875a452e5
doc_id: 921952
cord_uid: d5ynhhns

BACKGROUND: Various mathematical models were published to predict the epidemiological consequences of the COVID-19 pandemic. This systematic review has studied the initial epidemiological models. METHODS: Articles published from January to June 2020 were extracted from databases using search strings and those peer-reviewed with full text in English were included in the study. They were analysed as to whether they made definite predictions in terms of time and numbers, or contained only mathematical assumptions and open-ended predictions. Factors such as early vs. late prediction models, long-term vs. curve-fitting models and comparisons based on modelling techniques were analysed in detail. RESULTS: Among 56,922 hits in 05 databases, screening yielded 434 abstracts, of which 72 articles were included. Predictive models comprised over 70% (51/72) of the articles, with susceptible, exposed, infectious and recovered (SEIR) being the commonest type (mean duration of prediction being 3 months). Common predictions were regarding cumulative cases (44/72, 61.1%), time to reach total numbers (41/72, 56.9%), peak numbers (22/72, 30.5%), time to peak (24/72, 33.3%), hospital utilisation (7/72, 9.7%) and effect of lockdown and NPIs (50/72, 69.4%). The commonest countries for which models were predicted were China followed by USA, South Korea, Japan and India. Models were published by various professionals including Engineers (12.5%), Mathematicians (9.7%), Epidemiologists (11.1%) and Physicians (9.7%) with a third (32.9%) being the result of collaborative efforts between two or more professions. CONCLUSION: There was a wide diversity in the type of models, duration of prediction and the variable that they predicted, with SEIR model being the commonest type.

The first case of COVID-19 was detected in Wuhan on 17 November 2019. 1 On 11 Mar 2020, the WHO declared COVID-19 as a pandemic. 2 This pandemic is the defining global health crisis of our time and the greatest challenge we have faced since World War II. Since its emergence in Asia late last year, the virus has spread to every continent except Antarctica. 3 The total number of cases worldwide on 14 April 2021 was 138,297,267 with 2,976,297 deaths and 111,208,976 recoveries. 3 The seminal paper of Kermack et al. in 1927 introduced the Susceptible, Infectious and Recovered (SIR) model for infectious diseases. 4 Since then, with advances in information technology and fast computing methods, many variants of the SIR model have been developed. Mathematical models predicting the impact of COVID-19 have burgeoned since the onset of pandemic at a global level, with publications on models doubling almost every 20 days. 5 Published predictive models have looked at various aspects of the pandemic. The models have used specific modelling techniques, assumptions and data gathered from real cases. The predicted outcomes depend on these variables. Models developed early in the epidemic might have had greater impact in planning and allocation of healthcare resources than the later ones.

Numerous mathematical models with varied techniques and predictions often leave the readers confused. This systematic review was carried out to analyse the initial epidemiological predictive models of the COVID-19 pandemic.

The protocol for this systematic review was registered at www.osf.io on 30/07/2020. 6 The population for the review comprised all studies on mathematical modelling of the COVID-19 epidemic. The outcomes of interest were epidemiological predictions of the model including the techniques used for mathematical predictions. Only those mathematical models that were published in peer-reviewed journals were included in the study. Studies without explicit mathematical modelling, reviewing COVID-19 with guesstimates, with no full text freely available, with only model description but no predictions and pre-print articles, were excluded. Narrative reviews and commentaries, perspective articles too were excluded, unless they provided novel modelling analyses or outcomes. Publication date range included in our review was from January to June 2020, in the predefined databases to ensure that we considered the initial models.

A detailed literature search was carried out and the following databases were searched; Medline through PubMed, Web of Science, medRxiv, bioRxiv and arXiv. Key words used were mathematical modelling, predictive modelling, COVID-19, SARS-CoV-2, peak infected cases and total deaths. The search strategy followed a two-stage approach. In the first stage, databases were searched using the keywords, and in the second stage, a manual search of articles from the references of the selected article was carried out. The literature search was conducted by two researchers independently. The searches were then reviewed by two other authors.

A data extraction form was designed for the study. Two authors independently extracted the data from all the studies and then compared them. In case of any disagreement, a designated third author again extracted the data separately and the discrepancy was resolved through mutual consultation.

Models were assessed for various prediction characteristics such as peak numbers, total infections, point of time where the pandemic was predicted to peak at a particular location, time to end of pandemic, effectiveness of nonpharmaceutical interventions (NPIs) and hospital admissions. The models were classified as 'early' if they had made their predictions before cases reached 2000 at their respective geographical location. Others were classified as 'late' models. Short-term models were those that predicted up to two weeks while long-term models predicted beyond this period.

Risk of bias in mathematical model is difficult to assess including with the use of PROBAST checklist for predictive models in medical science, as there is no selection and measurement bias in mathematical models, only the approach and type of models may differ. 7 Quality assessment of the study articles was carried out based on five questions adapted from Holmdahl et al. published in June 2020. 8 Narrative synthesis was performed as per the synthesis without metaanalysis guideline. 9

The chosen search strings yielded 56,922 hits in the five databases. The screening of titles yielded 434 abstracts, of which 266 did not contain predictive models. Another 96 were excluded due to other reasons listed in the PRISMA chart ( Fig. 1) . Finally, 72 articles were included in the systematic review. 10e81 Over 70% (51/72) of the articles contained predictive mathematical models, with susceptible, exposed, infectious and recovered (SEIR) being the commonest one (41/72, 56.9%). The common predictions were regarding cumulative cases (44/72, 61.1%), time to reach total numbers (41/72, 56.9%), peak numbers (22/72, 30.5%), time to peak (24/72, 33.3%), hospital utilisation (7/72, 9.7%) and impact of lockdown and NPIs (50/72, 69.4%). Prediction characteristics studied are listed in Supplementary Table 1. It was found that of the 72 articles, 20 articles were based on hypothetical scenarios which predicted outcomes in numbers, but these outcomes (number of asymptomatic cases and effect of screening, etc) could not be compared with real data. Among those which did (n ¼ 52), the median duration of prediction was 3 months (interquartile range, 2e4 months; range: 10 dayse48 months). Curve-fitting models invariably made short-term predictions.

While certain models predicted definite outcomes in terms of time and numbers (51/72, 71%), others were pure mathematical models (21/72, 29%) which studied the impact of various parameters on the pandemic such as basic reproduction number (R 0 ), variable contact rates of infectious individuals, time spent in crowded zones and population density. 22 Of the 51 models which predicted definite outcomes, models were divided into long-term (n ¼ 29) and short-term (n ¼ 22) models. Furthermore, the long-term models were classified into early predictor models (n ¼ 5) and late predictor models (n ¼ 24) (Fig. 3 ). Among articles with definite predictions in terms of numbers or time, 22 were curve-fitting models with short-term outcomes, which modelled the epidemiological characteristics around the peak in their region. 11,13,14,20,22,23,25,26,34,37,40,42,48e50,53,57,60e62,64,74 While one model looked into the effects of lockdown and other NPIs like screening at airports another looked at specific impact of lockdown in India at 21, 42 and 60 days. 39, 44 Many models predicted the estimated burden on health care system and utilisation. 10, 23, 30, 38, 51, 52, 74 The countrywise distribution of the studies is shown in Fig. 2 . Most of the models predicted outcomes for China (22/ 71, 31%) followed by USA, South Korea, Japan, India, Iran, UK and Canada (Fig. 2) . While most models predicted for a 

single country or city, there were 12 models that predicted outcomes for more than one country and 3 models that made worldwide predictions. Predictions were also made for the following regionsdHubei Province in China, California, Michigan and Utah in USA, Ontario in Canada, cities such as New York, London and Wuhan and cities in India such as New Delhi, Kolkata, Mumbai and Bangalore. Most studies were published in March (26/72, 36.1%), followed by April (21/72, 29.2%).

Mathematical models were published by various professionals including Engineers (12.5%), Mathematicians (9.7%), Epidemiologists (11.1%), Physicians (9.7%), Biostatisticians (5.5%) and others (18%). A third of the articles (32.9%) were collaborative efforts between two professions (e.g., Mathematicians and Physicians).

Quality assessment of the models is depicted in Table 1 . All the models have clearly defined the purpose in their study. Only 11 (15.27%) considered the population density in the region studied.

A table of narrative synthesis of all the articles is provided depicting authors, title, geographical area of predictions, mathematical modelling technique and outcomes evaluated (Suppl Table 1 ). The characteristics of early and late predictors are also shown (Suppl Table 1 ).

"All models are wrong, but some are useful" said George Fox. 82 In a situation like the COVID-19 epidemic where uncertainty is rife, predictions from epidemiological models are one of the key tools available for early decision-making.

This review of all the published initial peer-reviewed mathematical models of COVID-19 attempts to summarise and synthesise their findings. The question of prediction arises in the initial stages of any situation (especially a pandemic like COVID-19) that is fraught with ambiguity, chaos and uncertainty. With the passage of time, most situations become clearer and hard data become available for computation. Hence, we did not consider subsequent models from July 2020 onwards. Our review showed that of the large numbers of models published, only a few were both predictive and peer reviewed. Because we studied only the peer-reviewed models, it is likely that models which were published largely for planning purposes by governments or models from news or social media are not included. It is also possible that some authors chose to forgo peer review and submitted their articles directly to various databases.

Systematic review of mathematical models has earlier been carried out in the context of other infectious diseases. Prieto et al. performed a systematic review to identify areas of enhancement of pandemic simulation models of Influenza epidemics for operational use at provincial and local levels. 83 A study by Harris et al. has reviewed mathematical models exploring the epidemiological impact of future TB vaccines. 84 There was a wide heterogeneity seen in the mathematical models evaluating epidemiology of the COVID epidemic ( Table  2 ). The commonest one was SIR model and its variant. The model divided the population into different compartments and the movement of population from one compartment to another is predicted by differential equations. The approach is flexible as more number of compartments may be added. Both stochastic and deterministic model are possible. There were many noneSEIR-based models based on regression models and techniques such as ARIMA (based on time series). However, the model used depends on the aim of the researcher and data available. None of the methods is established as superior to another. 85 While short-term models that predicted over just two weeks were more accurate, they were not particularly useful due to the short prediction horizon. These curve-fitting models predicted close to actual outcomes, but with minimal preparedness benefits. However, in special situations where short-term predictions are desired, these might still have a role. These models can be used for validating a new set of assumptions or a new modelling technique.

A few long-term models were constructed early in the epidemic. What they lacked in precision, they made up in usefulness. 86 Later models were more precise due to larger data sets of more cases, but their usefulness progressively declined. Pure mathematical models (open-ended models) were used to evaluate concepts for validation or preparedness, without predicting any particular quantifiable outcome.

Mathematical modelling was done by a variety of professionals from engineers and mathematicians to doctors and biologists. Epidemiologists comprised about a tenth of all modellers. This demonstrated the interest that the epidemic had generated among various scientists traversing traditional professional silos. Even more interesting was the fact that almost a third of all models were collaborations between multiple professions.

Our study is one of the first systematic reviews of mathematical models of COVID-19 and seeks to synthesise the key characteristics. A limitation of our study was that we were unable to combine the various modelling results into a common numerical estimate. We attempted to estimate the Mean Absolute Percentage Error (MAPE) of the models with real-time data but were constrained by the predicted numbers not being available for different time-points. Another possible limitation was that we have not included articles whose free full text was not available. 

An ideal model would be one which factors in the maximum number of relevant variables which could possibly affect study outcomes and predicts with closest proximity to the real outcomes. In this systematic review, the chosen mathematical models had different modelling and prediction characteristics, precluding such an analysis.

There was a wide range of variation in the outcomes predicted in various studies. Which is the model that a country should follow remains an unresolved question till date. It would be ideal for a model to have inputs from mathematicians, epidemiologists, and health care workers (involved in dealing with COVID). Inputs from varied professions help ensure the widest range of reasonable assumptions and make the model more robust. This epidemic is an opportunity for those interested in mathematical modelling (mathematicians, epidemiologists, HCWs and others) to collaborate and improve the models for their regions, considering various parameters based on their specific experience and training. It is also an opportunity for planners and administrators to utilise comprehensive models for informing the planning and preparedness for their regions.

The advent of newer technologies has made mathematical calculation relatively easy. Many free software such as 'R', 'PYTHON' etc. are now available for mathematical modelling. In the future, the use of more advanced and refined technologies such as artificial intelligence will allow more accurate real-time predictions.

Mathematical models have used varied assumptions and have predicted various outcomes. Actual data for comparisons remains dynamic as the pandemic is still evolving. Data are changing rapidly depending on time, place and effects of NPIs. Hence, it remains difficult to compare predictions with actual outcomes and also to compare all available models at a single platform. A similar review may require to be carried out after some time, when more epidemic data are available and more models are published.

This systematic review analyses all the initial epidemiological models of the COVID-19 pandemic from January to June 2020. The analysis of mathematical models was constrained by varied prediction parameters of different models and differing time horizons of predictions among different models. There was a wide range of assumptions and implications. Thus, no particular model was substantially superior to others.

This review revealed that majority of the mathematical models studied the effect of NPIs, which helped administrators to plan preventive measures. Early predictor models were possibly of greatest utility to administrators in taking planning decisions. SEIR variants were the commonest modelling technique. This systematic review has utility in helping future modellers choose among assumptions to incorporate in their Table 1 e Qualitative analysis of mathematical models. 8 

Question Yes (%) 1 Has the purpose of model been clearly depicted in the study? 72 (100%) 2

Has immunity been taken into account In the model? 54 (75%) 3 Has asymptomatic transmission been taken into account into the model? 37 (52%) 4 Has contact transmission been taken into account in the model? 72 (100%)  5 Has the statistical model displayed confidence intervals? (n ¼ 22) 22 (100%) 6 Has the mechanistic model depicted various parameters, ranges? (n ¼ 50) 50 (100%) 7 Has the prediction been made for a defined geographical region? 58 (81%) 8

Is population density taken into account in the model? 11 (16%) Highly dependent on estimation of parameters.

Do not, usually, take into account variability of parameters during the course of the epidemic.

Data-driven models (n ¼ 31)

Usually curve-fitting in nature. Can be predictive or pure mathematical (Open-ended). Used to evaluate concepts for validation or preparedness e.g. Regression model, ARIMA, Log logistic model etc.

Generally have a good fit to retrospective data. Good for short-term projections based on current estimated parameters.

Do not take into account dynamics of disease spread. Lack reliability for long-term predictions.

m e d i c a l j o u r n a l a r m e d f o r c e s i n d i a 7 7 ( 2 0 2 1 ) S 3 8 5 eS 3 9 2 models and also decide on preferred prediction parameters for a particular location or region.

Four of the authors have previously authored a mathematical model for the COVID epidemic.

1st known case of coronavirus traced back to November in China

COVID-19) -events as they happen

Live): 8,649,131 Cases and 458,122 Deaths from COVID-19 Virus Pandemic -Worldometer

A contribution to the mathematical theory of epidemics

Scientists are drowning in COVID-19 papers. Can new tools keep them afloat? [Internet] Science

PROBAST: a tool to assess the risk of bias and applicability of prediction model studies

Wrong but useful d what Covid-19 epidemiologic models can and cannot tell us

Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline

Projecting hospital utilization during the COVID-19 outbreaks in the United States

Mathematic modeling of COVID-19 in the United States

Naive forecast for COVID-19 in Utah based on the South Korea and Italy models-the fluctuation between two extremes

Forecasting the novel coronavirus COVID-19

A conceptual model for the coronavirus disease 2019 (COVID-19) outbreak in Wuhan, China with individual reaction and governmental action -ScienceDirect

Data-based analysis, modelling and forecasting of the COVID-19 outbreak

Strong correlations between power-law growth of COVID-19 in four continents and the inefficiency of soft quarantine strategies

A model for COVID-19 prediction in Iran based on China parameters

The exponentially increasing rate of patients infected with COVID-19 in Iran

When will the battle against novel coronavirus end in Wuhan: a SEIR modelinganalysis

Prediction of COVID-19 transmission dynamics using a mathematical model considering behavior changes in Korea

Estimation of the reproductive number of novel coronavirus (COVID-19) and the probable outbreak size on the Diamond Princess cruise ship: a data-driven analysis

Mathematical assessment of the impact of non-pharmaceutical interventions on curtailing the 2019 novel Coronavirus

Outbreak dynamics of COVID-19 in China and the United States

Propagation analysis and prediction of the COVID-19

Predicting the cumulative number of cases for the COVID-19 epidemic in China from early data. medRxiv

Modeling the epidemic dynamics and control of COVID-19 outbreak in China. medRxiv

Transmission potential of COVID-19 in South Korea. medRxiv

Estimation of the excess COVID-19 cases in Seoul, South Korea by the students arriving from China

Mathematical modelling of COVID-19 transmission and mitigation strategies in the population of Ontario

The effectiveness of quarantine of Wuhan city against the Corona Virus Disease 2019 (COVID-19): a well-mixed SEIR model analysis

COVID-19 virus outbreak forecasting of registered and recovered cases after sixty day lockdown in Italy: a data driven model approach

Prediction of the epidemic peak of coronavirus disease in Japan, 2020

Mathematical modeling of the spread of the coronavirus case of China

Preliminary prediction of the basic reproduction number of the Wuhan novel coronavirus 2019-nCoV

Spatial-temporal distribution of COVID-19 in China and its prediction: a data-driven modeling analysis

Healthcare impact of COVID-19 epidemic in India: a stochastic mathematical model

Prudent public health intervention strategies to control the coronavirus disease 2019 transmission in India: a mathematical model-based approach

Modified SEIR and AI prediction of the epidemics trend of COVID-19 in China under public health interventions

Estimating the reproductive number and the outbreak size of COVID-19 in Korea

Mathematical prediction of the time evolution of the COVID-19 pandemic in Italy by a Gauss error function and Monte Carlo simulations

COVID-19 epidemic prediction and the impact of public health interventions: a review of COVID-19 epidemic models

Analysis of COVID-19 infection spread in Japan based on stochastic transition model

Fitting and forecasting the trend of COVID-19 by SEIR(þCAQ) dynamic model

Mathematical modelling to assess the impact of lockdown on COVID-19 transmission in India: model development and validation

A possible scenario for the Covid-19 epidemic, based on the SI(R) model. SN Compr Clin Med

Early prediction of the 2019 novel coronavirus outbreak in the mainland China based on simple mathematical model

An efficient COVID-19 prediction model validated with the cases of China, Italy and Spain: total or partial lockdowns?

Spread and influence of COVID-19 in China

Can we predict the occurrence of COVID-19 cases? Considerations using a simple model of growth

Dynamic interventions to control COVID-19 pandemic: a multivariate prediction modelling study comparing 16 worldwide countries

Forecasting the impact of coronavirus disease during delivery hospitalization: an aid for resource utilization

Modelling SARS-COV2 spread in London: approaches to lift the lockdown

Impact of school closures for COVID-19 on the US health-care workforce and net mortality: a modelling study

The effect of control strategies to reduce social mixing on outcomes of the COVID-19 epidemic in Wuhan, China: a modelling study

The correlation between the spread of COVID-19 infections and weather variables in 30 Chinese provinces and the impact of Chinese government mitigation plans

SACEMA Modelling and Analysis Response Team, CMMID COVID-19 working group, et al. Projected early spread of COVID-19 in Africa through 1

Modeling COVID-19 latent prevalence to assess a public health intervention at a state and regional scale: retrospective cohort study

A multi-region discrete time mathematical modeling of the dynamics of Covid-19 virus propagation using optimal control

Forecasting the cumulative number of COVID-19 deaths in China: a Boltzmann function-based modeling study

Development of new hybrid model of discrete wavelet decomposition and autoregressive integrated moving average (ARIMA) models in application to one month forecast the casualties cases of COVID-19

Prediction of the COVID-19 pandemic for the top 15 affected countries: advanced autoregressive integrated moving average (ARIMA) model. JMIR Public Health Surveill

Prediction of the COVID-19 spread in African countries and implications for prevention and control: a case study in South Africa

Evaluation and prediction of the COVID-19 variations at different input population and quarantine strategies, a case study in Guangdong province, China

School opening delay effect on transmission dynamics of coronavirus disease 2019 in Korea: based on mathematical modeling and simulation study

Estimation of the transmission risk of the 2019-nCoV and its implication for public health interventions

Preparedness and vulnerability of African countries against importations of COVID-19: a modelling study

COVID-19 outbreak on the Diamond Princess cruise ship: estimating the epidemic potential and effectiveness of public health countermeasures

Estimation of the probability of reinfection with COVID-19 by the susceptible-exposed-infectiousremoved-undetectable-susceptible model

Prediction of COVID-19 outbreak in China and optimal return date for university students based on propagation dynamics

On a statistical transmission model in analysis of the early phase of COVID-19 outbreak

A simulation on potential secondary spread of novel coronavirus in an exported country using a stochastic epidemic SEIR model

Feasibility of controlling COVID-19 outbreaks by isolation of cases and contacts. Lancet Global Health

A Simulation of a COVID-19 Epidemic Based on a Deterministic SEIR Model. Front Public Health

COVID-19 spreading in Rio de Janeiro, Brazil: do the policies of social isolation really work? Chaos, Solit Fractals

COVID-19 UK lockdown forecasts and R 0. Front Public Health

Dynamics of COVID-19 pandemic at constant and time-dependent contact rates

Early transmission dynamics of COVID-19 in a southern hemisphere setting

Estimating the burden of SARS-CoV-2 in France

Modelling-based evaluation of the effect of quarantine control by the Chinese government in the coronavirus disease 2019 outbreak. Sci China Life Sci

Predicting COVID-19 peaks around the World

The relatively young and rural population may limit the spread and severity of COVID-19 in Africa: a modelling study

Science and statistics

A systematic review to identify areas of enhancements of pandemic simulation models for operational use at provincial and local levels

Systematic review of mathematical models exploring the epidemiological impact of future TB vaccines

Predictive models of COVID-19 in India: a rapid review

Levins-1966-Model_Building.pdf [Internet