key: cord-0868685-ohhe2qkh
authors: Tsai, Kang-Ting; Chien, Tsair-Wei; Lin, Ju-Kuo; Yeh, Yu-Tsen; Chou, Willy
title: Comparison of prediction accuracies between mathematical models to make projections of confirmed cases during the COVID-19 pandamic by country/region
date: 2021-12-17
journal: Medicine (Baltimore)
DOI: 10.1097/md.0000000000028134
sha: 07654534aa7b2a9d5195353347e9ed3579bd1823
doc_id: 868685
cord_uid: ohhe2qkh

BACKGROUND: The COVID-19 pandemic caused >0.228 billion infected cases as of September 18, 2021, implying an exponential growth for infection worldwide. Many mathematical models have been proposed to predict the future cumulative number of infected cases (CNICs). Nevertheless, none compared their prediction accuracies in models. In this work, we compared mathematical models recently published in scholarly journals and designed online dashboards that present actual information about COVID-19. METHODS: All CNICs were downloaded from GitHub. Comparison of model R(2) was made in 3 models based on quadratic equation (QE), modified QE (OE-m), and item response theory (IRT) using paired-t test and analysis of variance (ANOVA). The Kano diagram was applied to display the association and the difference in model R(2) on a dashboard. RESULTS: We observed that the correlation coefficient was 0.48 (t = 9.87, n = 265) between QE and IRT models based on R(2) when modeling CNICs in a short run (dated from January 1 to February 16, 2021). A significant difference in R(2) was found (P < .001, F = 53.32) in mean R(2) of 0.98, 0.92, and 0.84 for IRT, OE-mm, and QE, respectively. The IRT-based COVID-19 model is superior to the counterparts of QE-m and QE in model R(2) particularly in a longer period of infected days (i.e., in the entire year in 2020). CONCLUSION: An online dashboard was demonstrated to display the association and difference in prediction accuracy among predictive models. The IRT mathematical model was recommended to make projections about the evolution of CNICs for each county/region in future applications, not just limited to the COVID-19 epidemic.

The first infection case of COVID-19 was reported on December 30, 2019, in Wuhan city located in Hubei providence, China. [1, 2] The World Health Organization (WHO) recognized the disease as a pandemic on March 11, 2020. [3] As of September 18, 2021, the world has accumulated >0.228 billion of the COVID-19 cases, threatening people's health, economic development, and social stability. [4, 5] We can observe an exponential growth of the disease based on the daily increment of confirmed cases. [6] [7] [8] 1.1. The need to build a prediction model during the COVID-19 epidemic Based on the increasing COVID-19 data, it is necessary to analyze the evolution of the disease and create a prediction model to observe the trend of confirmed cases. Although many mathematical models [6, [9] [10] [11] [12] [13] [14] [15] [16] [17] have been suggested to predict the number of COVID-19 cases, none compared the prediction accuracy between models in the literature. The difficulty might be attributable to authors who are unfamiliar with algorithms developed by other authors. That is, the author who conducts a comparative study on 2 COVID-19 models should (and must) truly understand both predictive models thoroughly.

Furthermore, children and women of reproductive age might be disproportionately affected by the disruption of routine health services, particularly in low-income and middle-income countries (LMICs). [18] [19] [20] The indirect effects of the COVID-19 pandemic on maternal and child mortality in LMICs should be properly modeled and early estimated using an appropriate way.

One of those mathematical prediction models has applied the quadratic equation (QE) to demonstrate the projected cases in Colombia, and deaths in Russia, India, and worldwide using the past 31-day data up to May 29, 2020 . In which the constrained term is set at the middle point (i.e., P(x 2 ,y 2 )) of the observations in the QE model [6] that assumed the exponential growth during the COVID-19 epidemic. For instance, Colombia and India were increasing, Russia and the worldwide presented decreasing in a growth trend with exponential (i.e., increasing) and logarithmic (i.e., decreasing) phenomena prior to May 29, 2020 .

Because a constrained term was set at the middle point (i.e., P (x 2 ,y 2 )) of the observations to build the QE model, [6] a modified QE model (named QE-m without constraint in the model) was conceived to compare whether the QE-m model earns smaller residuals than the QE model. The evidence is required for verifications.

Another study [9] applied item response theory (IRT) [21, 22] to construct an ogive curve and determine the inflection point (IP) used for predicting the projected cases by country/region based on the cumulative number of infected cases (CNICs). We are motivated to compare the prediction accuracy among these 3 models (i.e., QE, QE-m, and IRT). The model accuracy is referred to R 2 (i.e., explained variance in model = 1residuals/total squared deviations from the mean of observations) across all countries/regions.

The Florez and Singh [6] stated that it is important to analyze the evolution of the disease in order to make decisions that tackle the growth rate of cases. An online dashboard was thus designed and developed in their work (called COVID-19 dashboard [23] ), displaying a projection in order to estimate the future CNICs by country based on the actual observed CNICs. We are thus interested in mimicking the development of an APP for showing the actual CNICs and the projections of the future CNICs on an online dashboard.

To achieve this projection of the growth rate of COVID-19 cases, we designed those 3 mathematical models (i.e., QE, OE-m, and IRT mentioned above) based on equivalently previous CNICs to conduct a comparative study.

Two phases are involved in this study, including comparison of prediction accuracy among the 3 models and application of ipcase-index [9] and angle-index used to design an online dashboard laid on Google Maps.

We downloaded COVID-19 outbreak CNICs between January 1 and Febuary 16, 2021, as well as the entire year in 2020, from GitHub, [24] a site that provides information on new CNICs in countries/regions around the world. All the downloaded data with 299 countries/regions, including the US States and provinces/ metropolitan cities/areas in China, are publicly displayed on the website. [24] Ethical approval is not necessary for this study because all the data are obtained via the Internet (see Supplemental Digital Content 1, http://links.lww.com/MD/G521).

Two prediction models used in this study 2.2.1. The QE model. The QE is described in the left panel of Fig. 1 . To perform the projection for estimating future cases, the QE model refers to a quadratic equation [25] based on data of present-day and previous several days (e.g., 30 days). With the QE, projected cases for the next days up to several days (e.g., 10

Tsai et al. Medicine (2021) 100:50 Medicine days) can be obtained and displayed on a dashboard, similar to that [23] in the previous study. [6] In the QE model, let 3 points P1 = (x 1 , y 1 ), P2 = (x 2 , y 2 ), and P3 = (x 3 , y 3 ) on the trajectory CNICs. As such, y 1 denotes the corresponding number of cases on the day 1 (i.e., x 1 = 0), y 2 corresponds to the CNICs on the middle (e.g., x 2 = 23 if the length of days is 47), and y 3 corresponds to the CNICS on the present day (e.g., Febuary 16, 2021 , in this study and x 3 = 46). The 3 quadratic equations from (1) to (3) are defined at the top-left part in Fig. 1 . The model parameters (i.e., a, b, and c) are derived from the 3 equations; see the calculations at the left part in Fig. 1 . One example of the observed and expected CNICs in the United Kingdom is illustrated in a logarithmic (i.e., decreasing) growth trend at the bottom-right corner in Fig. 1. 2.2.2. The IRT model. The IRT-based COVID-19 model was proposed in the previous study. [9] The probability function is present in Equation (11) , shown at the top-right corner in Fig. 1 . The parameters a and b would be calibrated for each country/ region using Microsoft Solver Add-in. [9] When viewing the 2 curves in Fig. 1 , the different assumptions exist in the 2 models, including the sigmoid (or logistic) function for IRT and the exponential (or logarithmic) growth for QE.

Another QE-m model derived from the QE model. Due to a constrained term was set at the middle point (i.e., P(x 2 ,y 2 ) to create the QE model), [6] the unconstrained QE model (i.e., QE-m) was thus proposed to compare the model accuracy in the 2 QE and IRT models. A hypothesis was made to the QE-m model having a smaller residual than the QE model due to no such constrained condition in the QE-m model. We adopted a similar technique of calibration for model parameters using the Microsoft Solver Add-in tool as the IRT model in the previous study. [9] Three steps were applied to calibrate the model parameter a and b in the QE-m model:

To minimize the total residuals using the Microsoft function below:

where Oi denotes the observed CNIC and Ei represents the expected CNIC via the model equation (i.e.,

), where c = y 1 (=CNIC at day 1) according to Equation (10) in Fig. 1 .

Parameters (i.e., a and b) in Equation (13) are calibrated in the model.

The parameter c is set at y 1 according to the Equation (10) in Fig. 1 .

The observed CNICs were compared with the corresponding expected CNICs using Equation (12) in Step A.

The Microsoft Solver add-in [9] was used to estimate the model parameters (Supplemental Digital Content 1, http:// links.lww.com/MD/G521). The QE projected curve can be plotted to predict the potential CNIC in the future.

Two comparisons were made to evaluate the prediction accuracy in models, including association of R 2 between models using correlation coefficient (CC) and difference in model R 2 using paired-t test. (The t value for CC was calculated by the [26] and analysis of variance (ANOVA).

The R 2 is defined in Equation (13):

where n is the observed days (e.g., 47 in this study), Oj is the observed CNIC at ith day, Ei is the expected CNIC at the ith day, and y-bar denotes the mean observed CNIC in a country/ region. The Kano diagram [27, 28] was used to interpret the association of R 2 between models. Three parts are classified in color, including the one-dimensional category in the middle, the attractive part at the top, and the must-be area at the bottom.

A choropleth map [29] was used to compare the ipcase-index [9] (based on IP days and the corresponding CNIC to represent the negative impact hit by COVID-19, [9] see Equation (14)) for a specific country/region.

Another index named as angle-index is defined in Equation (15):

where both Degrees() and Atan() are derived from the functions in Microsoft Excel, the IP denotes the inflection point [9] on the trajectory of DCNIC (e.g., DIP = 7 days, DCNIC = 27,100-11,177 = 15,934, ratio = 15,934/(7-1) = 2563, u ¼ DEGREESðATANð2653:8ÞÞ ¼ 89:97). The angle-index is from 0 to 90, the higher means ODCOVID is severely hit by COVID-19 in a given country/region.

A visual representation displaying the comparison of prediction accuracy across continents/regions was plotted on the Kano diagram. [27, 28] The projections were drawn using QE and IRT models, respectively, on their dashboards. The forest plot [30] was applied to display the difference in R 2 between the 2 models during the entire year in 2020.

To know the growth trend of CNICs by country, a dashboard displays a projection for future CNICs based on the 2 mathematical models. Figure 2 presents projected cases in Colombia, Russia, India, and Hebei Province in China. Black corresponds to the actual cases collected in data, [21] while the green curve corresponds to the projected CNICs, and the yellow to the predictive CNICs. These projections performed on February 16, 2021, produced the quadratic equations shown in the left panel in Fig. 2 , and other counterparts in the right panel using the IRT model.

It is worth noting that the residual is huge in Hebei Province in China due to the observed CNICs not totally fitted to the QE model when compared with the IRT model in the right panel in Fig. 2 . The reason might be the inflection point a at the earlier stage in China. We can be expected the pandemic situation with the form of ogive curve making the difference clearly in model residuals between the 2 study models. The longer period of infected days would be illustrated in the next section using the data in an entire year of 2020.

We observed that the correlation coefficient was 0.48 (t = 9.87, n = 265) between the 2 QE and IRT models based on R 2 when modeling CNICs dated from January 1 to February 16, 2021. Most bubbles (n = 235) in yellow are within the unidimensional category. In Fig. 3 , a few bubbles in green (n = 19) and red (n = 11) are outside the upper and lower limit lines in the Kano diagram. Bubbles are sized by the R 2 in the QE-m model. The top R 2 are Cuba, Malaysia, Saudi Arabia, and others with R 2 = 1.0. Readers are invited to scan the QR-code in Fig. 3 and click on the bubble of interest to see the respective trend curve on the IRT model. A significant difference in R 2 was found (P < .001, F = 53.32) in mean R 2 of 0.98, 0.92, and 0.84 for IRT, QE-m, and QE, respectively. The IRT-based COVID-19 model is superior to the QE-m and QE in R 2 when modeling COVID-19 for countries/ regions. In Table 1 , we can see that the QE-m model (mean R 2 = 0.92) is also superior to QE model (mean R 2 = 0.84). The hypothesis is supported by the finding that the QE-m model having a smaller residual (or a greater R 2 ) than the QE model. 

Comparisons in R 2 (in the entire year in 2020) are made in Fig. 6 using the mean difference method used in the forest plot. China earns the best-fit than other counterparts in continents and the United States using the IRT model. When comparing the model accuracy in R 2 between the 2 models, the IRT model is superior to the QE model across all continents and the 2 countries in the United States and China. From Fig. 6 , we can see that the pandemic stationarity (e.g., China) is not suitable for the QE model, but for the increasing stage such as in the United States.

All those line plots in Fig. 2 would appear once the bubble in Fig. 3 or the region in Figs. 4 and 5 is clicked using the links. [31] [32] [33] [34] The Fig. 6 is referred to the linkp30. All of these dashboards are laid on Google maps.

In the current study, we observed that the CC was 0.48 (t = 9.87, n = 265) on R 2 between the 2 QE and IRT models when modeling the CNICs for countries/regions. As could be expected, a significant difference in R 2 was found (P < .001, F = 53.32) in mean R 2 of 0.98, 0.92, and 0.84 for IRT, QE-m, and QE, respectively. The IRT-based COVID-19 model has a substantially higher prediction power than the QE-m and QE models, particularly in a longer perfiod of infeted days(e.g., in an entire year). The advanteage of IRT model is obviously due to the ogive curve fitting to the COVID-19 situation better than the OE trend.

An online dashboard on projections of COVID-19 cases was displayed for each county/region using either QE or IRT-based mathematical models once the region of interest on the choropleth map has been clicked. [32, 33] Two effects of model R 2 were found, including the optimization effect (=0.92-0.94 = 0.08 between QE and QE-m models; i.e., minimizing the residual in parameter estimation) and the assumption effect (=0.98-0.92 = 0.06 between QE-m and IRT models; i.e., exhbiting the differece in exponential and logistic growth of the disease outbreak).

COVID-19 has required analyses of data regarding NCICs reported worldwide and by country/region using an online dashboard. [6, 35, 36] Some COVID-19 dashboards developed by prestigious organizations, such as the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU), [4] the WHO, [37] Leszkiewicz personal dashboard, [38] Health-Map, [39] and the dashboard created by Schiffmann who is an 18-year-old high school senior from Washington State in the United States, [40] are full of interest and charm with visualizations for reporting the current state of the COVID-19. Nevertheless, these dashboards lack important information and analysis on making a projection about the evolution of CNICs during the COVID-19 epidemic. As authors Ivankovic´et al [35] stated in their reseach, only a handful of dashboards went further and employed predictive analytics by illustrating different future scenarios of "what could happen." The lack of precision of predictive models and simulations early in the pandemic likely stunted their use. Use of both descriptive and predictive approaches to dashboard design and tighter links between infection control policies and their effects should be further explored into the next phases of the pandemic.

The COVID-19 dashboard [6] was designed to offer additional valuable information (e.g., using a mathematical model to project future cases worldwide and by country). The QE mathematical model is based on the exponential growth of the disease with a QE instead of the sigmoid (or logistic) function with an IRT. [9] Althouhg Liu et al [41] found that the CNICs are well fitted by an exponential function in the 31 provincial-level regions in China, Lynch and Gore [42] in their research found that exponential growth is not the best representation of case growth during early onset. Similarly, Adebowale et al [43] in their study on the spread of COVID-19 outbreak in the first 120 days in Nigeria and 7 other countries found that the cubic polynomial model (CPM) provided the best fit for predicting COVID-19 CNICs www.md-journal.com across all the countries, but there was a clear deviation from the exponential growth model. Accordingly, the exponential growth is not totally suitable for the COVID-19 situation. Another more appropriate assumption for the CNIC trend is required for explorations. A real-time policy dashboard to aid global transparency in response to COVID-19 has been expected to present the localized reasoning behind COVID-19 policy decisions and allow the global health community to provide further support to governments and international stakeholders. [44] As such, dashboards barely presenting real-time descriptions of new daily cases and risk factors are insufficient. [35] A reliable mathematical model used to predict COVID-19 cases is thus important and necessary to the epidemiology and government in the world.

The IRT-based COVID-19 model has been proposed in the previous study. [9] Only 2 model parameters are required to be estimated, as did the QE model. [6] The 2 unique difference between the 2 models (i.e., QE and IRT) are the 2 influential effects of optimization (=0.08) and assumption (=0.06) in model R 2 . Of them, the optimization effect between QE and QE-m models has been shown in Table 1 .

Although a modeling study has been applied to early estimates of the indirect effects of the COVID-19 pandemic on maternal and child mortality in LMICs [18] based on the assumption of severe acute respiratory syndrome at a 23.9% reduction in ambulatory care and a 35.2% reduction in inpatient care observed in Taiwan, [45] the pandemic to affect the health of women and children is expected to yield accurate results through the IRT model along with the MS Excel module as we did in this study.

Two major topics were verified in this study, including whether the IRT-based COVID-19 model is better than the counterparts of QE and QE-m model and how a dashboard that makes projections of COVID-19 cases can be used for epidemiologists and policymakers.

Although numerous mathematical models [6, [9] [10] [11] [12] [13] [14] [15] [16] [17] 46] have been applied to predict the CNICs, none compared the prediction accuracy between (or among) models and further developed an online dashboard to make projections of COVID-19 cases for each county/region as we have illustrated in this study. The difficulty might exist for authors who need to implement and compare the unfamiliar algorithms developed by others in their research. Few article authors are willing to provide their detailed experimental data and algorithms to readers who hope to replicate the study on their own. Nonetheless, relevant studies are encouraged to compare predictive models of COVID-19 or other epidemic crises using our suggested approaches provided in Supplemental Digital Content 1, http://links.lww.com/MD/ G521, including data, codes, MP4 videos, and the MS Excel module, which is rarely seen before in the literature.

Furthermore, >30 articles were searched in PubMed using the search string of "online dashboard." [47] In which, only 9 articles were invloved with the keywords "online dashboard and COVID-19" [48] in title. We demonstrated the online dashboards of COVID-19 in all figures. Not only are the choropleth map and the Kano diagram displayed on dashboards, but the predicted curves are also laid on Google Maps.

The dashboard provided with IP days and the corresponding confirmed cases on the given ogive curve is modern and innovative. Visual displays on CNIC using the IRT model were designed and developed, particularly with the Kano model, to present the relationship between 2 (or more) variables on a diagram. In which, we can see that red bubbles are toward the Xaxis feature, the green toward the Y-axis attribute, and the red meeting the requirement of one-dimensional characteristic.

The 3 main strengths of the current study include the model R 2 was applied to compare the prediction accuracy shown on the Kano diagram, the 2 effects of optimation and assumption were evident to make the model residuals as small as possible using statistical methods (e.g., the Solver Add-in tool in Microsoft Excel), and an app was developed for understanding the growth trend of COVID-19 across countries/regions using the mathematical IRT model embedded in dashboards.

Our study has several limitations. First, many case numbers of COVID-19 are originated from different statistical offices such as the CDC in the United States of America and WHO. This particular agency has provided scientifically highly relevant data since its inception. However, other countries' related agencies may not have such a track record. Hence, the scientific soundness of this database needs to be established and verified further. The data source of this study was from GitHub, [24] which is similar to the use of data in JHU. [4] Fortunately, the JHU dashboard's 5 authoritative data sources include WHO, US CDC, National Health Commission of the People's Republic of China, European Centre for Disease Prevention and Control, and the Chinese online medical resource DXY.cn. The corresponding data repository is accessible as Google sheets in GitHub [49] (similar to ours [24] ).

Second, although we defined the CNIC as the proxy of negative impact hit by COVID-19 on countries/regions using the ipcaseindex in comparison, research is suggested to include deaths to gain the contrast impact by COVID-19 using ipcase-index in the future.

Third, the Microsoft Solver add-in used to estimated IP days in the IRT model is not a unique approach. Many other methods might have better the model R 2 than the IRT model. They are worthy of comparison in the future.

Fourth, visual dashboards are shown on Google Maps. However, these installments are not free of charge. For example, the Google Maps application programming interface (API) requires a paid project key for the cloud platform. Thus, the limitations of the dashboard are that it is not publicly accessible, and it is difficult to mimic by other authors or programmers for use in a short period of time.

Fifth, although IRT is common and popular in the educational and psychometric fields, many readers in public health are unfamiliar with IRT. The IRT model consists of 2 parameters that need some effort to understand through data and MP4 videos provided in Supplemental Digital Content 1, http://links.lww. com/MD/G521. We ensure that readers who are familiar with MS Excel can easily replicate the study on their own in the future.

Sixth, only the period from January 1 to Febuary 16, 2021 was included in this study. The model validation and comparison would be examined further involving other periods of infected days in the future. Finally, the 2 mathematical models [6, 9] combined with the QE-m model were compared in this study. Whether other mathematical models [6, [9] [10] [11] [12] [13] [14] [15] [16] [17] can yield smaller model residuals than the IRT-based COVID-19 model is required for further verification.

Two major topics are covered: confirming the R 2 in IRT model better than the counterparts in the QE and QE-m models and presenting the visual dashboard available for making projections of COVID-19 cases on Google Maps. Two hypotheses are supported: the QE-m model has a lower residual than the QE model, and the QE (or QE-m) model has a higher residual than the IRT model. The IP days can only be calculated using an ogive curve and not quadratic or exponential functions. The ODCOVID can then be measured using the ipcase-index and the angle-index.

It is recommended that other COVID-related mathematical models be used to compare model precision with the IRT model, particularly in a longer period of infected days.

Besides the model R 2 used for measuring model accuracy, using the mean absolute percentage error (MAPE) to calculate the prediction power on the basis of 2 time-frame scenarios (i.e., using training CNICs to predict the testing CNICs) between models (e.g., the residuals between the expected trend and the observed line shown in Fig. 2 ) is expected to be carried out in the future.

An app designed was developed to present the COVID-19 growth trend under the IRT model. An MP4 video in Supplemental Digital Content 1, http://links.lww.com/MD/G521 contains more information about the application that can be clearly learned on the research process in this study. An online dashboard showing future COVID-19 cases for each county/region is recommended for epidemiologists and policymakers in research and practices and not just for the COVID-19 pandemic.

Evolution of the novel coronavirus from the ongoing wuhan outbreak and modeling of its spike protein for risk of human transmission

Immediate psychological responses and associated factors during the initial stage of the 2019 Coronavirus Disease (COVID-19) epidemic among the general population in China

Review of the clinical characteristics of coronavirus disease 2019 (COVID-19)

Dashboard online for COVID-19 in near real time

Spatiotemporal spread pattern of the COVID-19 cases in China

Online dashboard and data analysis approach for assessing COVID-19 case and death data

The basic reproduction number of novel coronavirus (2019-nCoV) estimation based on exponential growth in the early outbreak in China from 2019 to 2020: a reply to Dhungana

Statistical analysis of the impact of environmental temperature on the exponential growth rate of cases infected by COVID-19

Using the Ipcase-Index with inflection points and the corresponding case numbers to identify the impact hit by COVID-19 in China: an observation study

Forecasting COVID-19

Transmission dynamics of the COVID-19 outbreak and effectiveness of government interventions: a data-driven analysis

Nowcasting and forecasting the potential domestic and international spread of the 2019-nCoV outbreak originating in Wuhan, China: a modelling study

Data-based analysis, modelling and forecasting of the COVID-19 outbreak

Modeling the epidemic dynamics and control of COVID-19 outbreak in China

Effect of delay in diagnosis on transmission of COVID-19

A model based study on the dynamics of COVID-19: Prediction and control

Effects of control measures on the dynamics of COVID-19 and double-peak behavior in Spain

Early estimates of the indirect effects of the COVID-19 pandemic on maternal and child mortality in low-income and middle-income countries: a modelling study

The risk to child nutrition during and after COVID-19 pandemic: what to expect and how to respond

Standing together for nutrition consortium. Impacts of COVID-19 on childhood malnutrition and nutrition-related mortality

Practical applications of item characteristic curve theory

Applications of Item Response Theory to Practical Testing Problems

COVID-19 Dashboard

Novel Coronavirus (nCoV) Data Repository

Piecewise rational quadratic interpolation to monotonic data

Predicting the number of article citations on the topic of pemphigus vulgaris with the 100 top-cited articles since 2011: a protocol for systematic review and meta-analysis

Using the Kano model to display the most cited authors and affiliated countries in schizophrenia research

Attractive quality and must-be quality

Choropleth map legend design for visualizing the most influential areas in article citation disparities: a bibliometric study

The use of forest plot to identify article similarity and differences in characteristics between journals using medical subject headings terms: a protocol for bibliometric study

Figure 3 in this study

Figure 4 in this study

Figure 5 in this study

Figure 6 in this study

Features constituting actionable COVID-19 dashboards: descriptive assessment and expert appraisal of 158 public web-based COVID-19 dashboards

An online time-toevent dashboard comparing the effective control of COVID-19 among continents using the inflection point on an ogive curve: observational study

Novel coronavirus (COVID-19) situation (public dashboard)

Dashboard online for COVID-19 in near real time

Novel Coronavirus 2019-nCoV (interactive map)

A creator of one COVID-19 dashboard

The spatial transmission of SARS-CoV-2 in China under the prevention and control measures at the early outbreak

Short-range forecasting of COVID-19 during early onset at county, health district, and state geographic levels using seven methods: comparative forecasting study

The spread of COVID-19 outbreak in the first 120 days: a comparison between Nigeria and seven other countries

A real-time policy dashboard can aid global transparency in the response to coronavirus disease 2019

The impact of the SARS epidemic on the utilization of medical services: SARS and the fear of SARS

Trends and prediction in daily incidence and deaths of COVID-19 in the United States: a search-interest based model

Articles related to online dashboard in Pubmed

Articles related to online dashboard and COVCID-19 in Pubmed

Geographical tracking and mapping of coronavirus disease COVID-19/severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) epidemic and associated events around the world: how 21st century GIS technologies are supporting the global fight against outbreaks and epidemics

The authors thank Enago (www.enago.tw) for the English language review of this manuscript.

Kang-Ting Tsai developed the study concept and design. Yu-Tsen Yeh and Ju-Kuo Lin analyzed and interpreted the data. Willy Chou monitored the process of this study and helped in responding to the reviewers' advice and comments. Tsair-Wei Chien drafted the manuscript, and all authors provided critical revisions for important intellectual content. The study was supervised by Willy Chou. All authors read and approved the final manuscript.