key: cord-0722730-r8infdtp
authors: Kamar, A.; Maalouf, N.; Hitti, E.; El Eid, G.; Isma'eel, H.; Elhajj, I. H.
title: Challenge of forecasting demand of medical resources and supplies during a pandemic: A comparative evaluation of three surge calculators for COVID-19
date: 2021-02-03
journal: Epidemiol Infect
DOI: 10.1017/s095026882100025x
sha: 1fe33d19868da30f83b1c630ab58d1c933679d68
doc_id: 722730
cord_uid: r8infdtp

Ever since the World Health Organization (WHO) declared the new coronavirus disease 2019 (COVID-19) as a pandemic, there has been a public health debate concerning medical resources and supplies including hospital beds, intensive care units (ICU), ventilators and protective personal equipment (PPE). Forecasting COVID-19 dissemination has played a key role in informing healthcare professionals and governments on how to manage overburdened healthcare systems. However, forecasting during the pandemic remained challenging and sometimes highly controversial. Here, we highlight this challenge by performing a comparative evaluation for the estimations obtained from three COVID-19 surge calculators under different social distancing approaches, taking Lebanon as a case study. Despite discrepancies in estimations, the three surge calculators used herein agree that there will be a relative shortage in the capacity of medical resources and a significant surge in PPE demand if the social distancing policy is removed. Our results underscore the importance of implementing containment interventions including social distancing in alleviating the demand for medical care during the COVID-19 pandemic in the absence of any medication or vaccine. The paper also highlights the value of employing several models in surge planning.

The outbreak of the new coronavirus disease 2019 , caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), started in Wuhan City, China, and has rapidly spread all over the world. The newly identified coronavirus has 96% homology with the genetic sequence of β-coronaviruses; which include SARS-CoV and the Middle-East respiratory syndrome CoV (MERS-CoV) [1, 2] . Despite similarities, SARS-CoV infection was limited to specific geographic areas, while COVID-19 has been declared as a global pandemic by the WHO on 11 March 2020. As of 17 September 2020, there are 29 737 453 COVID-19 confirmed cases, 937 391 deaths and around 213 countries or territories affected with cases around the world [3] .

COVID-19 has not only caused mortality, but has alsoput tremendous pressure on healthcare systems and led to shortages in personal protective equipment (PPE) [4, 5] . In the Middle East and North Africa Region (MENA) countries including Lebanon, the COVID-19 pandemic is leading to a health crisis and worsening the spike in demand for healthcare resources and supplies [6] . After detecting the first COVID-19 case in Lebanon, authorities took fast action and declared an emergency state in the country. Therefore, mitigation measures have been taken into effect early in March, while gradually being lifted as of 10 May. Yet, the activities in the airport were restarted at one-fifth capacity as of 1 July 2020 [7] . This fast action helped in the initial containment of the disease [8] . Unexpectedly, after the massive Beirut's port explosion on 4 August that caused hundreds of deaths, thousands of injuries and destroyed half of the city's medical centres, the country witnessed a surge in COVID-19 infections. This urged the Lebanese authorities to impose a partial lockdown which included the closing of some businesses, banning public gatherings and enforcing midnight curfew [7] . However, the two-week lockdown was prematurely eased due to the economic crisis [6] . At the time of writing this paper, despite the spike in COVID-19 cases (a total of 169 472 confirmed cases and 1379 deaths as of 27 December 2020) [9] , mitigation policies such as the lockdown, travel ban and the stay-at-home have been lifted in Lebanon while keeping social distancing and personal protective measures.

To monitor the strain on healthcare systems in each country, it is essential to closely follow the demand for hospital beds, intensive care unit (ICU) beds and ventilators [10] . Therefore, a growing number of models have been established around the world to aid in forecasting COVID-19 deaths, cases and the demand for medical supply (including ventilators, hospital beds and ICU beds, the timing of patient surges and more) [11] [12] [13] . While simple models apply the user's inputs on the local population and current status of COVID-19, more sophisticated models permit the user to modify other parameters (e.g. social distancing changes) that affect the trends. Some of the models match projected cases to existing capacity to project when and where a caseload surge will surpass capacity [14] . Data-driven modelling approaches have also appeared for the COVID-19 pandemic, in which statistical and machinelearning models are used for projecting cases, hospitalisation, deaths and the impact of social distancing [15] . Unfortunately, forecasting what is most likely to occur in the upcoming weeks during the COVID-19 pandemic is not available for all countries including some European countries or few states in the USA [11] . Where available, different models are providing widely varying numbers of needed medical resources and/or supplies which often lead to an incorrect distribution of what is available due to inconsistency in numbers. For example, the Centers for Disease Control and Prevention has reported that COVID-19 outbreaks in parts of the USA have resulted in surges in hospitalisations and ICU patients [16] . However, providing accurate predictions of the healthcare system capacity peak demand is controversial due to the scarcity and/or unreliability of data in addition to challenges associated with forecasting the effects of the rapid changes in mitigation policies [17] . So far, the efforts to accurately model any emerging outbreak's trajectory for the upcoming days are limited due to variabilities in assumptions and parameters including social distancing [17] [18] [19] . Accordingly, the use of a single forecasting model may not precisely predict how the pandemic evolves [20] .

Regardless of all the challenges, COVID-19 has put forecasting at the top of global public policymaking and developing effective preventive strategies [21, 22] . Since there is no 'gold standard' for predicting thresholds, the reasonable evaluation of the outputs of various forecasting models has remained an open question [23] . In this study, we highlight this challenge by comparing the projected demand for medical resources and supplies from three surge calculators for COVID-19 taking Lebanon as our case study. To this end, we adapted an available statistical model for estimating the daily impact of COVID-19 on hospital services based on the COVID-19 Hospital Impact Model for Epidemics (CHIME). The model was modified to incorporate longer projection periods and different social distancing policies (%). We then compared the hospital beds, ICU beds and PPE demand over 200 days projected by the three surge calculators (a) CHIME PPE calculator, (b) WHO COVID-19 Essential Supplies Forecasting Tool (COVID-19 ESFT) and (c) our own developed American University of Beirut Medical Center (COVID-19 AUBMC Surge needs) calculator. The results of these calculators differ depending on the parameters and assumptions implemented within to generate the forecasted data. Despite the discrepancies in estimations, the three surge calculators used herein consistently agree on trends demonstrating that social distancing policy can help reduce the demand for medical resources and supplies amid the COVID-19 pandemic.

Case counts, population and hospital capacity data We used the WHO and the Lebanese Ministry of Public Health (MoPH) websites to identify data on the estimated Lebanese population in 2020 (6 825 000) and confirmed COVID-19-infected individuals and death cases, respectively. Data on inpatient beds, ICU capacity and mechanical ventilators were obtained from the Lebanese MoPH website ( Table 1 ). The data for the COVID-19 AUBMC calculator regarding PPE demand were collected based on historical and current data estimates 

Our estimations were based on the CHIME model that was initially developed by the Predictive Health at Penn Medicine in the USA [24] , which permits healthcare systems to enter data about their population and modify the assumptions about the COVID-19 spread and behaviour [24] . [25, 26] . The CHIME model was built to help hospital systems to accurately project the needed resources during the COVID-19 pandemic, mainly hospital beds, ICU beds and ventilators [27] . The model accounts for four main parameter categories: (1) the hospital parameters, (2) spread and contact parameter, (3) severity parameters and (4) display parameters. The model allows integrating some information about social distancing policies that are emplaced by governments [27] .

In this study, we have introduced some modifications to the CHIME application to make it compatible with the input parameters available for the projections in Lebanon.

The modifications were threefold:

1. Automate the data projection process:

The application was updated allowing the user to enter the input parameters into an 'Excel workbook' and obtain the generated census compiled together in the same workbook. This alleviates the data processing part where the user needs to obtain each projection separately. It also enables faster simulations for different input parameters and most importantly different social distancing measures ( Table 2 ).

The application initially allowed for a maximum 30-day projection period. Our contribution enables for longer periods such as 200 days, as seen herein. This provides analysts and healthcare workers (HCW) with longer forecasts, thus giving them more time to prepare for periods of peak demand.

The CHIME projects the daily and the cumulative number of hospitalised, ICU, ventilated and newly admitted COVID-19 cases, with social distancing policy percentage being the variable.

The modification allows us to study the number of daily infected individuals after deducting the number of recovered cases. The number reflects the confirmed cases and does not consider possible infected individuals that are not confirmed yet. The aim behind this modification is to provide these numbers as inputs to the calculators, especially COVID-19 AUBMC and WHO COVID-19 ESFT calculators.

Our projections were based on three social distancing policy scenarios (0%, 30% and 50%) based on the National Health Strategic Preparedness and Response Plan for COVID-19 pandemic lock-down management and exit strategy implemented by the Lebanese Government [6] . The 0% social distancing scenario refers to when all restrictions and interventions are lifted without any action taken by the Government. The 30% social distancing scenario represents relaxing social distancing measures to obtain only a 30% reduction in contact rate through (1) re-opening of businesses with the public advocacy of social distancing and improved hygiene, (2) re-opening of the airport and borders, (3) partial opening of some public and private schools, (4) a nationwide overnight curfew between 9:00 p.m. and 5 a.m. (local time) and (5) banning events over 100 persons. The 50% social distancing scenario refers to a 50% reduction in contact rates where the following actions were taken: (1) imposing lockdown measure to stem the spread of the COVID-19 pandemic, (2) school and non-essential businesses closure, with individuals permitted to leave their homes wearing facemasks to buy food and medicine, (3) a nationwide travel ban, (4) a nationwide overnight curfew between 7:00 p.m. and 5 a.m. (local time), (5) and ban gatherings over 50 people, (6) and all individuals over the age of 65 are required to remain at home [28, 29] .

We also simulated the strict social distancing policy (92%) based on the Wuhan-style containment [28] . The latter was performed to assess how strict 'lock-down' could help better contain the COVID-19 outbreak and maintain the demand for hospital beds, ICU beds, ventilators and PPE. The Wuhan-style represents the strict social distancing measures taken through the lockdown of Wuhan City, Hubei province in China during the COVID-19 pandemic [28] . This style reflects a 92% reduction in contact rates for 12 consecutive weeks to fully contain the disease through suspending flights, trains and blocking roads in addition to the rigorous implementation of public health measures [28, 30, 31] .

Although Lebanon started re-opening gradually with limited capacity as of 4 May 2020, the country went again into a complete four-day lockdown starting 14 May. Therefore, we ran our estimations on 14 May 2020 (day 0) assuming the four social distancing policy scenarios. The other social distancing policy scenarios and the mitigation dates below were chosen as the per five-step re-opening plan implemented by the Lebanese Government [6] ( Table 2) .

We have used the WHO COVID-19 ESFT calculator (WHO 2020a) and the COVID-19 AUBMC surge needs calculator, to estimate daily and total hospital beds, ICU beds, ventilators and PPE demand. The results obtained from the CHIME PPE calculator were then compared to those derived from the WHO COVID-19 ESFT and COVID-19 AUBMC surge needs calculators. As the COVID-19 AUBMC surge needs calculator was developed by our team, we chose the CHIME PPE calculator because it is compatible with the CHIME model, and the WHO COVID-19 ESFT calculator because it is widely used as a worldwide reference to estimate medical resources and supplies to respond to the current COVID-19 pandemic. Also, the CHIME and COVID-19 ESFT calculators are freely available online for use by governments, stakeholders and healthcare centres.

The CHIME PPE calculator The CHIME PPE calculator was generated to work in parallel with CHIME-generated projections. The calculator uses forecasted patient censuses to output daily and cumulative projections for each type of PPE (including N95 masks, surgical masks, gloves, gown and eye disposable protection) quantities per day and computes the cumulative PPE predictions [32] . This tool also permits users to input their custom scenarios (standard, crisis, contingency and custom), tailored to the specific situations relevant to their hospital or healthcare system [32] . In our study, we chose the values for the standard scenario assuming we do not have exact publicly available data estimates of the exact number of staff and HCW in Lebanon. As social distancing policies and gradual lifting of restrictions are always the keep factors being debated in the intervention strategies, our estimations were based on the four social distancing policy scenarios mentioned above ( Table 2) . For each scenario, the projected hospitalised, ICU and ventilated censuses are inserted into the CHIME PPE calculator in addition to the daily admissions. These data are obtained from the Penn CHIME application model. Based on the provided data the calculator outputs the PPE predictions per day in addition to the total cumulative predictions over the chosen period of 200 days. Since other calculators do not forecast the daily PPE demand, we compared the average daily PPE demand from this calculator with the predictions of other calculators.

The 2020 WHO COVID-19 ESTF calculator is developed to aid countries in estimating potential needs and supplies to respond to the COVID-19 pandemic [33] . The tool is not meant to be used as an epidemiological model, yet it has simple exponential growth and SIR case forecast options built-in [34] . The calculator is a supply forecasting tool that helps in estimating potential requirements for essential supplies including PPE (e.g. surgical masks, gloves, gowns, goggles, respirators and face shields), biomedical equipment for case management (e.g. mechanical ventilators and oxygen concentrators), drugs for supportive treatment, hygiene and IPC commodities, diagnostics and consumable medical supplies. The calculator also estimates the weekly number of COVID-19 patients classified according to severity as follows: mild, moderate, severe and critical. Severe and critical cases are admitted to ICU and require oxygen and ventilation, respectively. Inpatient beds in this calculator refer to ICU beds occupied by the critical and severe COVID-19 cases per week and not day. Therefore, the estimated peak for inpatient and ICU beds will be the same throughout the chosen period.

Although the tool is suited for projections over a short period (12 weeks), it offers an option to enter data manually and make projections over longer periods. The WHO COVID-19-ESFT does not quantify or account for resources already available locally or those pending delivery. The calculator projects the PPE quantity per person per day for inpatient care, cases in isolation, screening and laboratory and the total daily costs (USD) of items over 28 weeks. Then, it adds the total quantity for each per day. In this study, we use the default input parameters set by the WHO COVID-19 ESFT calculator for Lebanon, including the population estimate, patients' case sensitivity, HCW and staff. To maintain consistency across the different calculators, we manually input the cumulative projected COVID-19 cases as obtained from the CHIME model application. The data are compiled in a weekly form (up to 28 weeks) to be compatible with the COVID-19 ESTF calculator. This approach guarantees that all calculators are using the same numbers for infections and the focus would be on the discrepancies/agreements between the models on the estimates of resource demand.

The COVID-19 AUBMC calculator (https://www.aub.edu.lb/fm/ vmp/Pages/calculators.aspx) was developed by our team based on AUBMC and MoPH data and is implemented as an Excel file that predicts the average total of PPE needs (e.g., gloves, surgical face masks, face shields and N95 masks) per day. The estimations are based on the average number of patients admitted and tested individuals per day using actual PPE demand data collected from AUBMC. Note that we assume AUBMC policies of PPE use (such as wearing surgical masks over N95 masks, replacing surgical masks per patient and replacing N95 masks per shift) are representative of policies at the national level. The average number of admitted patients is calculated from the Penn CHIME model application projections over the whole pandemic period (200 days) using the hospitalisation percentage of simultaneously infected individuals. The average number of tested individuals is calculated by dividing the total number of tests done as indicated in (https://www.worldometers.info/coronavirus/? #countries) by the number of pandemic days. The PPE estimations in this calculator are based on average values as compared to daily/weekly projections used in the other two calculators. This calculator also projects the total number of ventilators, hospital beds and ICU beds occupied during the COVID-19 pandemic using the same inputs of the CHIME application. The calculations of the beds and ventilators are designed to estimate the availability during peak periods of the pandemic. The calculator takes as input the currently available ventilators and beds dedicated to COVID-19 patients as declared by MoPH in addition to the peak simultaneous infections and admissions as projected by the Penn CHIME model application. This information helps in forecasting possible shortages in beds or ventilators and the dates when these shortages would occur ( Table 3) .

The CHIME application is based on the SIR model famous across pandemic forecasts. This application is used to provide the common forecast data used across all the calculators presented in this paper. The AUBMC calculator estimates the average PPE needs per day for admitted and tested patients as well as the total PPE needs across the chosen pandemic period. The estimate considers the equipment needed for three categories of patients: (1) on admission, (2) in the operating room and (3) ICU patients. The equipment needed per category per patient is estimated based on data collected by AUBMC and protocols followed by AUBMC medical staff. These estimates are assumed to be representative of the national level. The inpatient beds, ICU beds and ventilators 

Based on MoPH data, from 29 March 2020, Lebanon is estimated to have 2308 ICU beds and 11 794 inpatient (hospital) beds. We assume that 32% of ICU beds and 20% of hospital beds at the national level are currently occupied by non-COVID-19 patients (Table 4 ). These assumptions were based on calculating the total number of beds occupied per day for the last 6 months, yet taking a percentage below AUBMC occupancy for the national level. However, this occupancy number is, in reality, higher due to the casualties from the devastating port explosion in Beirut on 4 August 2020. Also, there has been a surge in cases nationally. One reason behind this surge is that many people have been unable to follow precautionary measures, such as social distancing, during the relief efforts in Beirut. Also, some of the major hospitals have been partially or heavily damaged as a result of the huge blast at Beirut's port [35, 36] . MoPH reported that almost all COVID-19 beds are full and hospitals are running out of space for new patients by the time of writing this paper. This incident highlights the challenge of planning during a pandemic, particularly when coupled with unexpected disasters.

In one scenario and based on our data, assuming strict social distancing policy or the Wuhan-style (92%), the modified CHIME model estimates that out of the remaining 9435 hospital beds, a peak of 67 inpatient beds is needed by the COVID-19 cases or 0.7% of the available hospital beds. Of the remaining available 1569 ICU beds and 687 ventilators, the model estimates a peak of 15 (1%) ICU beds and 10 ventilators (1.5%) are needed by COVID-19 patients respectively, with a decreasing pattern in the estimated COVID-19 infected patients over the projected period. Assuming the same scenario, the WHO COVID-19 ESFT calculator estimates a weekly peak of 11 inpatient beds (0.1%), 17 ICU beds (1.1%) and 9 ventilators (1.3%) are occupied. The COVID-19 AUBMC calculator estimates a different peak of 71 inpatient beds (0.8%), 18 ICU beds (1.1%) and 11 ventilators (1.6%) (Figs 1a, 2 and 3) . Upon relaxing social distancing measures to 50%, the CHIME model estimates the same peak capacity for the available inpatient beds, ICU beds and ventilators. Still, the COVID-19 AUBMC calculator differentially estimates that the peak of daily occupied inpatient beds increases to 94 (1%), while ICU beds and ventilators increase to 23, representing 1.47% and 3.35% of the available ICU beds and ventilators, respectively. On the other hand, the COVID-19 ESFT estimates an increase in the weekly peak demand of inpatient bed capacity to 1279 (13.75%), the ICU beds to 58 (3.7%) and the ventilators to 16 (1.31%) (Figs 1b, 2 and 3) .

When the social distancing policy is reduced to 30%, all the surge calculators variably show that the capacity of the daily peak of hospital beds and ventilators occupied by COVID-19 patients significantly increases. The CHIME estimates a daily peak of 4016 inpatient beds (43%), 912 ICU beds (58%) and 602 ventilators (88%) to be occupied. With a higher occupancy rate, the COVID-19 AUBMC calculator estimates a peak of 4896 inpatient beds (52%), 1224 ICU beds (78%) and 1224 ventilators. The latter indicates a shortage of 537 ventilators for COVID-19 patients in this scenario. On the contrary, the WHO COVID-19 ESFT calculator estimates a marked increase in the weekly peak of inpatient beds (2665) and ventilators (266) but a shortage of 1096 ICU beds (Figs 1c, 2 and 3) . To evaluate to which extent removing social distancing practices strain hospital capacity, we ran our estimations assuming 0% social distancing. Our results show that in addition to the variability in projections, all calculators estimated that there will be a shortage in inpatient and ICU bed capacity during the 200 days. Yet, only the WHO COVID-19 ESFT calculator estimates that ventilators available can accommodate the critical COVID-19 patients needing ventilation. The CHIME model estimates that the daily peak capacity significantly increases to 113 356 inpatient beds, 25 528 ICU beds and 17 003 ventilators. Similarly, this increase was marked by the COVID-19 AUBMC calculator that estimated a peak daily capacity of 124 658 inpatient beds, 33 696 ICU beds and 36 834 ventilators. The WHO COVID-19 ESFT calculator forecasted a daily peak capacity of 11 876 inpatient beds, 11 876 ICU beds and 657 ventilators (Figs 1d, 2 and 3) .

We also compared the projected demand for different PPE types (including N95, surgical masks, gloves, gowns, disposable eye protection/goggles and face shield) by the three surge calculators over 200 days assuming the same social distancing policy scenarios (%). The CHIME calculator estimates a minimum average daily demand of various PPE types when the Wuhan style (92%) policy is applied. The projected outputs include an average daily demand of 689 N95 masks, 1239 surgical masks and 13 086 gloves, gowns and disposable eye protection. When compared to the CHIME projections, the COVID-19 AUBMC calculator estimates a slightly more daily average demand of 817 N95 masks, however, it estimates an average daily peak demand of 11 516 surgical masks, 57 842 gloves and 11 811 gowns and 11 429 face shields. Interestingly, the WHO COVID-19 ESFT calculator estimates a significantly higher average daily demand values for all PPE types. The estimated average daily demand includes 75 082 surgical masks, 159 488 gloves, 42 331 gowns, 36 941 goggles and 160 face shields (Fig. 4a) . The estimated average daily demand for gloves in the WHO COVID-19 ESFT calculator refers (Fig. 4d) .

We then compared the change in the forecasted PPE folddemand from other scenarios with the Wuhan style scenario (92%) since we have the least contact rate, minimum COVID-19 cases and minimal projected hospital utilisation and PPE demands. Therefore, when we changed social policy to the CHIME estimates a significant increase in the average daily demand to 3000 folds for all PPE types, the COVID-19 AUBMC and the WHO COVID-19 ESFT forecast around 100and 200-fold increase in the average daily demand for all PPE types, respectively (Fig. 5 ).

In this study, our results indicate that there is a discrepancy in the forecasted data between the three surge calculators despite the use of identical input data. Yet, the three surge calculators used consistently show that relaxing social distancing policy and mitigation measures can dramatically overwhelm hospital capacity and lead to a dramatic surge in the daily PPE demands during the COVID-19 pandemic. These results are congruent with recommendations for implementing initiatives to 'flatten the curve' and avoid healthcare systems from being overwhelmed through the exponential growth of the disease [37] . Since prompting social distancing policy measures have helped in reducing the pandemic's spread as shown by COVID-19 mathematical models [38, 39] , our comparison was more based on testing the efficacy of this approach on the healthcare infrastructure capacity and PPE demand using the three aforementioned surge calculators.

When assuming the strict social distancing measures (Wuhan-style), the three surge calculators projected an unpronounced daily demand for hospital beds, ventilators and PPE during the pandemic. The variability in estimations between the calculators revealed the least margins of error given this scenario. When relaxing the social distancing measures to 50%, the calculators forecasted that the available hospital beds, ICU beds and ventilators are enough to accommodate the projected COVID-19 cases, yet with a very noticeable increase in the average daily PPE demand. By decreasing the social distancing policy to 30%, the calculators estimate that there is a substantial surge in the demand for PPE and hospital utilisation. At this stage, the difference becomes more apparent between the forecasted data. However, when the simulated social distancing measures are completely removed, all calculators show to some extent that the COVID-19 pandemic will put unprecedented strain on inpatient beds, ICU beds, ventilators and there will be a drastic increase in the average daily PPE demands. This underscores the urgency of implementing social distancing to help in limiting COVID-19 community transmission by reducing the circle of social contacts and the contact rate [40] . Our results are similar to those of the IHME team and Murray who forecasted that the healthcare system's capacity will be stressed in the USA and Europe during the COVID-19 pandemic and proposed some measures to increase the supply of key products and services [11, 12] . Interestingly, the data presented in this study indicate that the output predictions of all used surge calculators vary very widely upon relaxing the social distancing policy measures leading to a rise in the margins of agreement.

The discrepancy in the obtained results could be related to several reasons. For PPE, some of the items are subdivided into different categories such as the types of gloves, gowns and masks used by patients and staff as in the WHO COVID-19 ESFT calculator. Also, the WHO COVID-19 ESFT calculator gives a more detailed quantification for PPE, for example, the estimation for gloves refers to the sum of those used for surgery, examination (Table 4) are not used in the COVID-19 AUBMC surge needs calculator, and were only considered to estimate the available ICU and inpatient beds for COVID-19 patients. Therefore, those assumptions do not pose a limitation on the comparison in terms of relative numbers between the models, but can only affect the analysis of when an overload would occur. Also, our study is relative because the availability of beds was not monitored closely, and Lebanon has been provisioning additional beds especially after the Port explosion where several field hospitals were established. Since there is no absolute truth that we can compare to assess the accuracy, we only resort to the relative comparison. Nevertheless, this wide variability in estimations can add more uncertainty to the forecasted hospital utilisation and PPE demand and highlights the importance of maintaining social distancing in the absence of pharmaceutical intervention for COVID-19.

To be able to evaluate the surge calculators objectively, the results produced should be compared with their real known values. However, parameters used in models for forecasting the dissemination of infectious diseases are prone to uncertainties and limitations [41] . Besides, enhancing the current model forecasting abilities is directly proportional to the accuracy of the data provided [41] . In our case, we input the cumulative projected COVID-19 cases as obtained from the modified CHIME model application into all calculators to maintain consistency. Yet, in addition to using already built models, we still lack accuracy as we do not have tangible data in Lebanon on PPE consumption and the capacity of healthcare system infrastructure. Also, we are not very sure how much our input data are up-to-date and reliable especially that the healthcare capacity is subjected to change throughout an outbreak [41] . Aiming for a relatively long forecasting period (200 days) directly affects the uncertainty of predictions. While the CHIME application developers do not provide a quantitative measure of the uncertainty across the forecast period, they do mention that the projections beyond 30 days will hold higher uncertainties [42] . This is also evident in a recent study which shows that uncertainties increase when forecasting beyond 30 days [43] . The uncertainty is a property of the SIR model used by the CHIME tool developers to calculate the infection projections. These numbers are used as inputs for all surge calculators being compared. Accordingly, the effect of the uncertainty in the projections weighs equally on all calculators being evaluated and reduces the impact on the outcomes, which are relative in nature.

If we were to assess the accuracy of the CHIME forecasts and in turn the accuracy of the presented calculators, we would have to compare the forecasts with the actual national-level data. However, this is hindered by three factors: (1) the limitation of the CHIME application where it can only account for one social distancing scenario per forecasting period, (2) the fluctuating social distancing measures taken by the Lebanese government over the pandemic period which renders the true infection data incomparable with the CHIME forecasts and (3) the lack of actual national-level data on the consumptions and occupancies. It is important to note that the scope of this paper is to evaluate inconsistencies across surge calculators for the same infection forecasts. Future work is planned to evaluate the calculators over shorter periods where a consistent social distancing scenario is maintained and where some national-level data can be obtained. These data will most likely be related to the levels of occupancies of ICU and hospital beds. The results of this study were based on showing the differences in estimations done by three surge calculators for COVID-19 and a modified version of the CHIME model to measure needs for different health system resources based on the total of predicted simultaneously active cases. These calculations were carried out based on three important assets receiving significant attention worldwide: inpatient beds, ICU beds, ventilators and PPE [44, 45] . Other items and human resources required in the diagnostic and treatment chain can be forecasted including staff and HCW in the frontline of COVID-19 response, and therapeutics for supportive treatment. We believe that one of the major limitations for forecasting COVID-19 is based on the limited evidence since neither the magnitude nor duration of the COVID-19 wave is known with certainty. Another limitation is that we did not factor in PPE re-use measures such as sterilisation of used N95 masks. Therefore, the adequate management of medical resources (including PPE, beds, ventilators and healthcare providers) is highly recommended at this stage as some countries have started to experience a resurgence of COVID-19 cases as the pandemic continues to accelerate. Experience from several countries including China, South Korea and Singapore in addition to mathematical modelling has revealed that the pandemic can be contained even in the exponential growth phase using a combination of interventions [40, 46] . The latter mainly includes social and physical distancing, public awareness and wearing masks [40] .

In conclusion, the surge calculators used here, regardless of the variability in outputs, can be powerful tools for measuring the impact of social distancing policy through highlighting the dangers of scaling down non-pharmaceutical public and social health measures in the absence of any vaccine or therapy against COVID-19. Characterising forecasting uncertainty can be improved by some promising avenues including methodological advances in model comparison and averaging [41] . Thus, the use of more than one model is recommended to generate more accurate and better predictions of the pandemic's evolution [47] . In other words, policymakers can use these calculators interconnected with each other based on the available data for each country to understand the pandemic from all its angles to be able to generate policymaking frameworks. This urges the need for a clear methodology that allows policymakers to decide which model is more applicable or adaptable for their context [20] and underscores the necessity of enabling calculators to be adopted to local policies and behaviours beyond social distancing. Although gaps in the present data streams provide a challenge for the current epidemic forecasting, recent breakthroughs in this field afford the possibility for refining future predictive models [41] . Since the most essential piece of the puzzle in forecasting is data or the quality of data source, we suggest that should the data be more accurate, one can provide pandemic forecasting with fewer constraints.

The novel coronavirus (SARS-CoV-2) pandemic

Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Wuhan

World Health Organisation (2020) WHO Coronavirus Disease (COVID-19) Dashboard

Facial protection for healthcare workers during pandemics: a scoping review

The COVID-19 pandemic in Brazil: analysis of supply and demand of hospital and ICU beds and mechanical ventilators under different scenarios

COVID-19 crisis response in MENA countries

Forecasting the outbreak of COVID-19 in Lebanon. medRxiv

The Lebanese COVID-19 cohort; A challenge for the ABO blood group system

Lebanon: WHO Coronavirus Disease (COVID-19) Dashboard

Pressure on Healthcare Systems: Coping with Demand for ICU and Hospital Beds

IHME COVID-19 health service utilization forecasting Team, Murray CJ (2020) Forecasting the impact of the first wave of the COVID-19 pandemic on hospital demand and deaths for the USA and European economic area countries

Forecasting COVID-19 impact on hospital bed-days, ICU-days, ventilator-days and deaths by US state in the next 4 months

Real-time tracking and forecasting of the COVID-19 outbreak in Kuwait: a mathematical modeling study. medRxiv

COVID-19 Models: Forecasting the Pandemic ' s Spread

Curating a COVID-19 data repository and forecasting county-level death counts in the United States

COVIDView: A Weekly Surveillance Summary of U.S. COVID-19 Activity CDC

Disease and healthcare burden of COVID-19 in the United States

Where the U.S. Stands Now on coronavirus testing -The New York Times [internet]. The New York Times

Pandemic Politics: Timing State-Level Social Distancing Responses to COVID-19

Forecasting and planning during a pandemic: COVID-19 growth rates, supply chain disruptions, and governmental decisions

The challenges of modeling and forecasting the spread of COVID-19

Forecasting COVID-19

A framework for evaluating epidemic forecasts

The Model -CHIME

SIR Model

Kermack−McKendrick Model

Model Parameters-CHIME

Covid act now model reference/assumptions how the model works

COVID-19 response in Lebanon current experience and challenges in a low-resource setting

To tame coronavirus, mao-style social control blankets China

What China's coronavirus the rest of the world

Introducing a PPE calculator for use with CHIME

Essential resource planning [Internet] (2020) World Health Organisation

World Health Organisation (2020) WHO COVID-19 Essential Supplies Forecasting Tool (COVID-ESFT) v1.1 [Internet

The 2020 blast in the port of Beirut: can the Lebanese health system "build back better

2020) Shaken, The Beirut blast left Lebanon's health system badly

Why outbreaks like coronavirus spread exponentially, and how to "flatten the curve" -Washington Post [Internet]. The Washington Post

Social distancing strategies for curbing the COVID-19 epidemic

First-wave COVID-19 transmissibility and severity in China outside Hubei after control measures, and second-wave scenario planning: a modelling impact assessment

Can the COVID-19 pandemic still be suppressed? Putting essential pieces together

Real-time epidemic forecasting: challenges and opportunities. Health Security

Penn Medicine (2020) CHIME app

A relationship between SIR model and generalized logistic distribution with applications to SARS and COVID-19

World Health Organisation (2020) Shortage of personal protective equipment endangering health workers worldwide

COVID-19: how to prepare for the pandemic?

Report 9 -impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand

Horses for Courses' in demand forecasting

Author contribution. AK and NM have contributed to the design and implementation of the research, to the analysis of the results and the writing of the paper. EH and GE contributed to data collection. IE and HI conceived the study and were in charge of overall direction and planning. All authors discussed the results and contributed to the final paper.Financial support. None.

Data availability statement. Requests for access to the data that support this study should be made to the corresponding authors: Dr Hussain Isma'eel and Dr Imad Elhajj.