key: cord-0690065-49w0vsfx authors: Ramgopal, Sriram; Pelletier, Jonathan H.; Rakkar, Jaskaran; Horvat, Christopher M. title: Forecast modeling to identify changes in pediatric emergency department utilization during the COVID-19 pandemic date: 2021-05-21 journal: Am J Emerg Med DOI: 10.1016/j.ajem.2021.05.047 sha: b96e924570eade668e53d3e7dcf497709a776a21 doc_id: 690065 cord_uid: 49w0vsfx OBJECTIVE: To identify trends in pediatric emergency department (ED) utilization following the COVID-19 pandemic. METHODS: We performed a cross-sectional study from 37 geographically diverse US children's hospitals. We included ED encounters between January 1, 2010 and December 31, 2020, transformed into time-series data. We constructed ensemble forecasting models of the most common presenting diagnoses and the most common diagnoses leading to admission, using data from 2010 through 2019. We then compared the most common presenting diagnoses and the most common diagnoses leading to admission in 2020 to the forecasts. RESULTS: 29,787,815 encounters were included, of which 1,913,085 (6.4%) occurred during 2020. ED encounters during 2020 were lower compared to prior years, with a 65.1% decrease in April relative to 2010–2019. In forecasting models, encounters for depression and diabetic ketoacidosis remained within the 95% confidence interval [CI]; fever, bronchiolitis, hyperbilirubinemia, skin/subcutaneous infections and seizures occurred within the 80–95% CI during the portions of 2020, and all other diagnoses (abdominal pain, otitis media, asthma, pneumonia, trauma, upper respiratory tract infections, and urinary tract infections) occurred below the predicted 95% CI. CONCLUSION: Pediatric ED utilization has remained low following the COVID-19 pandemic, and below forecasted utilization for most diagnoses. Nearly all conditions demonstrated substantial declines below forecasted rates from the prior decade and which persisted through the end of the year. Some declines in non-communicable diseases may represent unmet healthcare needs among children. Further study is warranted to understand the impact of policies aimed at curbing pandemic disease on children. Forecast modeling to identify changes in pediatric emergency department utilization during the COVID-19 pandemic The first United States (US) case of the SARS-CoV-2 (COVID- 19) was reported in January 2020 [1] , with a pandemic declared in March 2020 [2] . Though COVID-19 is less severe in children than in adults [3] , social distancing, stay at home orders, and mask mandates are thought to have substantially changed rates of presentation of many pediatric diseases [4] [5] [6] [7] [8] [9] . Additionally, the pandemic has led to changes in the delivery of pediatric healthcare in the US, with a rise in telemedicine visits and reports of lower utilization across a variety of settings [4, 7, 10] . Emergency department (ED) visits are common during childhood, with one study estimating that 1 in 7 US children are seen in an ED annually [11] . Several studies have identified a decline in pediatric ED utilization corresponding with the onset of COVID-19 in the US and related efforts to curb the spread of infection [7, 12, 13] . These studies have used unadjusted time series data or differences in proportions to characterize changes in utilization over time. Forecasting models, which use time series data to predict future utilization, are an alternative analytic approach that can help policymakers and health systems anticipate resource allocation, characterize seasonal patterns of disease, and identify changes in healthcare utilization [4, 14, 15] . In this investigation, we sought to identify changes in the presentations of pediatric acute care conditions in 2020 compared to prior years, using a geographically diverse multicenter pediatric ED dataset and ensemble (defined as use of a set of) forecasting models. We performed a multicenter retrospective study of 37 freestanding and non-freestanding pediatric hospitals which contribute data to the Pediatric Health Information System (Children's Hospital Association, Lenexa, KS). This study was considered exempt by our Institutional Review Board and approved by the Children's Hospital Association. We abstracted data from all encounters between January 1, 2010 to December 31, 2020 and with an associated ED charge. No exclusions were applied. The following data were collected for each encounter: a patient-level identifier, date of presentation and discharge/intensive care unit (ICU) stay, cost, disposition, age in years, and principal diagnosis, as International Classification of Disease, 9th and 10th revision (ICD-9 and ICD-10). We identified three groupings of ICD level principal diagnoses. The first group consisted of the most common pediatric ED diagnoses [16] : fever (ICD-9 780.6; ICD-10 R50.9), upper respiratory infection (URI; ICD-9 465.9; ICD-10 J06.9), acute otitis media (AOM; ICD-9 382.9; ICD-10 H66.9), abdominal pain (ICD-9 789.0; ICD-10 R10), bronchiolitis (ICD-9 466.1; ICD-10 J21), asthma (ICD-9 493; ICD-10 J45). The second group consisted of common diagnoses resulting in admission [17] : seizure (ICD-9 780.3; ICD-10 G40), pneumonia (ICD-9 485 and 486; ICD-10 J18), urinary tract infection (UTI; ICD-9 599.0; ICD-10 N39.0), depressive disorder (ICD-9 296; ICD-10 F33), skin/subcutaneous infections (ICD-9 682; ICD-10 L03), diabetic ketoacidosis (ICD-9 250.1; ICD-10 E10.1), and trauma (ICD-9 800-959, 990-999; ICD-10 codes starting with S, T07-T34, T66-T88). Finally, because previous work has shown COVID-19 not to affect birth rates [4, 18, 19] , the third group consisted of neonatal hyperbilirubinemia as a control group (ICD-9 774; ICD-10 P59) that was not expected to be impacted by pandemicrelated measures. We compared demographics (age, race, ethnicity, payor status, time of year, census region), diagnoses, and measures of acuity (hospitalization, ICU admission, and in-hospital mortality), and abstract based charges between 2010 and 2019 and 2020. We developed forecasting models according to methods of Pelletier et al. [4] . We transformed groupings of ED encounters (by month and year) into a time series object and identified seasonal trends between 2010 and 2019 for all encounters with estimated scatterplot smoothing and results displayed graphically. We assessed for seasonality of each diagnosis of interest using Weber and Ollech's test [20] . We constructed ensemble machine-learning forecasting models using autoregressive integrated moving average, neural networks, and locally estimated scatterplot smoothing algorithms with publicly available code [21] . Each algorithm was weighted on time-series cross validation [4] . We trained the model using data from January 12,010 to June 30, 2019, and tested the model performance from July 1, 2019 to December 31, 2020. We compared the predicted and actual rate of presentations using the mean absolute percentage error (MAPE), and we compared error rates from July 2019 to December 2019 to those from 2020. We performed exploratory analyses to explore changes in disease acuity during the COVID-19 pandemic: admissions (which included inpatient and observation status [22] ), hospitalization in an ICU (including pediatric or neonatal ICU), and encounters ending in in-hospital mortality. Analyses were performed using R 4.0.5 (R Foundation for Statistical Computing, Vienna, Austria). An α value of 0.05 was set as the threshold for statistical significance, and all tests were 2-tailed. A total of 29,787,815 ED encounters from 11,796,228 unique patients were included. 93.6% (27, 874, 730) of the encounters occurred between 2010 and 2019. Of these, 13.1% were admitted to the hospital (as observation or inpatient). The median age was 4.9 years (interquartile range, 1.6-10.8 years) and 52.7% were male. Characteristics of patients, rates of key diagnoses, cost factors and measures of acuity comparing the years 2010-2019 to 2020 are presented in the Table 1 . Encounter patterns are provided in Fig. 1 . There was an increase in encounters during the winter (January-March) and fall (September-December) compared to the spring (March-May) and summer (June-August). The proportion of encounters during 2020 was lower compared to prior years, with a nadir in April 2020 representing a 65.1% decrease from the past decade (89,404 in 2020 versus a median of 236,741 [interquartile range 223,020-252,451] from 2010 to 2019). All studied diagnoses demonstrated a seasonal trend (Fig. 2) . Abdominal pain, asthma, acute otitis media, bronchiolitis, depressive disorders, DKA, fever, pneumonia, and upper respiratory infections occurred more in winter months. Neonatal hyperbilirubinemia occurred more during fall to early winter. Skin and subcutaneous infections, urinary tract infections, and trauma occurred more during spring and summer. After the nadir of presentations in April 2020, some diagnoses, such as abdominal pain, hyperbilirubinemia, UTI, and trauma, demonstrated a positive slope, with decreases during the latter three months of the year. Other diagnoses remained low with minimal increase during the later months of the study period. Ensemble models were able to identify seasonal patterns in admission diagnoses and accurately predicted diagnosis-specific ED encounter rates from July 2019 until December 2019 but not from January 2020 to December 2020 (Fig. 3 , MAPE range, 1.4%-14.0% vs 9.6%-739.4%, p = 0.01). Encounters for depressive disorder remained within the 95% CI limits. Encounters for all other diagnoses decreased below the lower 95% CI of the forecast during 2020. Encounters for bronchiolitis and DKA recovered to within the forecast by June 2020. All other diagnoses remained below projected volumes (Fig. 3) . In models investigating acuity of presentation, overall hospitalizations in 2020 were decreased from years prior. ICU admissions were lower than prior years only in December, and in-hospital mortality occurred in similar numbers (Fig. S1 ). In forecasted models, admissions occurred below forecasted rates, ICU admissions and in-hospital mortalities occurred below forecasted rates at the end of the studied period (Fig. S2 ). We implemented a machine learning forecasting model to predict encounters to pediatric EDs during the year 2020 relative to prior years. Our findings support a substantial decrease in the rate of ED encounters representing the most common presenting ED diagnoses in 2020 compared to prior years overall. Compared to prior reports, the present work examines ED utilization across the entirety 2020 [7, 12] . We identified differences disease presentation for specific diagnoses in 2020 compared to the preceding decade. A study looking at a subset of these data from 27 hospitals and using three years of historical controls reported a decrease in encounters during the period between March 15, 2020 to August 31, 2020, with a decline in respiratory disease and low-resource intensity visits [7] . These findings also corroborate a single-center study which noted a decrease in volume, but increase in higher acuity cases, including those brought to the hospital by emergency medical services and a higher admission rate. This study also reported a decrease in the rate of patients left without being seen [23] . Another study evaluating EDs staffed by a general emergency medicine physician group noted decreases in encounters between January to June 2020 relative to the year prior, noting that the decline in pediatric encounters was greater than among adults [12] . The greater decline in pediatric encounters compared to those reported among adults may relate to the higher proportion of communicable or infectious etiologies as visit reasons for children compared to adults [20] . There were notable trends evident for the studied diagnoses in 2020 compared to preceding years. Healthcare utilization for UTIs was lower in 2020 compared to prior years. As UTIs are not communicable infections in children, it is possible that this decline in ED diagnoses was offset by care seeking in other settings, such as virtual primary care visits or urgent care centers. We found an unchanged number of emergency encounters for depressive disorder, despite a previously reported decrease in the number of mental health admissions [4] . Our finding corroborates reports of a mental health crisis [8] and findings for ingestions [24] during the pandemic. Surprisingly, we also identified a decrease in the number of admissions for neonatal hyperbilirubinemia compared to forecasts, despite reports indicating that COVID-19 did not appear to impact birth rates in 2020 [4, 18, 19] . Indeed, the original planned use of the neonatal hyperbilirubinemia group was as a control. It is unlikely that the COVID-19 pandemic could meaningfully reduce the true rate of neonatal hyperbilirubinemia [25] . This reduction may reflect changes in pediatricians' neonatal hyperbilirubinemia screening practices during the pandemic. Alternatively, the relative decline in hyperbilirubinemia visits may be indicative of alternative management strategies such as home nursing visits and proactive phototherapy. The lower rates of hyperbilirubemia noted in some prior years may be related to care practice patterns, utilization of hyperbilirubinemia as a primary diagnosis (with potential changes in coding related to the ICD-9 to ICD-10 transition in 2015), and year to year birth rates. Additional investigations are warranted to determine the cause and potential repercussions of this change in utilization patterns. We observed healthcare utilization for potentially emergent conditions such as asthma, bronchiolitis, pneumonia, seizures and trauma to be below forecasted rates in 2020. Additionally, overall proportions of admission were lower in 2020 compared to prior years, though ICU admissions remained stable, corroborating with reports that the proportion of patients with higher acuity disease during the pandemic has been higher [23, 26] . The reasons for these changes in utilization are likely multifactorial. Some findings may relate to social distancing measures, leading to a decrease in communicable infectious diseases. Other changes may represent caregivers and patients seeking care from non-ED sites. However, one report has noted a decline by 30% among primary care pediatric practice visits during the pandemic [27] , suggesting that at least some of the observed declines may represent concerning delays in care. In one survey conducted in metropolitan Chicago, nearly one in four caregivers expressed hesitancy with seeking ED care for their children in a hypothetical emergency [28] . Our findings are subject to limitations. Though the PHIS database is rigorously quality controlled [29] , there may still be errors in the data abstraction and coding processes. Given the period assessed, we were required to use ICD-9 and ICD-10 codes, which may represent a potential source of error. Our findings may not generalize to other ED settings, particularly outside of the US, where the timing of COVID-19 and local responses may have differed. We identified decreases in the utilization of pediatric EDs in the US during the COVID-19 pandemic. Subsequently, we identified a modest increase in patient encounters. COVID-19 dramatically changed the paradigm of access and utilization of healthcare. A shift in local policies from social distancing to masking mandates likely reduced the ED burden for some pediatric infectious disease but also potentially contributed to delays in the evaluation of other diseases. Further research is required to identify how care-seeking has been impacted by the COVID-19 pandemic and evaluate its implications on pediatric care delivery. Dr. Pelletier is supported by grant 5T32HD040686-20 from the National Institutes of Health. Dr Rakker is supported by grant Fig. 3 . Forecasted ED encounters for specific diagnoses developed using ensemble forecasting models including autoregressive integrated moving average, neural network, and locally estimated scatterplot smoothing algorithms weighted on time-series cross-validation. The gray line is the model estimate, and the dark and light gray shaded regions are the model 80% and 95% CIs, respectively. The black line is the actual number of encounters for each month. The vertical line represents January 2020. MAPE indicates mean absolute percentage error. AOM, acute otitis media; URI, upper respiratory tract infection; UTI, urinary tract infection. Dr Horvat is supported by grant 1K23HD099331-01A1 from National Institute of Child Health and Human Development. Dr. Ramgopal is supposed by PEDSnet (Department of Pediatrics; Ann and Robert H. Lurie Children's Hospital of Chicago Writing -review & editing. Declaration of Competing Interest Authors S Coronavirus disease 2019 case surveillance -United States COVID-19 data from the national center for health statistics Clinical characteristics, treatment and outcomes of paediatric COVID-19: a systematic review and metaanalysis Trends in US pediatric hospital admissions in 2020 compared with the decade before the COVID-19 pandemic Spotlight on child abuse and neglect response in the time of COVID-19. The lancet public health COVID-19 related reduction in pediatric emergency healthcare utilization-a concerning trend Pediatric emergency department visits at US Children's Hospitals during the COVID-19 pandemic Mental healthrelated emergency department visits among children aged 18 years during the COVID-19 pandemic -United States Service use among Medicaid & CHIP beneficiaries age 18 and under during COVID-19 Reinventing pediatrics through video care first Emergency department utilization by children in the USA Characterizing pediatric emergency department visits during the COVID-19 pandemic Trends in pediatric emergency department utilization after institution of coronavirus disease-19 mandatory social distancing Human resource allocation in an emergency department: a metamodel-based simulation optimization An overview of health forecasting. Environmental health and preventive medicine Epidemiology of a pediatric emergency medicine research network: the PECARN core data project Reasons for being admitted to the hospital through the emergency department for children and adolescents Stillbirths during the COVID-19 pandemic in England Changes in preterm birth phenotypes and stillbirth at 2 Philadelphia hospitals during the SARS-CoV-2 pandemic Overview of emergency department visits in the United States GitHub -drjonpelly/pediatric_seasonality Resource utilization for observation-status stays at children's hospitals Pediatric emergency department volumes and throughput during the COVID-19 pandemic Hospital encounters for pediatric ingestions before and during the COVID-19 pandemic Bilirubin levels in patients with mild and severe Covid-19: a pooled analysis Pediatric admissions to emergency departments of North-Western Italy during COVID-19 pandemic: a retrospective observational study American academy of pediatrics and children's hospital association letter to the US Department of Health and Human Services regarding federal emergency funding from the Coronavirus Aid, Relief and Economic Security (CARES) Act Parent-reported hesitancy to seek emergency care for children at the crest of the first wave of COVID-19 in Chicago Corticosteroids and mortality in children with bacterial meningitis Supplementary data to this article can be found online at https://doi. org/10.1016/j.ajem.2021.05.047.