key: cord-0201451-h9ao55v9 authors: Xie, Jin; Zhang, Teng; Blanchet, Jose; Glynn, Peter; Randolph, Matthew; Scheinker, David title: The Design and Implementation of a Broadly Applicable Algorithm for Optimizing Intra-Day Surgical Scheduling date: 2022-03-14 journal: nan DOI: nan sha: c834788d71ee52e1f5c97849c99b41c953d9d207 doc_id: 201451 cord_uid: h9ao55v9 Surgical scheduling optimization is an active area of research. However, few algorithms to optimize surgical scheduling are implemented and see sustained use. An algorithm is more likely to be implemented, if it allows for surgeon autonomy, i.e., requires only limited scheduling centralization, and functions in the limited technical infrastructure of widely used electronic medical records (EMRs). In order for an algorithm to see sustained use, it must be compatible with changes to hospital capacity, patient volumes, and scheduling practices. To meet these objectives, we developed the BEDS (better elective day of surgery) algorithm, a greedy heuristic for smoothing unit-specific surgical admissions across days. We implemented BEDS in the EMR of a large pediatric academic medical center. The use of BEDS was associated with a reduction in the variability in the number of admissions. BEDS is freely available as a dashboard in Tableau, a commercial software used by numerous hospitals. BEDS is readily implementable with the limited tools available to most hospitals, does not require reductions to surgeon autonomy or centralized scheduling, and is compatible with changes to hospital capacity or patient volumes. We present a general algorithmic framework from which BEDS is derived based on a particular choice of objectives and constraints. We argue that algorithms generated by this framework retain many of the desirable characteristics of BEDS while being compatible with a wide range of objectives and constraints. Aggregated annual expenditures on surgical services in the United States, those including the direct cost of surgery and post-surgical hospitalization, are estimated at close to $1 trillion. This figure amounts to about a quarter of the total health care costs in the U.S., which in turn are nearly 50% higher on a per capita basis than those of any other country [11] . The surgical environment is often the most expensive and highly resourced in the hospital, with one estimate putting the sum of direct costs, wages and benefits, and surgical supplies of one minute of operating room time at $36 [5] . Delays in access to recovery beds in the post anesthesia care unit, the intensive care unit, or the acute care unit are associated with adverse patient outcomes, surgical cancellations and delays, and higher spending. Improvements to the efficiency of surgical services and post-surgical recovery have the potential to meaningfully impact United States healthcare quality and expenditures. At most US hospitals, the majority of surgical procedures are electively scheduled and offer an opportunity to smooth demand through better scheduling (although procedure-urgency mix varies across hospitals). For most US hospitals, surgical scheduling is a largely decentralized process in which each surgical service (e.g. neurosurgery or orthopedic surgery) is assigned "blocks" of operating room time (e.g., two rooms 7am-5pm every Monday and Wednesday). Each block is then assigned to each surgeon in each service, e.g., a particular surgeon may have a room every Monday and Wednesday, or are used for the entire service, e.g., a Wednesday room may be reserved for urgent orthopedics procedures to be performed by whichever surgeon is available. Elective procedures are scheduled when a surgeon sees a patient in clinic and determines whether surgery is necessary and how soon it should occur. The clinic visit may be weeks to months in advance of surgery. In most hospitals, only a minority of procedures are emergent, ideally initiated within an hour of a patient's arrival, or urgent, ideally performed within 24-48 hours of a determination of the need for surgery. Hospital managers and schedulers face four common challenges to optimizing operating room and postoperative bed utilization. The first is variation in scheduling lead times. Patients requiring urgent or emergent procedures arrive randomly and patients requiring elective procedures have different clinical needs and preferences for when to have the procedure. In the United States, where elective surgery is relatively profitable, hospitals face financial pressure to accommodate patient preferences. The second challenge is that surgical services and individual surgeons enjoy significant autonomy, limiting the decision making power of any central scheduler and creating challenges for implementing centralized optimized scheduling. Typically, each surgical service, e.g,. orthopedics, is allocated operating room time in blocks that are then assigned to individual surgeons within the service. Each surgeon enjoys significant discretion in how to use their blocks, which allows them to prioritize patients based on clinical acuity, accommodate patient preference, and attend to other professional commitments. However, this flexibility may also create perverse incentives that may lead to inefficient use of OR time. For example, if a surgeon plans to attend a conference, she may make informal arrangements to swap blocks with other surgeons or, if she does not bare the costs of low OR utilization, may leave her block unused on the off chance she does not attend the conference. The third, closely related, challenge is that surgeons and operating rooms are non-fungible and subject to frequent changes in availability. A large institution may have to deal with the resignation or hiring of one or more surgeons per month and at any given time one or more operating rooms may be closed for repairs or upgrades. New surgeons bring their own patient mix and may require access to different resources and operating room blocks. Approaches to scheduling optimization that incorporate surgeon-specific profiles (e.g., average patient post-operative length of stay) and operating room characteristics (e.g., only some operating rooms are suitable for cardiac surgery) may require relatively frequent updating corresponding to such changes. The fourth challenge is that most hospitals have limited technical infrastructure and expertise to implement or update sophisticated scheduling algorithms. EPIC and Cerner, the two largest US electronic medical record (EMR) vendors, account for over 65% of patient beds in the US but offer tools to operationalize only a few simple machine learning algorithms, no functionality to implement optimization algorithms such as dynamic programming, and limited or no functionality to automate the import of optimized schedules generated externally. Efforts to optimize scheduling often rely on extracting data from the EMR, processing it externally to generate optimal schedules, and a manual process to modify the schedule to match the one produced through optimization. Two principled approaches to optimizing operating room and postoperative bed utilization are to change the block schedule or to change how surgeries are scheduled into a fixed block schedule. Each presents challenges associated with complex constraints and changes to surgical demand and hospital resources, and relatively few have been implemented [14] . The surgical block schedule coordinates the availability of numerous stakeholders and resources and, in some institutions, may remain largely unchanged for decades. There have been a few successful examples of the use of operations methodology, simulation and integer programming, to redesign the block schedule of a major hospital to reduce wait times for non-elective cases and smooth the demand for postoperative beds; see [17, 18] . These projects required detailed models representing the hospitals at which the work took place. They utilized analytical resources beyond those available to the hospital. These studies document a number of significant practical barriers involved in changing block schedules and the sustained leadership required to make the appropriate changes in block scheduling; including altering surgeon research and clinical schedules. Non-stationarity in the surgical volumes add to the practical considerations discussed in [17, 18] with regard to the time required to implement changes in blocks. Both works describe the difficulty of evaluating the impact of the interventions presented as a result of significant variations in surgical volume and changes in surgical case mix. The policy described in [18] is updated every couple of years to account for the changes in volume as well as the availability of surgeons, operating rooms, and patient characteristics. Coordinating surgical scheduling into a fixed block schedule faces similarly complex challenges. Surgeons usually enjoy significant independence in scheduling cases into their assigned blocks. This allows surgeons to prioritize patients based on clinical urgency and difficulty. A surgeon may decide that for some elective cases it is safe to wait months while others should be completed in days; a surgeon may prefer performing difficult cases early in the morning; or a patient may require concurrent procedures from more than one surgeon requiring that surgeon schedules be coordinated. A few operations-based approaches to surgical scheduling, using machine learning, optimization, and simulation, have been developed and deployed [7] . Significant effort is required to generate detailed models that capture the idiosyncratic constraints of the institutions where the work is performed, develop specialized software to deploy the scheduling algorithms, and secure the cooperation of surgeons whose incentives may not be fully aligned with the overall efficiency of the system [7] . The work in [15] documents how the combination of complex constraints and non-stationary system characteristics led to the suspension of the use of the algorithm designed in [7] . In particular, the institution opened new operating rooms, opened new recovery beds, and repurposed existing operating rooms and recovery beds far faster than the tool could be updated and redeployed. Institutions seeking to improve surgical block allocation or optimize scheduling have to develop models that incorporate their operational constraints, develop internal analytics expertise or partner with an external vendor, and overcome political and change-management barriers to deployment. Furthermore, they must be able to do so sufficiently quickly to update or redesign the models as their volumes or resources change. This may explain why, among the few papers reporting an optimization model implemented at a single institution, we found no publications documenting the successful reproduction of the work at other institutions. We develop the BEDS (Better Elective Day of Surgery) algorithm, a simple heuristic for interday surgical scheduling. We show that it has the following desirable properties: 1. Compatible with a common surgical scheduling workflow 2. Requires as input a minimal set of routinely collected data 3. Produces interpretable recommendations 4. Does not require significant reductions to surgeon autonomy or centralized scheduling 5. Is compatible with changes to demand, capacity, and operations such as surgeon vacations, illness, or joining/leaving the hospital workforce; long-term changes in surgical volumes; the opening or closing of new operating rooms and post-procedural beds; and the introduction of intra-day scheduling optimization 6. The implementation of BEDS at an academic medical center was associated with reduced variability in the daily number of post-surgical admissions 7. It is readily implementable in the limited technical capabilities of modern EMRs 8. A free version is available for use on Tableau Server, a commercial software widely used by hospitals in the US. We also present the BEDS framework, of which the BEDS algorithm is one application. We think of the BEDS framework as a structured formalism that can be use to guide the design of algorithms that retain many of the desirable characteristics of BEDS while being compatible with a wide range of objectives and constraints. Algorithms derived from the BEDS framework are readily implementable for surgical scheduling and may serve as benchmarks for more sophisticated scheduling algorithms. The rest of the paper is organized as follows. We provide a review of the literature on surgical scheduling in Section 2. We discuss the technical challenges involved in surgical scheduling under non-stationary settings in Section 3. We describe the BEDS algorithm in detail in Section 4. In order to test and validate the BEDS algorithm and inform our hospital's decision to potentially adopt BEDS, we constructed a discrete event simulation model based on historical data, which is described in Section 5. The validation results of the simulation model are described in Section 6. The results of implementation at the hospital are discussed in Section 7. We present the general BEDS framework in 8. Final considerations and conclusions are given in Section 9. Technical details such as data processing and data structure architectures for our validation, testing and simulations are provided in the appendix Section A. Surgical scheduling has been an active research area for decades [8, 13, 19] . Traditionally, the surgical operating room (OR) scheduling problem has been studied with operating room time as the single resource to be allocated and a single objective such as minimizing overtime, maximizing operating room utilization, and minimizing patient wait time, etc. However, surgical patients do not interact with the hospital only through the OR. Patients require many resources including postoperative units such as the post anaesthesia care unit (PACU), intensive care unit (ICU), and acute care unit or ward. Surgical planning with consideration of downstream unit demand has attracted significant attention in recent years. Two lines of research relate to our work in designing surgical scheduling that consider postprocedural units. One stream of work relies on the idea of directly adding the post-procedural unit capacity as constraints of the mathematical problem for OR-centered optimization, see [16] for a comprehensive review of recent developments along this line of research. Another type of work adopts the idea of using the performance metrics of downstream units as the objective function of the surgical scheduling optimization. Our work falls into the second category, for which we provider a closer review of the literature. We summarize the related literature together with our work in Table 1 as a comparison. All papers propose a solution to the problem of minimizing variability in the post-op units as the objective by optimizing surgical schedules, and are based on the surgical block system. We categorize by two different settings: master surgery schedule (MaSS), in which the assignment of blocks to services is optimized, and advance schedule (AdS), in which scheduling procedures into fixed blocks is optimized. For each publication we show its post-op unit considered, the modeling objective and method, information needed to implement the proposed method and whether the solution had been implemented in a hospital at the time of publication. In the master surgery schedule (MaSS) literature, [2] proposes using length of stay (LOS) distributions to calculate the distribution of the patient census given patient volume, then directly Table 1 Comparison of related papers. By "expected", we mean a point forecast for the mean. minimize the variance within a mixed integer program. Based on this idea, follow up work instead uses only certain statistics, such as the expected census, other than the entire distribution. Objectives such as the minimization of the peak census are also considered in [3, 6, 12, 18] . In the AdS literature, [1] provides a queuing theory-based method of estimating the optimal daily number of admissions accommodating a lower number of staffed beds during the weekends, and then solving a mixed integer programming (MIP) problem with the objective minimizing the difference between scheduled load and target load. A key assumption underlying this queuing theory analysis is stationarity, which does not necessarily hold in many applications. Other work, including [4, 9] , consider the stochasticity of LOS and surgical duration, and add the post-op units performance metric into the weighted objective function. Via different choices of institutional objectives and constraints, [4] formulates the scheduling problem as a two-stage MIP and [9] formulates the problem as a chance-constrained stochastic program. Our method, compared to these methods, is simple enough that we do not need to solve any mathematical programming problem and thus easy to implement and incorporate into the current system that hospitals are deploying. In order to motivate the scheduling heuristic that we introduce in the next section, we describe in this section the challenges that arise in finding a desirable scheduling policy from a practical perspective. In general, the resource management of an operating room (OR) is a decentralized yet highly inter-connected process. As a result, most existing works propose a centralized approach, which is usually realized by a large mathematical programming problem that incorporates all surgeon/services and independent stakeholders in the system (as in Table 1 ). When modeling a system with multiple resources and stakeholders, like the hospital or other medical institutions, researchers are inclined to integrate as much information as possible so that the optimal solution produced by the optimization model can achieve comparable performance as in real-world systems. As we can see in Table 1 , to generate a good advance scheduling policy, complicated mathematical optimization problems are constructed with complex objective functions and integrated constraints that require significant implementation efforts. For example, [18] reports the following challenge when implementing a MIP (Mixed Integer Programming)based block scheduling optimizer at the Massachusetts General Hospital (MGH). The authors found that some scheduled times produced by the integer programming program were actually infeasible in the real-world system due to constraints not initially incorporated by the model such as surgeon unavailability due to academic, administrative or clinical reasons. As a result, several rounds of iteration and negotiation with surgical leadership were required as part of 6-month effort to incorporate additional constraints into the model or change surgeon schedules. Hospitals routinely experience changes in their surgical workforce, changes to the physical operating room space and changes to patient volume and type. In [15, 17, 18] , such changes, including changes in volume and the opening of new operating rooms, are described as part of the reason why it was difficult to evaluate the impact of the interventions. At the same time, in mathematical optimization, especially integer programming models (which are the models adopted by most works in Table 1 ), when the objective or a constraint changes, the original solution may no longer be optimal or feasible [18] . Maintaining such models requires frequent updates when, for example, a new surgeon is hired, a new building/unit is opened, the number of operating or recovery rooms changes, a surgeon's schedule changes, a surgeon's block or clinic schedule changes, etc. If updating the algorithm and the tool through which it is deployed requires significant effort, it may fall out of use as was the case in [7] . As reported in [18] , a more serious problem arises when there is no feasible solution of the mathematical program after new constraints are introduced. In this case, the researchers have to make significant efforts based on institutionalspecific needs to deal with the trade-off between optimizing the institutional goal and mathematical soundness of the algorithm. Compared to robust optimization methods applied in surgical scheduling problems as in [10] , the approach we propose in the next section is easier to implement at the hospital, since it does not require a fixed time horizon for an offline optimization problem or batch scheduling of the the patients in implementation. The method we propose is an online heuristic that is applied each time a patient is scheduled. Meanwhile, as pointed out in [10] , robust optimization methods can be over-conservative in its application to surgical scheduling problems, which might lead to significant surgical delays in practice. We propose the BEDS (Better Elective Day of Surgery) algorithm 1. At the institution studied, the objective was to minimize the maximum total number of daily admissions while minimizing patient wait time for surgery. We designed BEDS to embed in a common surgical scheduling workflow. A surgeon examines a patient in clinic, recommends a type of surgery, and estimates how soon it should be performed. The patient then works with a scheduler to choose a day of surgery. BEDS performs a greedy-search among the feasible surgical days to minimize current post-op admissions and patient wait time. The only data required are: clinically acceptable wait time for the procedure, surgeon availability, patient availability, estimated procedure duration, and occupancy data for the post-operative unit. Only a small set of input data are required for implementation (Tables 2, 3 , and 4). Note that the greedy recommendation step of the algorithm is modular; the logic may be modified without perturbing the remainder of the algorithm. In the general BEDS framework the recommendation is modifiable based on the objectives or constraints of the institution or surgeon where it is to be deployed. For example, how much patient preferences shape the process may be modified by the institution by having the algorithm return the n top-ranked days to be offered to the patient to choose from. Many US hospitals use tableau (While specific numbers were not available, Tableau advertises that all of the top 20 US News & World Report ranked hospitals do). We have created a tableau workbook, available on Github (https://github. com/tengz-sudo/BEDS Tableau), for other hospitals to deploy BEDS on their tableau server. Below, as shown in Figure 1 , is an example of the user interface when the recommendation is set to present all feasible days color-coded by the number of admissions. When a patient comes to the clinic, the scheduler uses BEDS to choose the patient's primary surgeon, target post-procedure unit, and to filter out days on which the surgeon lack sufficient time for the case. The different colors on the heat map represent the number of patients already scheduled to be admitted to the target unit on the same day (green -fewest patients). The scheduler can then work with the patient to find a suitable "green" day. A different choice of objective and ranking would require only a modification to the definition of the color coding. for patient i being scheduled for a procedure do Determine that: 1. Surgeon j that will perform the procedure. 2. Scheduled procedure duration, g i . 3 . Clinical and institutional window of availability for the procedure, [L i , R i ]. 4. Patient's window of availability for the procedure, [l i , r i ]. 5. The unit u i to which the patient will be admitted after the procedure. Based on the above: Table 4 The header of the patient availability data. The column 'Primary CSN' is a unique ID for each patient visit to the hospital. In the use case at our institution, BEDS was designed to help schedulers avoid inadvertently selecting a day for which many patients requiring admission to the same unit have already been scheduled. As an example, we show real historical data for patients scheduled in March and June of 2019 for procedures after which they required admission to the pediatric intensive care unit (PICU) ( Table 5) . Each was scheduled for a date for which six surgical admissions had already been scheduled for the PICU, despite the surgeon having availability on days with significantly fewer scheduled cases (Figure 4 left) . Predictably, on the days the patients were admitted the total number of admissions to the pediatric ICU was significantly higher than on other days when the surgeon was available (Figure 4 left). In order to build an overall evaluation tool to analyze the impact of different scheduling policies on the patient flow, we build a discrete event simulation (DES) model. The simulation model allows us to combine different data resources like the midnight census data, procedure data, and staff availability data, etc. It generates a discrete event queue representing the patient flow under specific scheduling logistic. Each event in the queue is a representation of the admission, transfer between units, or discharge of a patient who visits the hospital (summarized in Table 7 ). For the scope of units in the simulation, our model focuses on pediatric areas (acute care and ICU areas). In this section, we elaborate on the high-level design and mechanism of our discrete event simulation model (summerized in Figure 2 ) with more details given in Section A.1. To use the simulation model, it is required that the user provides the midnight census data as well as the procedure record data of their medical institution as inputs. The midnight census data are records of all the patients staying in the hospital overnight with an assigned bed at midnight. The procedure record data are records of every surgery that happen at the hospital (This includes the time, duration, type, who performed the surgery and in which operating room), see Section A.1.1. The user has an option of either providing surgeon availability data or choosing a built-in function that generates surgeon availability from the data procedure record. We summarize the inputs of the model in Table 2 , 3 and 4. The major steps of simulation is summarized as follows: • Step 1: Feed the input data midnight census, procedure record and surgeon availability (if applicable) to the pre-processor and generate the data patient profile and surgeon availability (if not provided). Each row of the data patient profile summarizes the trajectory of one patient visit in the hospital, see Table 17 in Section A.1.1. • Step 2: Use the patient profile and surgeon availability data to generate the discrete event queue under the scheduling logic encoded in the Scheduler. • Step 3: Output the simulated event series and generate the simulated hospital record data for further data analysis. In this section, we present the results of simulating the patient flow at the Lucile Packard Stanford Children's Hospital (LPCH) from January 2019 to March 2021 using the simulation model described in Section 5. We model the outcome of implementing BEDS at the hospital with two experiments, as described in Table 8 . In both experiments, we apply BEDS to schedule all elective surgical patients to be admitted to the pediatric intensive care units (PICUs) or acute care units (PCUs). We include all the patient records with arrival times from 2018-01-01 to 2018-12-31 as a oneyear warm-up period for the simulation system since the occupancy of patients admitted during the warm-up period can affect the scheduling of the patients admitted afterwards. We include the warm-up period in the simulation system in order to obtain a valid initialization of the system. Otherwise, those patients who are already scheduled for surgery in the real hospital system would not be captured in the simulation system. For data pre-processing, we exclude patients and records with important missing information as described in Section A.1.1. As a result, 99.8% of the total 334274 records in the midnight census data are included and none of the total 5558 surgical patients to be admitted to PICUs or PCUs in the record during the period of simulation is excluded in data pre-processing. In the first experiment, the scheduler of the simulation model is set to the 'BEDS' mode on 2020-07-27, as in the real world implementation. Before that, the scheduler assigns patients to their original scheduled time in the hospital record. This experiment tests whether our simulation model reproduces historical hospital data and models the outcome of scheduling policies. We compare the daily elective surgical admissions into PICUs and PCUs in historical and simulated data in Figure 4 and Figure 5 . As a result, we conclude that the simulation model is capable of reproducing historical data in the hospital record during 2019.1.1 to 2020.7.26 before Fig. 2 In this figure, we show two example patients for whom the BEDS scheduling heuristic can be helpful for scheduling his/her admission to a day with less scheduled admissions in the patient's available window. Each patient was scheduled for a date on which six surgical admissions had already been scheduled for the PICU, despite the surgeon having availability on days with significantly fewer scheduled cases (as in (a) and (c)). The first patient requested a schedule for his admission to the hospital on 2019-03-11 and was scheduled to a day (2019-03-27) on which six surgical patients have already been scheduled to be admitted. The second patient requested a schedule for his admission to the hospital on 2019-06-28 and was scheduled to a day (2019-07-10) on which six surgical admissions have already been scheduled. For the both of them, there are days within their available window, as the days marked on the x-axis, when their surgeons are available, as the orange bar indicates, and there are less than six surgical admissions scheduled. Predictably, on the days the patients were admitted, the total number of admissions to the pediatric ICU was significantly higher than on other days when the surgeon was available (as in (b) and (d)). Table 7 Events considered in the simulation. The simulator generates the next event based on the current event type and follows historical data or a random time generator bounded in given time interval. the BEDS implementation. And during the period of 2020.7.27 to 2021.3.31, after the BEDS implementation, there are discrepancies between the actual data in the hospital record and simulated data. The simulation environment may be optimistic when estimating the real-world performance of BEDS. In real-world deployment, the actual performance could be less significant than the simulated one due to implementation details like human execution errors of hospital schedulers executing the scheduling logic, deviations from the Fig. 3 In this figure, we show the discrete event simulation (DES) model outline. As in the three modules on the left, we first collect the procedure data, the census data, and the surgeon availability data (if applicable) and feed all of them into the data pre-processor. Then the data pre-processor combines all the different data resources and reorganize them to two data frames: the patients data and the surgeon availability data. Then, the data frames are fed to a scheduler module and a priority queue is generated by a processor module as in the box in the middle. In the end, an event writer records the information of the event queue in the simulated data as the output of the model. assumptions on patient available windows and surgeon availability (as explained in detail in Section A.1.2), and missing data in the hospital records. The distribution of the difference between the simulated scheduled time and original time in record is in Figure 6 . In the second experiment, we simulate the potential outcome of implementing BEDS at LPCH at the beginning of 2019. The result was presented to the administrative team of LPCH and they decided to adopt BEDS as their scheduling policy after reviewing the simulated outcome. According to the simulation results summarized in Table 9 , the implementation of BEDS in 2019 could have reduced the coefficient of variation (the standard deviation divided by the mean) of the daily elective surgical admission by 35.2% in PICUs and by 26.0% in PCUs in that year. The hospital had a surgical level loading goal of decreasing the percentage of the days with less than two or more than five daily elective surgical admissions in PICUs. According to the simulation results, if BEDS had been implemented in 2019, the total number of days with less than two or more than five daily elective surgical admissions in PICUs would have been reduced from 120 to 77 in 2019. Table 9 Comparison of the historical data and simulated results with BEDS implemented in 2019 for daily elective surgical admissions in PICUs and PCUs. The BEDS algorithm was incorporated into the scheduling system of the Stanford children's hospital on 2020-07-27 for all elective surgical patients that were known, at the time of scheduling, to be cases that would require further admission to the pediatric intensive care units (PICUs) or acute care units (PCUs). The implementation of BEDS required buy-in from various stakeholders throughout the health system. A team of surgeons, schedulers, information services representatives, and peri-operative leadership finalized a design for the tool that did not require a significant change to the existing surgery scheduling process. BEDS was built over a six-month period directly in EPIC, Stanford's electronic medical record system. When a patient is identified by the surgeon as needing surgery in clinic, the surgeon will contact their scheduler with details about the procedure, urgency of the case, and expected postoperative recovery unit. The scheduler will open the surgical scheduling flowsheet in EPIC and begin to fill in the necessary information from the surgeon. Schedulers will input the postoperative unit and a possible future date for surgery in the scheduling flowsheet. This will automatically populate a calendar heatmap in EPIC that will show dates two weeks prior to the reference date and a month post the reference date. Using the heatmap as a guide, schedulers will select a few days that have a low number of surgery admissions to propose to the patient. The schedulers will then contact the patient to confirm a date of surgery. The main function of BEDS is the heatmap to visualize the best and worst days to schedule patients for surgery based on the number of currently scheduled surgical patients with the same in-hospital postoperative recovery destination. The BEDS heatmap also works for scheduling outpatient surgeries, cases that do not require a hospital bed post-operation. BEDS is used by all surgical services, with the exception of Cardiology, to schedule elective surgical cases. Design of the functionality and features of BEDS required input from stakeholders throughout the patient journey, especially the end users (i.e. the surgery schedulers). Challenges with leveling the number of hospital beds needed for surgery patients were handled by the operating room teams on the day of surgery. With BEDS giving visibility to future hospital bed needs, level loading challenges are now solved at the time of scheduling the surgical case. BEDS has been designed so the ranges for the colors within the heatmap can be changed if adjustments need to be made as surgical volumes grow. To demonstrate the impact of implementing BEDS at the Stanford Children's Hospital, we present a comparison between the period of 2020-8 to 2021-3 after BEDS and the same months in the previous year (2019-8 to 2020-3) before BEDS in Table 10 , 11 and Figure 9 . We compare the same months during the year to accommodate seasonality issues. We compare the histogram, the mean, the coefficient of variation, the median, the 90% quantile, and the 90% quantile -median ratio of the number of daily elective surgical admissions in PICUs and PCUs before and after BEDS. We notice that the volume of the patient flow at the Stanford Children's Hospital has been fluctuating in 2020 due to the outbreak of COVID-19 and other changes at the hospital, as reflected in Figure 10 . Hence, to compare different periods before and after BEDS, we focus on a metric that is not proportional to the volume of the patient flow: the 90% quantile to median ratio of the daily elective surgical admission (QMRA). As in Table 10 , the QMRA in PICUs dropped from 2.0 (2019.8-12) to 1.5 (2020. [8] [9] [10] [11] [12] and from 3.0 (2020.1-3) to 2.0 (2021.1-3) after BEDS implementation. In PCUs, the QMRA changed from 1.500 (2019.8-12) to 2.333 (2020. [8] [9] [10] [11] [12] and dropped from 2.333 (2020.1-3) to 2.000 (2021.1-3) after BEDS implementation. As the result shows, it took a warm-up period in PCUs for BEDS to start to take effect. The daily admissions of elective surgical patients on different weekdays follow different patterns due to the arrangement of the block schedule of the surgeons, as described in Section 1. Hence, we partition the weekdays into five groups and compare the QMRA in each group before (2019.8 -2020.3) and after (2020.8 -2021.3) BEDS as in Table 12 and Table 13 . Within the two periods that we are comparing, there are in total n = 34 weeks. As a result, in PICUs, the QMRA dropped on Mondays, Tuesdays, and Thursdays, which are the three weekdays with the highest ratios before BEDS. And no weekday has a QMRA higher than 2.0 after BEDS. Similarly, in PCUs, the QMRA dropped on Mondays, Tuesdays and Wednesdays, which are the three weekdays with the highest ratios before BEDS. We propose a onesided hypothesis test for the size of the change of the QMRA after BEDS. For the distribution of the test statistics, we estimate it by non-parametric bootstrapping with m = 100, 000 samples. The results are summarized in Table 12 and Table 13 . For PICUs, the null hypothesis is that the change of the QMRA in the post-BEDS period (2020.8. Hence there is no evidence of a significant change of the QMRA after BEDS in PCUs. In conclusion, based on these results, overall, BEDS improves the performance on these metrics. The BEDS algorithm can be seen as one particular instance of a general BEDS framework 2 designed to produce algorithms with several desirable properties. The most important such property is that any algorithm generated by the BEDS framework is compatible with a very common surgical scheduling workflow: a surgeon examines a patient in clinic, recommends a type of surgery, and estimates how soon it should be performed. The patient then works with a scheduler to choose a day of surgery. Based on the constraints and objectives of the institution, the BEDS framework can be used to produce an algorithm to help the scheduler choose which dates to offer the patient. The only data required are the same as those for the BEDS algorithm (clinically acceptable wait time for the procedure, surgeon availability, patient availability, estimated procedure duration, and occupancy data for the post-operative unit), but additional data may be used for more sophisticated scheduling. We provide pseudo-code to explain the BEDS framework. The ranking algorithm chosen for the modifiable step may be fully-interpretable or based on a sophisticated optimization model. In either case, the results may be used to generate a simple, color coded calendar such as Figure 4 . The amount of choice patients have in scheduling their procedure may be altered by modifying the parameter n and offering each patient a choice amongst the top n-ranked days. We developed and deployed BEDS (better elective day of surgery) a flexible, interpretable algorithm Table 13 Comparison of the 90% quantile -mean ratio of the number of daily elective surgical admissions to PCUs between periods during the same time of year before and after the implementation of BEDS. to assist in the selection of the day for which a surgical procedure is to be scheduled. When deployed at an academic medical center, the use of BEDS over an 8 month period reduced variation in the number of daily elective surgical admissions compared to the same 8 months in earlier years. We showed that BEDS relies on very few, commonly collected data points, offers fully interpretable recommendations, and is compatible with implementation in the very technically limited functionality commonly available in large EMRs. We then introduced the BEDS framework, a general algorithmic framework of which BEDS is one application. The algorithms generated by the BEDS framework may retain many of the desirable characteristics of BEDS while being compatible with a wide range of objectives and constraints. The evaluation of the deployment of BEDS should be interpreted in light of a significant limitation. The disruption caused by COVID-19 changed healthcare provision, especially elective surgical care, in numerous ways for which we could not control. The BEDS algorithm is ready and available for use by other institutions and freely available for download as a Tableau workbook. The BEDS framework may be useful for institutions seeking to design a readily implementable approach to improving scheduling or as a performance benchmark for researchers developing more sophisticated scheduling algorithms. Below, we show the headers of the data midnight census, procedure record, and surgeon availability in Table 14 , 15 and 16 as examples of the inputs. In addition, we point out in the column 'Required' if each column in the headers is a required input for using the discrete event simulation model. The unrequired columns can be useful for either model validation or follow-up analysis of the results and are recommended during data collection. The column 'Primary CSN' is an encoded patient-visit ID. Each patient has a unique ID for each time he/she visits the hospital. With the data inputs in Table 6 provided, the simulation system will first pre-process them with the following steps: • Scope of Units: focus on Pediatric Areas (Acute and ICU areas). -patients with duplicate census record on the same day or inconsecutive census records during their stay, -patients with overnight surgeries, -surgery records missing the column 'Patient in Room' or 'Patient out of room'. We re-organize the input data into two data sets: patients and surgeon availability. The patient profile data contains the information on each patient visit to the hospital, as shown in Table 17 . The surgeon availability data record the available times of the surgeons, as in Table 16 . When surgeon availability is not provided, the model has a built-in function that infers surgeon availability with the following logic: The surgeon is assumed to be available at a specific date with a full block size, for example, seven hours, if the total number of hours of scheduled cases on record on the same Table 16 Surgeon availability data header. The column 'Available Hours' is the number of hours a surgeon is available for performing surgeries on a given day. day exceeds a threshold, for example, three hours. The threshold and full block size are inputs that users can adjust by themselves. We present here a list of assumptions we make relative to the patient flow and the hospital policy when designing the simulation model. • (Elective patients) A patient is an elective surgical patient and can be rescheduled if he/she satisfies one of the following criteria: 1. The patient is a surgical outpatient or surgical admit, 2. The patient's lead time is no less than one day. • (Available window) For an elective surgical patient whose arrival date is d 1 and his/her admission date is d 2 (d 2 > d 1 ), the length of the patient's lead day is d l = d 2 − d 1 . If the date the patient originally receives his/her surgery is d 0 , we assume that the available window for the patient to receive surgery is Table 17 Patient profile header. The column 'Patient Class' can take values as 'Surgical Outpatient', 'Surgical Admit', 'Surgical Inpatient', or 'Medical Inpatient'. A patient classified to be 'Surgical Outpatient' or 'Surgical Admit' is an elective surgical patient. The column 'Arrival Time' is the time a patient comes to schedule his/her visit or arrives at the hospital without an appointment. The column 'Admission Time' is the time of a patient's admission into the hospital. The column 'Available Window' is the inferred period that a patient is available for surgery. The column 'Unit List' is a list of the units that a patient visits during his/her stay in a timely order. The column 'LOS List' is his/her length of stay in each unit. The column 'In OR Times' is a list of timestamps that a patient receives surgeries. The column 'Primary Surgeon ID' records the ID of the primary surgeon of a patient's first surgery at the hospital. We introduce more details on our discrete event simulation (DES) model. Figure 2 shows the outline of the DES model. The model consists of three major components: 1. Pre-processor: the processor prepares and reorganizes the input midnight census data and procedure record data into patients and surgeon availability as described in A.1.1. 2. Scheduler (A module encoding the scheduling logic): It is called every time a scheduling request is made, and generates a specific scheduled time following a given scheduling rule. Currently, the scheduler has two modes: historical data and BEDS. The logic of BEDS is encoded into the scheduler, so that the simulator is able to generate the potential outcome associated with implementing BEDS. 3. Event queue: It is a priority queue with time being the index. In other words, the head of the queue will always be the event that has the earliest time. The idea of DES is implemented utilizing this queue, where the simulator repetitively reads from the head of the event queue, pops this event from the queue, generates the next event following a given logic, and pushes the new event into the queue. The event types we consider are summarized in Table 7 . Each event is associated with a specific patient. We present more results for the simulation model validation in Section 6. For the first experiment in Section 6, the comparison of the mean, coefficient of variation, and the 90% quantilemedian ratio of daily elective surgical admission in PICUs between historical data and simulated data are in Figure 13 . . We show the comparison of the mean (a), the coefficient of variation (b) and the 90% quantile -median ratio (c) of the number of daily elective surgical patient admissions into PICUs in actual (blue bars) and simulated data (orange bars) during the seven periods mentioned above. Scheduling admissions and reducing variability in bed demand Building cyclic master surgery schedules with leveled resulting bed occupancy A decision support system for cyclic master surgery scheduling with multiple objectives Minimizing bed occupancy variance by scheduling patients under uncertainty Understanding costs of care in the operating room Reducing surgical ward congestion through improved surgical scheduling and uncapacitated simulation Improving the efficiency of the operating room environment with an optimization and machine learning model Operational research in the management of the operating theatre: a survey A chanceconstrained operating room planning with elective and emergency cases under downstream capacity constraints Two-stage robust optimization approach to elective surgery and downstream capacity planning Health Care Spending in the United States and Other High-Income Countries Reducing boarding in a post-anesthesia care unit. Production and Operations Management A comprehensive review and analysis of operating room and surgery scheduling Scheduling operating rooms: achievements, challenges and pitfalls Implementing analytics projects in a hospital: Successes, failures, and opportunities Stochastic optimization approaches for elective surgery scheduling and downstream capacity planning: Models, challenges, and opportunities Pooled open blocks shorten wait times for nonelective surgical cases Systematic or block allocation at a large academic medical center Operating room planning and surgical case scheduling: a review of literature We thank numerous members of the hospital leadership for their partnership in implementing BEDS. In particular, Dr. Rebecca Claure, Dr. James Dunn, Patrick Kane, Ryan Bruvold, and Derek Garnholz. The authors report no conflicts or external funding. As a background of the variability of the volume of the demand at the hospital during different periods we are comparing in Section 7, we present statistics of the daily elective surgical patient arrivals in PICUs as well as the lead time in each month during 2019.1 -2021.3 in Figure 11 . As in Figure 12 , we calculate the auto-correlation of the daily elective surgical admissions in PICUs on weekdays and find that the auto-correlation peaks regularly for the lags of 5, 10 and 15 days, etc. This phenomenon can be explained by the weekly surgeon OR time block described in Section 1. In this section, we will explain more details of the simulation model.