key: cord-0146706-j40q12i5
authors: OBrien, Tyler C.; Foster, Steven; Tucker, Emily L.; Hegde, Sudeep
title: Iterative location modeling of hand sanitizer deployment based upon qualitative interviews
date: 2022-04-01
journal: nan
DOI: nan
sha: 760dc78c2f95c5167513f47effd1006b6816c4ea
doc_id: 146706
cord_uid: j40q12i5

The onset of the COVID-19 pandemic in Spring 2020 forced universities in the United States to quickly shift to remote learning in order to protect university students, faculty, and staff. During the summer of 2020, Clemson University prioritized a safe return to in-person learning by public health guidance. Because of concerns that COVID-19 was transmissible via physical contact between individuals and surfaces, adequate hand sanitation for all university personnel became a top priority. In this study, we examine how the Clemson University Facilities department team placed hand sanitizer dispensers across the university campus in order to maximize availability to students, faculty, and staff through qualitative interviews with the facilities personnel responsible for placing the dispensers. We present an iterative design process of how a facility location model was designed to encapsulate the same considerations of the facilities team. The iterative modeling approach blends human factors analysis of decision-maker interview data with model development practices. Through the creation of our mathematical model, we compare the effectiveness of the initial deployment of sanitizer dispensers with the optimal allocation based on our model results.

The onset of the COVID-19 pandemic for the domestic United States forced colleges and universities to shut down all in-person activities to protect their students, faculty, and staff (Birmingham et al., 2021) . Following this rapid shift to remote operations in the spring of 2020, many public universities began exploring methods through which a return to in-person learning would be possible by the fall. During the summer of 2020, Clemson University (CU) analyzed available public health data concerning COVID-19 in order to develop university-wide policies that, once implemented, would ensure a safe return to campus for university students, faculty, and staff. In accordance with CDC guidance for frequent hand washing (Centers for Disease Control and Prevention, 2020) , hand sanitation dispensers were deployed in buildings across CU's campus.

Due to significant disruption of hand sanitizer supply chains, the facilities department (hereafter CU Facilities) began gathering as many gallons of sanitizer and disinfectants as they could acquire. During the summer of 2020, the facilities department placed hand sanitizer dispensers across the university campus using ad hoc strategies and rules-of-thumb that simultaneously accounted for building traffic patterns and the equal allocation of dispensers between buildings. Because the initial deployment of the dispensers was made under significant time pressure due to the university goal of a return to in-person learning in the fall of 2020, the decisions on locating dispensers did not follow a formal methodology, such as the use of models.

With the increased availability of public health data and the benefit of hindsight, the university's facilities department saw an opportunity to improve their campus-wide hand sanitizer dispenser deployment in comparison to their initial dispenser deployment.

Building a model that represents and accounts for human decision-making raises a larger question: how can we develop an effective mathematical model of ideal decision-making that integrates (i) contextual knowledge of decision-makers and (ii) the modeling expertise of the modelers throughout the modeling process? Typically, in applied facility location modeling research, the modelers' understanding of the problem will increase over time, but the decisionmakers' understanding of the problem is largely static (ReVelle et al., 1977) as the modelers continue to iterate upon a developed model separate from the decision-makers. Instead, we propose involving the decision-makers at each iterative turn to simultaneously increase modelers' understanding of the initial problem and the transparency of the iterations for the original decision-makers. This enables the decision-makers to provide further insights to the limitations of the model and increase their understanding of the problem through the context of the current model iteration. This model development process serves as an alternative to modeling support and model evaluation as described by Chung et al. (2000) and Pidd et al. (2010) , respectively.

In this paper, we present an iterative modeling process that focuses on representative decision-making in the context of changing information. We use hand sanitizer station deployment at Clemson as a case study for iterative model development process in a partnership with CU Facilities. Initial models allocate hand sanitizer dispensers across CU's campus using the considerations and limitations that guided the facilities department in the summer of 2020.

We use semi-structured interviews (Hegde et al., 2020) and regular meetings to understand initial and follow-up decisions of decision-makers and the priorities of key stakeholders. Models are iteratively developed and refined in accordance with the interview findings. The result is a representative model that reflects current policy and decision-making, which can later be iterated upon through further data collection. Through the iterative process, limitations of each model iteration are revealed, model results are compared with real-world decisions, and model modifications can be made as needed.

The contributions of the paper are as follows: (i) an iterative model development process that supports changing decision-maker and modeler perspectives over time and (ii) the integration of qualitative research into facility location model development. The paper is part of a larger project that is focused on campus adaptation due to . One of the model iterations was published as a brief work-in-progress paper in conference proceedings (O'Brien et al., 2021) , and a separate paper that focuses on the qualitative research is in preparation (Foster et al., 2022) . In this work, we focus on the intersection and impact of the qualitative research on iterative model development.

The remainder of the paper has the following structure. In Section 2, we discuss related literature. In Section 3, we introduce the iterative model development process and the model iterations for the case example of campus hand sanitizer placement. Solution methods and data collection are described in Section 4. In Sections 5 and 6, we present results and sensitivity analyses, respectively. In Section 6, we discuss results, further implications of the study, and conclude.

When the COVID-19 pandemic hit, several college universities asked for their students to be sent home and not return until further notice (Birmingham et al., 2021) . Most universities made these decisions in order to protect their health and safety. Once students returned home, a majority of the universities began planning ways to respond to the virus and restructure their standard policies such that the students could be brought back to campus safely. Several studies have reviewed universities' responses to the COVID-19 pandemic (Hinrichs, 2021) . A blueprint is available to help campuses safely reopen (Cheng et al., 2020) , and surveys have been used to understand students' views of campus pandemic responses, including class modality changes (Steimle et al., 2022) .

Agent-based and mathematical models have been developed to monitor the spread of the virus on campus and provide universities with intervention strategies (Cashore et al., 2020; Bahl et al., 2021) . Wi-Fi infrastructures have been used to generate contact networks and to support agent-based simulations to identify potential transmission risk (Swain et al., 2021) . A tool is available to optimally distribute classroom seating to maximize occupancy while following social distancing (Swain et al., 2021) . The current study demonstrates CU's responses to the pandemic and their adaptation to its challenges using semi-structured interviews with key stakeholders in the decision-making process; specifically related to hand sanitizer station placement in buildings on campus. The current study then designs a representative model of the decision-makers' initial deployment method in order to offer an optimal solution to the problem.

We provide them with feedback to evaluate the relative efficacy of their decisions and improve their understanding of the problem.

Existing literature shows methods that formulate several model variations that each correspond individually to a specific decision scenario in order to demonstrate a single application scenario (Golany et al., 2012) . Likewise, other studies have developed multiple models to find the best optimal solution and generate different outcomes (Sarir et al., 2021; Song et al., 2013) . Even further, studies have also modeled iterative processes (Sexton and Ren, 2017) .

Unlike previous studies, however, the current study systematically integrates iterative modeling development process with insights based on qualitative techniques to develop models to represent human decision-making strategies. The framework that is presented in this study can be applied to other areas where it would be beneficial to blend qualitative methods into modeling for optimization.

Interdisciplinary approaches have been assessed and applied in many applications, including healthcare (Mansilla and Gardner, 2003; Slatin et al., 2004) . This study specifically relies heavily on using both qualitative (semi-structured interviews) and quantitative (optimization models) 6 research methods. This approach is similar to traditional mixed methods studies which seek to integrate qualitative and quantitative data to offer a more comprehensive and synergistic utilization of the data (O'Cathain et al, 2007) . There are several advantages to mixed-method studies (e.g., they offer the opportunity to gather much more rich, comprehensive data, they offer more flexibility, and they better reflect the decision-makers perspective). Mixed-method studies have been widely used in healthcare settings and primary care (Baskerville et al., 2001; Cashore et al., 2020; Creswell et al., 2004; McVea et al., 1996) . In their article, Creswell et al. (2004) offer several models that have been designed in order to apply a mixed-method strategy. A major focus is to gather qualitative data in order strengthen the design of data collection instruments (Howell et al., 2020) .

We distinguish this approach from other mixed-methods studies as, traditionally, they combine qualitative methods with statistical analysis to better understand system characteristics and decisions. In our blended approach, we combine similar qualitative methods (semi-structured interviews of key decision makers) with the process of mathematical optimization to create a mathematical model of decision-making. This model is designed to be transparent for decision makers while also able to accurately evaluate past decisions and support and improve decisions in the future. Such an approach is unique in that it allows us to understand implemented adaptation strategies and iteratively form a mathematical model that mirrors those strategic considerations.

In this section, we propose an iterative model development process, present five model iterations, and discuss the qualitative methods used to inform those iterations. The first iteration is a heuristic approach, and the following four are optimization models. For each model version, this section will provide: an overview of the model; formulation (which includes the solution methods or how the model was solved); assumptions; limitations; and why another iteration was performed. The iterative model development process is presented in Section 3.1, model iterations in Section 3.2, qualitative methods in Section 3.3, and quantitative data collection in Section 3.4.

The framework for the iterative model development process is presented in Figure 1 . The framework represents an integrated approach to blend quantitative optimization modeling with qualitative research to enable better decisions. There are three main stages. These are problem identification; model development; and implementation. We will use the deployment of the hand sanitizer stations across CU's campus as a case study.

The iterative development process begins with identifying a problem that can be addressed with an optimization model. Accordingly, a semi-structured interview guide is developed with appropriate questions. This step is completed in partnership with key stakeholders involved in the hand sanitizer deployment process. In our case example, the overarching problem was to locate hand sanitizer stations across campus to make it as convenient as possible for pedestrians and building occupants to access the stations. No further detail on objectives, specific decisions, constraints, or data was available at this stage. The initial interviews were focused on understanding the priorities, challenges and key decisions facing the facilities department during the pandemic.

The next stage is the model development. This stage as a whole is cyclical (Figure 1) , in that the steps may be completed multiple times. Each completed cycle represents the development of one model iteration. For each cycle, there are several recommended steps, though not every step is necessary each time. The model development stage concludes when an iteration of the model is considered sufficiently representative of the decision-process in practice.

Within the model development stage, the first two steps (conduct semi-structured interviews and analysis of qualitative content) are driven by qualitative methods. The goal is to understand what has been done to address the problem as well as priorities in the future.

Interviews. Semi-structured interviews with key stakeholders and decision-makers are conducted to gain insight into the decision-makers' context and methods. Participants include those in leadership from the emergency operations response body at Clemson University and related departments (housing, safety and health, and information technology), and decisionmakers (directors, managers, and operational staff) from the university facilities department. An interview guide was prepared based on knowledge elicitation techniques, primarily, the method developed by Hegde et al. (2020) , which is based on the Critical Decision Method by Klein et al. (1989) . The interview guide consisted of question-probes designed to elicit, from stakeholders, information pertaining to their goals, uncertainties, constraints, decisions and strategies in terms of the respondent's role in the university's response to the COVID-19 pandemic. Over the course of the model iterations, follow-up interviews were conducted that probed specific aspects related to their decisions, challenges and strategies, such as the types of data used, coordination with coworkers and other groups within and outside the campus, and details related to specific building-use patterns.

Analysis. Qualitative data from the interviews are analyzed to identify themes and insights that are related to model development. In the analysis, we aim to find connections and identify valuable information that can be used to support model development. As the data is updated with additional interviews, the themes and insights too are updated.

After the qualitative data has been analyzed, the framework moves to identify an appropriate model design for the current iteration. The design may remain the same across multiple iterations, though we highlight the importance of iteratively revisiting it to ensure it continues to be representative of the current understanding of the decision-process. New quantitative data may be collected to support the current model design. Additional sources for data may have been identified through the interviews. The next step is to develop a new iteration of the model. The model iteration is based on feedback on the results of each stage from the stakeholders, along with any additional insights on their decision-making that they may provide. This may occur concurrently with data collection.

More granular changes, such as revising the objective function or constraints, may be needed to reflect a deeper or changed understanding of priorities and restrictions. Larger revisions to the design may also be necessary, e.g., switching the framework from a network flow optimization to integer program. New solution methods are also developed at this step, as appropriate.

To evaluate the model iteration, the model is solved using current data, and sensitivity analyses are conducted. To identify model discordances with decisions in practice, these results are compared with historical decisions in practice, if available. Assumptions are evaluated to determine if they are limiting or representative of practice. The modelers may also discuss the model structure with decision-makers. If the model is determined to be sufficiently representative of practice, the iterative cycle concludes, and the development process transitions into the next stage: implementation.

Within the implementation stage, analyses are conducted using the developed, representative model. The model-recommended decisions are then implemented by the decisionmakers. Over time, tactical updates are made. These may include re-running analyses using updated data or minor modeling changes. If the decision-making context changes or substantially new information is revealed, the model may be revised by re-starting the iterative development stage. 

To demonstrate the iterative development modeling framework, we considered the case example of locating hand sanitizer stations across CU's campus. In this subsection, we describe each of the five model iterations. For each iteration, we present an overview of the model; the model formulation; model assumptions; model limitations; and why or why not another iteration was performed. An overview of the model version is presented in Table 1 . The version numbers in the first column will be used to distinguish between the different model iterations. The third and fourth columns identify the scope and focus of each of the model versions. For example, Single Building-Time Threshold (Model Version 2) aims to locate stations in a single academic building to cover classrooms. Notation used in each of the models is presented in Table 2 . Decision variables 1 if classroom ∈ is covered, 0 otherwise (binary) 1 1 if location ∈ is selected, 0 otherwise (binary) 1 Number of dispensers located in building ∈ (integer) 0, 2 ̂ Number of stations located in building ∈ at door ∈ (integer) 3 ̅ 1 if dispenser is located in building ∈ at door ∈ , 0 otherwise (binary) 4 Parameters

Number of dispensers to be located 1 Coverage threshold time 1

Time to walk from classroom ∈ to candidate location ∈ 1 1 if < , otherwise 0 (binary) 1 Population data for building ∈ 0, 2 Total number of dispensers available 0, 2, 4 Number of available pumps per station 2 ̂

Population data for door ∈ in building ∈ 3, 4 Number of dispenser stations pre-allocated to building ∈ 3 1 if door ∈ is in building ∈ , 0 otherwise (binary) 4

We first identified one COVID-19 adaptation by CUthe placement of hand sanitizers across campus. During introductory meetings with CU Facilities, we developed a general idea about the station deployment.

Based on our initial understanding of the decision-process, we brainstormed potential modeling approaches to support the station deployment around campus. We first considered network flow-type models, e.g., flow-capture, path-based, located-based, that could incorporate the movement of students. However, CU Facilities noted that each station needed to be inside a building, and hence it was not necessary to consider exterior, between-building flow.

Furthermore, it was unlikely that we would be able to collect sufficient data on within-building flow. Following these discussions, we decided that the basis of the initial modeling framework would be to allocate dispensers to buildings.

The initial model version is a heuristic that quickly estimates how many sanitizer stations to put in each campus building (Model Version 0). The aim is to cover a larger portion of the campus population, and the heuristic allocates dispensers to buildings with higher demand. It allocates a total of sanitizer stations across the set of buildings, , based on each building's demand, . The number of sanitizer stations allocated to each building ∈ is integer variable . It is given in equation (0).

The number of dispensers assigned to each building is rounded up to be an integer value.

Demand is defined as the total number of events that an exterior door in a building ∈ was accessed. We used this to identify where people are concentrated across campus.

An underlying assumption is that allocating dispensers to locations with higher demand will result in the dispensers being used more frequently. Rounding the number of dispensers up to the nearest integer may require a greater number of dispensers than the total number available . One limitation of the heuristic is that it does not provide specific locations for stations to be placed within the buildings themselves.

The heuristic has no guarantee of optimality and did not provide specific within-building locations as desired.

To address two limitations of the heuristic (optimality guarantee and within-building locations), we developed a model to optimize locations within a single building (Model Version 1). A type of max coverage facility location model was developed to determine the optimal locations for dispensers within Freeman Hall, an academic building at CU. There are two types of locations, i.e., classrooms, , and candidate dispenser locations, . The goal was to maximize the number of classrooms covered by dispensers. A classroom ∈ is covered by a dispenser ∈ if the travel time, , from classroom to dispenser is less than the threshold time, ; this is designated via a binary coverage parameter, , ∀ ∈ , ∈ .

Candidate dispenser locations, , included doorways, hallway intersections, and other locations along interior hallways. These were selected as candidates based on our experience with traffic flow through Freeman Hall.

The model formulation for Model Version 1 is as follows:

The objective function ( 

Model Version 1 could be applied to individual buildings across campus, but data availability is a barrier. Identifying candidate dispenser locations as well as the times between candidate locations and classrooms is time-consuming and would require involvement of building staff members. In addition, a goal was to determine how many stations to allocate in each building campus-wide, not only locations of stations. In Model Version 1, the number of stations in each building is fixed, .

Furthermore, the focus of the real-world decision-process shifted. CU Facilities gained access to smaller sanitizer dispenser stations and placed one in each classroom on campus. The framing of Model Version 1, i.e., locating dispensers to cover classrooms, was no longer necessary. The new focus of locating sanitizer stations was to cover exterior doors to provide people the opportunity to sanitize when they entered a building.

Iterating from the single building model, the focus shifted to be a campus-wide allocation of stations. The goal is to determine how many hand sanitizer stations to locate in each building based on each building's demand. We applied a target coverage objective to match the number of allocated stations with demand (Model Version 2). The aim is to reduce the difference between demand in each building, , ∀ ∈ and the number of available hand sanitizer uses. Available uses is defined as the number of pumps per station multiplied by the number of stations allocated to the building.

The model formulation is as follows:

The objective function ( 

After meeting with CU Facilities, we decided that one of the limitations, i.e., specific location recommendations within buildings, was important to address. Model Version 2 gives priority to high volume buildings rather than priority to high volume doors. CU Facilities felt that it would be beneficial to specify at which doors the stations should be placed.

The purpose of Model Version 3 is to provide exact locations for the stations within buildings. It aims to maximize the coverage of population across campus. Qualitative results from semi-structured interviews identified that exterior doors were among the most important locations for the stations around campus. We reflect this in this model version; eligible locations are the exterior doors, i.e., the entry and exits to each building. Doors may have up to two dispensers assigned to each. For this iteration, we also fix the number of stations per building ∈ to be a user-defined parameter, , e.g., representing the initial campus-wide allocation.

The set of candidate locations in each building, , is indexed, where | | is the maximum number of doors in any building. Not every building has the same number of exterior doors, but the model does not assign stations to non-existent doors because demand is 0. Integer decision variables ̂ reflect the number of stations placed at door ∈ in building ∈ .

The model formulation for Model Version 3 is as follows:

The objective function (3.1) maximizes the coverage of the population in buildings ∈ . Constraint (3.2) ensures that the total number of stations allocated to doors in each building is equal to a user-defined parameter, e.g., current number of stations in building. Constraint (3.3) allows up to two dispensers per door. Constraint (3.4) sets the decision variables to be binary.

Stations will be located only at exterior doors, and each door can have up to two dispensers but no more. We assume, as in previous models, that placing stations near highdemand locations will increase usage.

One limitation is that each building is restricted to its current capacity, which means that the model does not address cross-campus allocation. Because the model is trying to maximize population coverage and some doors can get more than one station, the model automatically allocates the maximum number of dispensers to the highest demand doors. This is reasonable in some cases; however, if doors have similar demand, the lower demand door may not be allocated a station.

Despite determining at which doors to locate stations, Model Version 3 uses a fixed number of stations per building. CU Facilities was interested in campus-wide allocation as well.

Model Version 4 identifies locations for dispensers in buildings as well as the number of stations to place in each building. The set of candidate locations in each building, , is indexed similarly to Model Version 3. Binary decision variables ̅ indicate whether a dispenser is placed at door ∈ in building ∈ .

The formulation for Model Version 4 is as follows:

max ∑ ∑̂̅ ∈ ∈ (4.1)

The objective function (4.1) maximizes the coverage of the population. Constraint (4.2) ensures that each building has at least one hand sanitizing dispenser. Constraint (4.3) makes sure that a door can only receive, at most, one dispenser. Constraint (4.4) ensures that the number of dispensers that are located does not exceed the number available (N). Constraint (4.5) sets the decision variables to be binary.

The model accounts for exterior doors only, and we assume putting doors at these locations will increase usage.

The results of the current model (Model Version 4) were provided to CU Facilities. With their approval, this model was effectively represented their decision-making process and would provide valuable insights to decisions moving forward. This concludes the final iteration of this case study.

Throughout the iterative model development process, qualitative methods from human factors engineering are used to better understand the current decision-making context and priorities moving forward. These steps involve conducting semi-structured interviews and analysis of qualitative content. In this subsection, we describe how the interviews were structured for our case example of sanitizer deployment at CU. The study was reviewed by CU's IRB and approved under the 'exempt' category. The protocol script and questions are provided in the Appendix.

The purpose was to develop a better understanding about how the safety-related decisions with regards to hand sanitation policies at CU. Semi-structured interviews are held with key stakeholders. We identified key figures at CU who were involved in the decision-making process for COVID-19 policies and procedures. There were 11 participants from departments including the facilities, procurement, environmental safety, and IT departments as well as the body responsible for emergency operations across the university. The interviews were conducted on Zoom and each participant granted us permission to record them. The purpose of recording these interviews was so that they could be analyzed.

There were five major categories of questions in the interview process. The first set of questions was related to the participant's role, specifically at the beginning of the pandemic and how it related to the university's sanitizer deployment. They also described the difficulties that they and their related department/team faced at the start of the pandemic in the Spring of 2020.

The next set was related to sanitizer station placement considerations. For example, they were questioned about the information they gathered from different external sources like the CDC and internal sources at CU who advocated for proper hand sanitation policies. These types of interview questions served as a strategy to find out more about ways that these departments shared information (interdepartmental communication).

Thirdly, the participants were questioned about their placement strategies. For example, they were asked about some of the goals and metrics they tried to set for placement totals. In addition, we wanted to know if they used any types of specific data or if they applied modeling techniques when coming up with a solution for where to locate the stations. We also tried to find out what their considerations were for the ideal or optimal deployment. After this, the fourth set of interview questions looked into how they monitored the situation once class modality changed from online to a hybrid model. These questions also asked about monitoring student and faculty usage at the stations. Along with this, the participants were asked if any adjustments to their initial policies were made. Lastly, the participants detailed directions for the future, and how policies may or may not change.

Initial questions included figuring out which decision-makers were involved in the process, deciding how locations for stations were considered the "best", and also getting a better understanding about the type of information that was shared between different departments.

Because these were semi-structured interviews, several probes were included which allowed us to dive deeper into a question and gain more insight.

Quantitative data was collected throughout the model development process. We obtained door access control data from CU's TigerOne department. TigerOne identification cards are required to scan into each building during the pandemic, and the TigerOne department tracked how many times each door on campus was opened. Data for each door was anonymized and aggregated by week. Because location decisions are intended to be strategic and dispensers not moved more than once per semester, more granular data was not necessary. A heat map of data aggregated by building is presented in Appendix Figure 1 .

In this section, we present results for the case study of hand sanitizer location deployment at CU. The initial deployment from Spring 2020 is presented in Section 4.1. The baseline data used in each analysis was from February 2021. We focused on 36 academic buildings of campus, and these buildings represent 55% of campus foot traffic areas.

The results for each of the five model iterations are in Sections 4.2 to 0. In each subsection, we present the deployment of sanitizers as the difference versus the initial deployment as counts and as histograms. For each model, demand at each door was populated using a week of door access control data from February 2021.The optimization models were coded in Python using the Pyomo package (Hart et al., 2017) and solved with the Gurobi optimizer (Gurobi Optimization LLC, 2022).

In Spring 2020, CU Facilities located 102 sanitizer stations across 36 academic buildings on campus. Each building received at least one station, and the average number of stations per building was 2.8. Cooper Library received 4 stations, and the College of Business Building received the most with 20. The full initial deployment is presented in Appendix Table 1 .

Model Version 0 is a heuristic that aims to allocate dispensers across 36 campus buildings. The full allocation is shown in Appendix Table 1 , and in summary form in Figure 2 .

The latter is a histogram that compares the results of the heuristic to the initial deployment. The

x-axis are bins which represent the difference in number of stations between the heuristic results and the initial deployment. For example, the bin "1" represents buildings needing one more dispenser than in the initial allocation. The y-axis is the number of buildings that fall into each bin. The distribution of the histogram shown in Figure 2 is right skewed, which indicates that CU Facilities might have over-covered some of the campus buildings. This directly relates to the qualitative analysis where members of CU Facilities noted they aimed to almost "smother" campus with sanitation supplies. Nine of the buildings have allocations via the heuristic that are the same as the initial deployment. For example, the Academic Success Center is shown to maintain the two stations that it initially had. There are nineteen buildings that are shown to need more dispensers than in the initial allocation. For example, Fluor Daniel initially had one dispenser but based on the heuristic results it should be receiving 2 dispensers. Finally, eight of the buildings had heuristic results which were less than the initial number of stations. One example is Barre Hall which initially had four stations where the heuristic suggest that it should receive just two dispensers.

We also performed sensitivity analyses and compared results using one week in February 2021 to three separate weeks in April 2021. The allocations are very similar and suggest the recommendations are robust to changes in the underlying data.

We note that the sum of all the dispensers located in buildings for the heuristic is greater than the initial number of dispensers, , because non-integer allocations were rounded up so that a building would not receive a fraction of a dispenser.

Model Version 1 aims to maximize coverage of classrooms in a single building. We considered Freeman Hall in our case example (Figure 3 ) (Clemson University Facilities Support Services, 2022). We considered 10 candidate locations for stations. These included the locations in the initial deployment as well as hallway intersections and interiors. The travel time between each of the candidate locations and classrooms were recorded by the research team.

We evaluated the optimal locations for four different allocations of dispensers, i.e., from 1 dispenser in the building to 4, by running Model Version 1 for = 1, 2, 3, and 4. We identified both the optimal locations for the stations within Freeman Hall as well as the maximum threshold for coverage time. The results are summarized in Table 3 . The first column shows the number of dispensers allocated to the building ( ), the middle column shows the optimal station locations for each of those p values, and the last column is the maximum coverage threshold that enables all three classrooms to be considered covered.

The optimal deployment based on Model Version 1 differs from the initial deployment of 

The Building Allocation model seeks to allocate dispensers to buildings across campus to meet each building's total demand. The supply of sanitizer for each dispenser, = 500 pumps, was estimated by CU Facilities.

Maximum Coverage Time p = 4 0, 1, 5, 9 T = 8 sec p = 3 0, 5, 9 T = 8 sec p = 2 0, 5 T = 24 sec p = 1 1 T = 30 sec 29 The full allocation is shown in Appendix Table 1 . A histogram (Figure 4 ) represents the differences between the results of Model Version 2 and the initial deployment. In general, the distribution in the difference in model-recommended vs. initial deployment is left-skewed; this

indicates that decision-makers may have under, rather than over-covered campus. Sixteen buildings have a recommended amount that matches the initial deployment, e.g., the Administrative Services Building is recommended to maintain its one station. Fourteen buildings may require fewer dispensers, e.g., Sikes Hall initially had three stations, but the model recommends one. Finally, six buildings may need more, e.g., Brackett Hall initially had 2 but is recommended to have 4.

The Exterior Door-Restricted Allocation Model determines at which doors to place stations in each building. Each door may receive 0, 1, or 2 stations. The total number of dispensers in each building is set equal to the initial allocation for the building. 30 The results for Freeman Hall are shown in Table 4 . One of the doors is receives two stations, and one of the doors receives one station. In the initial deployment, Freeman Hall had three doors with one station each. The model results suggest that it may be more effective to place two stations at one of the doors that receives more traffic. 

The final version, the Exterior Door-Unrestricted Allocation model both allocates dispensers to buildings and optimizes the placement at individual doors. The allocation results as compared to the initial deployment are represented in the histogram in Figure 5 and in Appendix Results were shared with CU Facilities, and they indicated that both the recommendations and the underlying model would support decisions related to station and other item placement around campus.

We present an iterative model development process that uses qualitative methods to incorporate context and priorities from key stakeholders. We use the hand sanitizer location decisions made at CU as a case study of this approach. By leveraging the quantitative and qualitative context, we suggest that this process may enhance the effectiveness of key decisions when both stakeholders' and modelers' understanding of situations change over time.

Each of the model iterations offers a way to address the decision of where to locate hand sanitizer stations at CU. The CU Facilities Heuristic is a basic depiction of the initial deployment and can be easily used in practice. The first optimization model, Single Building Time-Threshold, recommends locations within a specific building to cover classrooms, and the subsequent (Building Allocation) distributes stations among buildings based on demand. The

Exterior Door (Restricted Allocation) and Exterior Door (Unrestricted Allocation) models locates stations at specific doors, and the latter also distributes stations across campus. Which model is "correct" or appropriate in practice depends entirely on campus stakeholders. The model development process with embedded qualitative context is a mechanism to iterate towards decision modeling that is reflective of practice. In this subsection, we will note insights from each of the developed models.

The Single Building Time-Threshold model provides results that would maximize the coverage of classrooms in use during a typical semester. The optimal station locations for each value of from one to four (middle column of The models could also be used to locate other demand-based items at CU, including garbage cans, marketing techniques related to healthy COVID-19 guidelines, mask dispensers, etc. Additionally, the presented location models could be applied at other universities. Many schools similarly use card access in the same manner as CU to gain access into buildings. In these cases, data for the specific school could be incorporated into the Exterior Door-Unrestricted Allocation model to distribute hand sanitizers or other demand-based resources.

Alternatively, if constraints or priorities were slightly different, modelers could use the Exterior

Door-Unrestricted Allocation model as the initial iteration in a model development process applied to their campus, essentially a warm start.

This paper presents an iterative model development process (Figure 5 ) to develop and refine location models based on stakeholder interviews. The incorporation of qualitative research methods (i.e., semi-structured interviews) into model development augments the process with insights based on the experience of stakeholders making decisions represented in the model.

The aim is to ensure models are useful for decision-makers and sufficiently represent practice. This occurs in the iterative model development process in two ways. First, model development, including the structure, output, and assumptions at each iteration, is transparent for the decision-makers. They are aware of the limits for each model, and the iterative process only stops when decision-makers believe the model version is appropriate for the given context.

Second, through semi-structured interviews, we were able to understand more about the decision-making process underlying sanitizer station placement and how CU Facilities and others devised an initial solution. Using this input, we were able to iteratively design models that effectively represented their priorities and constraints. Throughout the process, partnership with stakeholders was pivotal.

We note that, in general, the results of several of the models suggest that the initial station deployment across campus was somewhat effective in covering the campus population. Yet, this initial deployment was limited in the documentation and transparency that would support the replication of this effort in a similar situation due to extreme time and resource pressure. Using the qualitative input to shape the model objective and constraints provides an opportunity to rigorously justify decisions made in the past and enable future decision-makers to perform tactical updates to the current deployment in order to better meet the goal of adequately covering the campus population.

This iterative development framework can also be applied in situations outside of the specific case study of sanitizer station deployment we considered. The core idea is that iterative model development can lead to optimization models that better reflect human decision-making.

It reduces discordance between the model and the actual decision-making and embeds stakeholder input into the development process.

In this paper, we present a method to integrate qualitative and optimization research methods to develop better optimization models. The developed models reflect the decisionmaker strategies, priorities, and constraints as well as input from other key stakeholders. The application of this type of integrated methodology is not limited to hand sanitation decisions nor university COVID-19 response. By identifying stakeholder adaptations through qualitative analysis, modeling can be better informed and more accurately represent other real-world Sikes Hall  3  3  1  3  Sirrine Hall  4  5  4  5  Strode Tower  1  2  1  2  Tillman Hall  6  5  4 

Brief introduction about the study and purpose of the interview. "The purpose of this study is to understand how Clemson University adapted to the COVID-19 pandemic. We are particularly interested in understanding how decisions were made relating to hand sanitation strategies on campus and how these strategies have been implemented. During this interview, we will ask you to describe your role in Clemson's response and hand sanitation policies, and seek your perspectives on what worked and what didn't. We seek your permission to continue recording

Modeling COVID-19 spread in small colleges

Process evaluation of a tailored multifaceted approach to changing family physician practice patterns improving preventive care

COVID-19 lockdown: Impact on college students' lives

Addendum: COVID-19 Mathematical Modeling for Cornell's Fall Semester

Cleaning, Disinfection, and Hand Hygiene in Schools-a Toolkit for School Administrators

How to safely reopen colleges and universities during COVID-19: experiences from Taiwan

Influence of model management systems on decision making: empirical evidence and implications

Floorplan for Freeman Hall

Designing a mixed methods study in primary care

Organizational Adaptive Capacity During a Large-Scale Surprise Event: A Case Study at an Academic Institution During the COVID-19 Pandemic

Network optimization models for resource allocation in developing military countermeasures

Gurobi Optimizer Reference Manual

Knowledge elicitation to understand resilience: A method and findings from a health care case study

COVID-19 and education: A survey of the research

Modeling the Use of Mixed Methods-Grounded Theory: Developing Scales for a New Measurement Model

Assessing interdisciplinary work at the frontier: an empirical exploration of symptoms of quality. Interdisciplinary Studies Project, Project Zero

An ounce of prevention? Evaluation of the 'Put Prevention into Practice' program

COVID Response: A Blended Approach to Studying Sanitizer Station Deployment at a Large Public University

Why, and how, mixed methods research is undertaken in health services research in England: A mixed methods study

Why modelling and model use matter

Facility location: a review of context-free and EMS models

Developing GEP tree-based, neuro-swarm, and whale optimization models for evaluation of bearing capacity of concrete-filled steel tube columns

Learning an optimization algorithm through human design iterations

& Phase in Healthcare Research Team

Conducting interdisciplinary research to promote healthy and safe employment in health care: promises and pitfalls

Crashworthiness optimization of foam-filled tapered thin-walled structure using multiple surrogate models. Structural and Multidisciplinary Optimization

Students' preferences for returning to colleges and universities during the COVID-19 pandemic: A discrete choice experiment. Socio-economic planning sciences

WiFi mobility models for COVID-19 enable less burdensome and more localized interventions for university campuses

Pyomo -Optimization Modeling in Python

Note to interviewer: The questions listed below represent important themes or categories of information to be obtained from the participant. The order of questions can be adjusted according to the flow of the interview and topics emerging from the participant's responses. 'Probes' listed alongside some of the questions indicate items or themes that the interviewer should look out for in the responses and ensure are covered during the interview. They may be used as follow up questions to guide the response of the participant or to have them elaborate on a response

) in terms of decisions around facilities / hand sanitation? Was there a contingency plan available for such a situation? What types of information were needed or sought regarding hand sanitation? Were these available? Probes: Categories and sources of information How was this information shared? Probes: Media of communication -emails, dashboards, databases, messaging systems How did you know what information to look at and when? Probes: Government and external reports and guidelines

Did you consider students' and their families' perspectives during this process? How? What were some of the considerations that emerged? Probes: Safety; flexibility; accommodation (esp. for those coming from outside of Clemson)

What were your main priorities in terms of hand sanitation? Probes: Infection control; graduation times; student stress; resources for remote classes Did you use any data analytics or modeling in the station placement?

Probes: no. of classrooms or spaces; classroom size; no. of students who wanted to be on campus vs. remote; other How did variability and uncertainty influence these methods? Probes: changes in advisories/guidelines; infection rates; student attendance; other Was there any simulation or testing involved? Once classes resumed (hybrid / full in-person), was there a way to monitor how things were going? Probes: attendance; infection rates; testing protocols; compliance; other Did plans/strategies have to be adjusted based on actual dynamics of classes and studentpresence on campus? What were some of these changes? Looking back, what types of information/data have been most important or useful in this process? What additional data or information would have been useful? How was information and decisions communicated across institutional layers -President and Provost's offices

How were decisions and implementation actions coordinated across various bodies? Who were the facilitators? What directions are being looked at for the future? Will strategies drastically change? What are the current benchmark goals for right now and future semesters

systems. We suggest that use of the framework in other application areas could improve the quality of optimization models to solve real-world problems.