key: cord-266480-u8o4eitu
authors: Colubri, Andrés; Kemball, Molly; Sani, Kian; Boehm, Chloe; Mutch-Jones, Karen; Fry, Ben; Brown, Todd; Sabeti, Pardis C.
title: Preventing outbreaks through interactive, experiential real-life simulations
date: 2020-09-02
journal: Cell
DOI: 10.1016/j.cell.2020.08.042
sha: 
doc_id: 266480
cord_uid: u8o4eitu

Operation Outbreak (OO) is a simulation platform that teaches students how pathogens spread and the impact of interventions, thereby facilitating the safe re-opening of schools. In addition, OO generates data to inform epidemiological models and prevent future outbreaks. Before SARS-CoV-2 was reported we repeatedly simulated a virus with similar features, correctly predicting many human behaviors later observed during the pandemic.

Introduction Introduction Introduction As countries shut down by SARS-CoV-2 reopen, decision-makers are debating how best to resume all levels of education to mitigate further spread of the virus (Vogel and Couzin-Frankel 2020; Edmunds 2020) . Public health officials and school administrators have championed a wide range of interventions, including mask usage, social distancing, and small classes. The efficacy of these interventions depends on two key factors that are as yet unknown: (1) how likely each intervention is to modify behavior and transmission and (2) whether students and other stakeholders are educated, equipped, and empowered enough to remain compliant.

Here we present a new way to address both problems in an integrated manner. Operation Outbreak (OO) is an educational curriculum and simulation platform that uses Bluetooth to spread a virtual "pathogen" in real-time across smartphones in close proximity. Students engage with OO by first learning about key topics in outbreak prevention and response. They then participate in an appfacilitated outbreak simulation designed to vividly illustrate what they have learned. Finally, we J o u r n a l P r e -p r o o f administer post-simulation reflection and analytical exercises to reinforce key points that can inform students' future responses to real outbreaks. OO simulations at SMA. Clockwise, from top left: "sick" student presenting to a health responder, epidemiologists analyzing outbreak data, members of the public health team bringing an "infected" student to the treatment center, "recovered" student showing her immune health status. B. Components of the OO Platform. Transmission model defines the parameters dictating the probabilistic spread of the virtual virus among participant phones.

During simulated outbreaks, the OO platform automates real-time contact tracing by recording all "transmission events" between phones, as well as observable changes in behavior that result. This automation yields critical data that are often missing from standard real-life outbreak datasets. The data are accessible via a web-based dashboard where users can visualize real-time information on simulated infection and transmission patterns or view raw data for analysis.

Weeks before SARS-CoV-2 was first identified in humans (Andersen et al. 2020) , we ran several OO simulations that mimicked outbreaks of a very similar SARS-like virus in which pre-symptomatic carriers caused a significant fraction of transmissions. Other epidemiological parameters representing early SARS and MERS outbreaks (e.g., basic reproductive number, R0, of 2-3) were also programmed into the app. Seeking to complicate traditional transmission dynamics (where participants know if they are sick), we built in asymptomatic transmission with high transmissibility to allow the virus to spread widely at the beginning of the simulations. These simulations took place in both school and conference settings, with hundreds of participants in close proximity. The app-generated data from these simulations represented the "ground truth" of the mock outbreaks, captured several essential features of SARS-CoV-2, and allowed us to observe behavioral changes among participants--many of which are now being mirrored in real life.

In this article, we describe the predictive power of our OO simulations, share the resulting epidemiological data, and propose ways to use OO to bring students back to campus safely by J o u r n a l P r e -p r o o f teaching them the fundamentals of pandemic response--a critical effort in fighting the current pandemic and preparing for the next one.

Initial design and use of OO. Initial design and use of OO. Initial design and use of OO. Initial design and use of OO.

We created OO in collaboration with Sarasota Military Academy (SMA) Preparatory School in 2015 as a two-week curriculum in pandemic preparedness, culminating in a class-wide outbreak simulation. We initially used stickers to "transmit disease." In late 2017, we introduced the OO app and platform, which triggers infection and recovery events using probabilities that can be flexibly configured, offering virtually limitless possibilities to simulate additional elements such as falsepositive cases, clinically-diverse strains and personal protective equipment (PPE). The OO platform includes three interconnected components ( Figure 1 ): (1) The mobile app uses the proximity and location-sensing capabilities of smartphones to propagate the virtual pathogen. The app currently supports the use of Bluetooth Low Energy (BLE) beacons and QR codes, which can be used to represent zoonotic infectious sources, protective items (e.g., facemasks and hazmat suits), and other interventions that attenuate pathogen transmission (vaccines or therapies). (2) An administrator website enables organizers of simulations to set parameters for each simulation (e.g., number of participants, duration, symptoms, outcomes). (3) A graphical dashboard retrieves data from simulated outbreaks (e.g., number of cases, transmission events, participant health status), and allows for visualizations, calculations, and other activities that develop skills in data science. The dashboard data can also be extracted for more sophisticated computational analyses.

Our OO app-based simulations at SMA over the last five years have involved more than 180 eighthgrade students who took on roles as general population, clinical workers, epidemiologists, and government officials. Their goal: to "win the game" by preventing the virtual pathogen from infecting more than a predetermined threshold of players.

OO allows organizers to parameterize different outbreak scenarios with known pathogens, de novo pathogens based on real microbes, or even fictional diseases. For our 2018 simulation at SMA, we chose Ebola as the pathogen and configured the symptoms and fatality rate accordingly. In 2019, given reported risks of emerging respiratory viruses (Cui et al. 2019 ), we simulated a coronavirus modeling the SARS R0 of 2-3 (Lipsitch et al. 2003 ) and the clinical symptoms of MERS (Assiri et al. 2013 ). We added one more key parameter: a period of asymptomatic transmission (twice the duration of the symptomatic period), to allow the virus to spread widely at the beginning of the game.

In early December 2019, we simulated outbreaks of the SARS-CoV-2-like virus at SMA (185 participants) and the annual retreat of the Broad Institute of MIT and Harvard (100 participants). We also simulated this virus in February 2020 at the day-long Florida Undergraduate Research Conference (FURC); 260 of the 590 attendees installed the app to run an unsupervised simulation for the full conference.

Realistic outbreak scenarios predict population behavior and increase engagement. Realistic outbreak scenarios predict population behavior and increase engagement. Realistic outbreak scenarios predict population behavior and increase engagement. Realistic outbreak scenarios predict population behavior and increase engagement.

The socio-behavioral parallels between our past simulations and the current pandemic are striking.

Notably, OO simulations have repeatedly foreshadowed the political distrust and altercations that have increased alongside COVID-19 in the US. They have vividly illustrated that viral outbreaks reveal and exacerbate existing rifts in society (Kim and Bostwick 2020) .

For example, in one simulation, students acting as "government officials" tried to spread disinformation to manipulate public behavior. This strategy backfired when students acting as "media" discovered the truth and informed the general population. "Citizens" who had previously complied with "government" orders immediately broke quarantine, further driving viral transmission. The government's refusal to properly "fund" its epidemiology team also drew widespread criticism--portending similar arguments now being made about fiscal allocation at all levels of the US government.

In another simulation, a member of the student "police" was approached by a classmate who refused to comply with orders to disclose his infection status (as indicated on the app). The officer "shot" the student (with a NERF gun) for non-compliance. Similar real-life incidents have been reported in multiple countries (Snyder et al. 2020; Hayes and Seucharan 2020) .

We have also consistently observed that student "family units" with fewer in-game "credits" (simulated money) are more likely to be infected and die than their more privileged counterparts.

This functional inequality is likely because the less fortunate "families" regularly spend their tokens on periodic "food distribution," leaving little left over to purchase "PPE."

We have simulated many of the interventions currently being considered for COVID-19, such as face masks, PPE, and even vaccines. In some cases, these interventions initially caused problems of their own. For instance, when masks were first introduced, a group of students "bought" them in bulk and tried to sell them at higher prices, only relenting under "public pressure" (precisely as was observed with the hoarding of medical-grade masks, toilet paper, and disinfectants at the start of the COVID-19 pandemic). However, in the long run, all three measures reduced infection transmission in the simulation, especially when given to highly-vulnerable participants (e.g., "healthcare workers").

Students themselves have proved to be an organic test of other proposed initiatives. In each simulation, without prompting, they implemented social distancing and a form of "remote work" (photographing and sharing educational material online to limit physical interaction). They also developed a way to assess players' health status and limit movements accordingly, paralleling the real-world use of health/immunity passports and containment strategies. However, as trust in the "government" eroded, some students tried to "game" the game by faking their health status screenshots.

Active learning exercises like OO have been repeatedly shown to improve STEM learning outcomes (Balicer 2007; Freeman et al. 2014 ). Our preliminary pedagogical data suggest this is true for OO.

Average test scores for the OO unit are higher than those for other units at SMA, across all genders and ethnicities. For the past three years, post-simulation survey data have shown that OO is the most anticipated lesson by all classes in any subject. Students have been especially eager to play the roles of epidemiologists and triage workers. In the last two years, 70 of 185 students signed up for this role; over half were female and 30% were underrepresented Hispanic or Black minorities.

The role of simulation in exploring outbreak dynamics. The role of simulation in exploring outbreak dynamics. The role of simulation in exploring outbreak dynamics. The role of simulation in exploring outbreak dynamics.

Realistic simulated outbreaks provide a unique opportunity to capture not only behavioral changes in response to viral spread, but also the "ground truth" of transmission, i.e., documentation of every single event (Fuller 2020) . The OO app produces real-time anonymous "contact tracing" data using Bluetooth, recording who "infects" whom and when, and the subsequent series of events for each participant, ending in "recovery" or "death." This data reflects the spread of the virtual pathogen among the participants with a granularity that is nearly impossible to replicate in the real world--and it can be used like real outbreak data for epidemiological modeling and visualization. It also allows us to quantitatively explore the effects of changing parameters (e.g., R0) and the impact of containment and prevention measures (e.g., social distancing and vaccination).

Our 2018 SMA Ebola simulation first showed how student social-distancing could affect an "outbreak's" trajectory ( Figure 2A More detailed data from the 2019 simulation allowed us to reconstruct transmission chains over time and identify important features of the outbreak, such as the existence of two super-spreaders causing 4 and 5 secondary infections early in the game ( Figure 2C ). As with COVID-19 (Kupferschmidt 2020), these super-spreader events accounted for a significant fraction of cases--in the simulation, 30% of all secondary infections were caused by these two participants.

The 2020 simulation at FURC using SARS-like parameters allowed us to explore the effect of herd immunity. Only 40% of conference attendees installed the app, leaving susceptible players buffered from each other by non-participants--just as vaccinated or otherwise immune individuals buffer the more vulnerable from the transmission of real diseases. Consistent with this observation, simulated transmission levels peaked throughout the day but never showed the exponential growth expected in an entirely susceptible population. The FURC data were particularly revelatory when paired with the conference program. The effective reproductive number as a function of time, Rt, remained below 2--again, consistent with a population with significant herd immunity--but spiked during activities that required attendees to be in close proximity to each other: two presentation sessions (posters and oral), a workshop session, and lunch ( Figure 2D ).

A roadmap for the near future: pandemic education, preparedness, and data gen A roadmap for the near future: pandemic education, preparedness, and data gen A roadmap for the near future: pandemic education, preparedness, and data gen A roadmap for the near future: pandemic education, preparedness, and data generation. eration. eration.

eration. The COVID-19 pandemic presents a unique opportunity to rethink the way we educate students and other stakeholders about outbreak response--and to do so in a way that can facilitate the students' return to the classroom. We envision OO as playing two key roles: (1) as a pedagogical platform for teaching fundamentals of pandemic response that are vital for the public to understand and (2) as a novel system for simulating outbreaks and evaluating real-world mitigation strategies, including those needed to restart in-person education.

We have already begun to leverage OO to help mitigate the COVID-19 pandemic. In summer 2020, we partnered with the One Summer Chicago program to train 2,000 students as social distancing ambassadors. As part of the training, the ambassadors integrated the app into their daily lives over a seven-day period. Each of three regions of the city was randomly seeded with the same number of index cases, while the app tracked social contacts and transmission events. The simulation results and post-simulation survey demonstrated that the students retained the knowledge they learned and had a significantly increased interest in public health careers after the program.

To further increase OO's realism, we are enhancing the platform with components specifically informed by and focused on SARS-CoV-2. These include:

• A multi A multi A multi A multi----faceted "health score." faceted "health score." faceted "health score." faceted "health score." This feature aggregates physical movement (quantified by a step counter or changes in GPS location), social interactions (quantified by Bluetooth proximity measurements), and infectious disease knowledge (quantified by quizzing users about outbreak science). This score influences participants' risk and recovery probabilities based on behaviors and responses during the simulation--effectively gamifying OO and incentivizing behaviors and responses that are beneficial during real-life pandemics, especially those in which underlying health conditions play an important role in determining outcomes.

• Tools to evaluate response readiness. Tools to evaluate response readiness. Tools to evaluate response readiness. Tools to evaluate response readiness. We are adding features that allow students and stakeholders to evaluate their mitigation strategies in real-time based on changing data. One new feature lets stakeholders choose which individuals can be "diagnosed" given available in-J o u r n a l P r e -p r o o f game funding, and assess resulting efforts to track and trace. We will also allow for simulated changes in pathogen genetics. This feature will generate more realistic data on pathogen transmission and evolution, and will support OO's use in more advanced classes (e.g., genetic epidemiology courses).

• Comprehensive educational curriculum on outbreak science. Comprehensive educational curriculum on outbreak science. Comprehensive educational curriculum on outbreak science. Comprehensive educational curriculum on outbreak science. We are developing a robust, modular, scalable curriculum on outbreak science in the form of an online and print textbook, online lectures, learning assessments, and an online video series. We are currently working on two curricula: one for middle schools and another for high schools and colleges.

We have already begun pilots at schools across the US.

• Remote learning capabilities, including add Remote learning capabilities, including add Remote learning capabilities, including add Remote learning capabilities, including add----ons to existing multiplayer online games. ons to existing multiplayer online games. ons to existing multiplayer online games. ons to existing multiplayer online games. To account for the dramatically increased numbers of students now in remote learning--and to mimic disease transmission in close quarters--we have created options for people to play OO with family members at home. We also are working on an online multiplayer version of OO, inspired by the so-called "Corrupted Blood Incident," a virtual--and unintended--pandemic in World of Warcraft (WoW) that occurred in 2005 due to an error in the game's code.

Epidemiologists later found many correlations between players' reactions to the virtual pandemic and documented historical responses to real outbreaks (Balicer 2007) , including failed quarantine attempts and a high potential for rapid global spread.

Conclusions. Conclusions. Conclusions. Unprecedented times yield unprecedented opportunities. The COVID-19 pandemic has rendered the traditional in-person school experience impossible without mitigation strategies, and such measures, from masks to hybrid learning, may combine to make the coming school year 'less than' what would have been. Yet the pandemic also presents a unique opportunity. We know that students engage most deeply with topics that affect them directly and daily--those they care about most. If we give students a new way to actively learn about epidemiology and public health through the lens of the pandemic, we can train them to play important roles in mitigating its spread and transitioning from lockdowns to reopening. We can also give them a 'more than' experience--one that ignites their interest in STEM and other education and gives them agency to prevent future pandemics.

Declaration of Interests Declaration of Interests Declaration of Interests Declaration of Interests P.C.S. is a co-founder and shareholder of Sherlock Biosciences, and is a non-executive board member and shareholder of Danaher Corporation

The proximal origin of SARS-CoV-2

Epidemiological, demographic, and clinical characteristics of 47 cases of Middle East respiratory syndrome coronavirus disease from Saudi Arabia: a descriptive study

Modeling infectious diseases dissemination through online role-playing games

Origin and evolution of pathogenic coronaviruses

Finding a path to reopen schools during the COVID-19 pandemic. The Lancet. Child & Adolescent Health

Active learning increases student performance in science, engineering, and mathematics

What's Missing in Pandemic Models

Ontario man dies in police shooting after mask dispute in grocery store

Social Vulnerability and Racial Inequality in COVID-19 Deaths in

Health education & behavior : the official publication of the Society for Public Health Education

Statistical Inference for Partially Observed Markov Processes via the R Package pomp

Why do some COVID-19 patients infect many others, whereas most don't spread the virus at all?

Transmission dynamics and control of severe acute respiratory syndrome

Three family members charged in shooting death of security guard who told a customer to put on a face mask

Should schools reopen? Kids' role in pandemic still a mystery