key: cord-289439-jrvl0ykn
authors: Nelson, Martha I.; Lloyd-Smith, James O.; Simonsen, Lone; Rambaut, Andrew; Holmes, Edward C.; Chowell, Gerardo; Miller, Mark A.; Spiro, David J.; Grenfell, Bryan; Viboud, Cécile
title: Fogarty International Center collaborative networks in infectious disease modeling: Lessons learnt in research and capacity building
date: 2018-10-23
journal: Epidemics
DOI: 10.1016/j.epidem.2018.10.004
sha: 
doc_id: 289439
cord_uid: jrvl0ykn

Due to a combination of ecological, political, and demographic factors, the emergence of novel pathogens has been increasingly observed in animals and humans in recent decades. Enhancing global capacity to study and interpret infectious disease surveillance data, and to develop data-driven computational models to guide policy, represents one of the most cost-effective, and yet overlooked, ways to prepare for the next pandemic. Epidemiological and behavioral data from recent pandemics and historic scourges have provided rich opportunities for validation of computational models, while new sequencing technologies and the ‘big data’ revolution present new tools for studying the epidemiology of outbreaks in real time. For the past two decades, the Division of International Epidemiology and Population Studies (DIEPS) of the NIH Fogarty International Center has spearheaded two synergistic programs to better understand and devise control strategies for global infectious disease threats. The Multinational Influenza Seasonal Mortality Study (MISMS) has strengthened global capacity to study the epidemiology and evolutionary dynamics of influenza viruses in 80 countries by organizing international research activities and training workshops. The Research and Policy in Infectious Disease Dynamics (RAPIDD) program and its precursor activities has established a network of global experts in infectious disease modeling operating at the research-policy interface, with collaborators in 78 countries. These activities have provided evidence-based recommendations for disease control, including during large-scale outbreaks of pandemic influenza, Ebola and Zika virus. Together, these programs have coordinated international collaborative networks to advance the study of emerging disease threats and the field of computational epidemic modeling. A global community of researchers and policy-makers have used the tools and trainings developed by these programs to interpret infectious disease patterns in their countries, understand modeling concepts, and inform control policies. Here we reflect on the scientific achievements and lessons learnt from these programs (h-index = 106 for RAPIDD and 79 for MISMS), including the identification of outstanding researchers and fellows; funding flexibility for timely research workshops and working groups (particularly relative to more traditional investigator-based grant programs); emphasis on group activities such as large-scale modeling reviews, model comparisons, forecasting challenges and special journal issues; strong quality control with a light touch on outputs; and prominence of training, data-sharing, and joint publications.

In 2018 the Fogarty International Center celebrates its 50th year as the only part of the US National Institutes of Health focused singularly on global health. The Fogarty International Center (FIC) uses high return-on-investment approaches to maximally impact global health, including (a) establishing strong international collaborative research networks, (b) building capacity for research in lowand middle-income countries (LMICs) through training, and (c) liaising with a wide range of other US government agencies and private partners with a shared interest in advancing global health. Here, we describe two flagship FIC-led programs that identified critical gaps in infectious disease research and formed global networks of researchers to address the scientific questions most needed to inform policy and outbreak response.

The 2001 foot-and-mouth disease outbreak in the UK, the 2009 influenza A/H1N1 pandemic, and the 2013-2016 epidemic of Ebola in West Africa underscore the continual threat of emerging and re-emerging acute infections. Knowledge of the mode of spread of a pathogen and the subpopulations at highest risk of transmitting the pathogen and experiencing severe disease can inform key decisions about social distancing and prioritization of therapeutics and vaccines. Designing disease models rooted in empirical data, enhancing existing tools for data analysis, and teaching collaborators around the world how to apply these tools represent highly cost-effective ways to prepare for the next infectious disease outbreak. Past disease events have highlighted the capabilities of infectious disease modeling and molecular epidemiology to effectively guide outbreak responses, but also revealed persistent challenges in making predictions in data-scarce and rapidly changing field settings (Woolhouse, Chase-Topping et al. 2001; Keeling, Woolhouse et al. 2003; Heesterbeek, Anderson et al. 2015; Woolhouse, Rambaut et al. 2015; Dudas, Carvalho et al. 2017) .

The increasing availability of large, electronic datasets, or "big data," presents new opportunities for scientists to understand drivers of disease. Electronic health records, social media, satellite imagery, and cell phone records provide highly granular information on human behavior, the environment, transmission patterns, and disease burden (Lazer, Kennedy et al. 2014; Bansal, Chowell et al. 2016; Simonsen et al., 2016) . Recent advances in next-generation sequencing techniques have markedly increased the amount of pathogen genetic data available for study, providing almost real-time insights into how pathogens from different locations are genetically related and infer transmission patterns (Dudas, Carvalho et al. 2017) . Increasingly, researchers are devising methods to collect data even in resource-limited settings. The highly portable MinION sequencer demonstrated how real-time wholegenome sequence data could be generated in remote locales during the Zika epidemic (Quick, Loman et al. 2016; Faria, Quick et al. 2017) .

Mobile phone records and remotely-sensed nightlights present additional strategies to track human movements and contacts in resourcelimited settings Wesolowski, Eagle et al. 2012; Wesolowski et al., 2016b) .

Since its inception in 2001, the Division of International Epidemiology and Population Studies (DIEPS) has played a central role in promoting the growth of a vibrant and multi-disciplinary infectious disease modeling community through its MISMS and RAPIDD programs. Together, these programs have strengthened infectious disease modeling on a global scale by developing research and training networks, facilitating collaboration and data sharing, developing and disseminating new methodologies, and building a stronger collective voice for evidence-based policymaking. Most critically, these programs have been catalysts for other larger US agencies and governments worldwide to expand support for infectious disease modeling research and to incorporate models into policy decision-making.

The MISMS project was initiated in 2001 to build global research capacity for the study of influenza viruses using computational methods (http://misms.net, Box 1). The acronym originally referred to the Multinational Influenza Seasonal Mortality Study, although the program quickly expanded beyond mortality studies. MISMS has a mission to promote research, training and inform policy, more specifically (i) to characterize the global epidemiological and evolutionary dynamics of influenza in different host species, and (ii) to empower influenza researchers globally to study influenza and inform policymaking in their own countries. Towards this end, 18 technical training workshops were conducted on six continents during 2007-2018. These workshops focus on reviewing the state of influenza research and epidemiology in different global regions, and hands-on teaching of time series analysis of contemporary and historical outbreaks, control measures, mathematical transmission models, and phylogenetic approaches (Fig. 1) . MISMS supports interdisciplinary approaches by fostering collaborations between historians, demographers, ecologists, evolutionary biologists, vaccine manufacturers, microbiologists, epidemiologists, and policymakers, as evidenced by the rich diversity of publications (Fig. 2 ).

None of the last three pandemics -influenza A/H2N2 in 1957, A/ H3N2 in 1968, and A/H1N1 in 2009 -originated in a high-income country. Building capacity for influenza research in low-and middleincome countries is an essential but underfunded part of global pandemic preparedness and a key goal of the MISMS program. A central feature of MISMS workshops has been partnership with local organizations to help identify talented scientists in under-resourced settings. Workshops are held either in low-or middle-income countries, or the program provides funding for researchers from these countries to attend workshops in high-income countries (Box 1). The 2009 MISMS workshop in Dakar, Senegal, was co-organized with Institut Pasteur and was the first influenza workshop ever held in Africa. In addition to periodic workshops that provide intense short-term training to 30-100 participants, MISMS staff also host junior and mid-career researchers from Central America, Europe, Africa, and Asia for further training at the NIH for periods ranging from a few months to several years. These scientists gain deep expertise in epidemiological and phylogenetic approaches and establish research programs to inform policy in their home countries.

Organizing workshops in under-studied tropical settings brought critical attention to the burden of influenza and the variability of seasonal patterns in less connected locales and warm, humid climates Bloom-Feshbach, Alonso et al. 2012) . Increased recognition of the global burden of influenza in tropical and semi-tropical regions also has encouraged uptake of routine annual vaccination in many middle-income countries. However, several MISMS studies rooted in epidemiological and virological data have pointed out that the semi-annual schedule is designed for wealthier countries in the temperate regions in the Northern and Southern hemispheres, and are suboptimal for many tropical countries that experience influenza at different times of the year, or have multiple peaks (Alonso et al., 2007; de Mello, de Paiva et al. 2009; Tamerius et al., 2011; Bloom-Feshbach, Alonso et al. 2012; Green, Andrews et al. 2013; Alonso et al., 2015a,b; Ayora-Talavera, Flores et al. 2017) . MISMS research has informed whether a tropical country should opt for the Northern or Southern hemisphere vaccine formulation -or in some cases both. Large countries with heterogenous influenza patterns and climate, such as China, Mexico and Brazil, may require different vaccine recommendations for their northern and southern regions (Alonso et al., 2007; de Mello, de Paiva et al. 2009; Ayora-Talavera, Flores et al. 2017) . Additionally, MISMS research demonstrating the low impact of senior vaccination programs in the US and Italy paved the way for expanded vaccination

Why have RAPIDD and MISMS workshops been so successful?

Since 2007, RAPIDD has sponsored 118 workshops on topics ranging from gain-of-function experiments to the latest developments in particle filtering methods. These workshops have had a remarkable track record of enabling constructive syntheses of current frontiers in the science of disease modeling, and in spurring innovation and analysis of case studies to advance those frontiers. They also have earned a strong reputation in the field as important venues for scientific discourse. Here we summarize distinctive elements of those workshops that we believe have contributed to their success.

• Lightweight and nimble proposal process, steered by two generous and visionary scientists that had deep expertise in the field. The program 'let a thousand flowers bloom' by accepting workshop proposals on a rolling basis with a two-page initial proposal. For promising proposals, program leadership then iterates with proposers to refine the focus on RAPIDD goals, ensure solid deliverables, and optimize the participant list.

• Small size and balanced composition. Research workshops are generally kept to 10-20 participants, to ensure free-flowing and natural discussion. Age structure is key, with emphasis on including junior scientists (especially postdocs) so there are people with the time and incentive to do follow-up work.

• Flexible structure. Adapt meeting structure to meet scientific needs. Some workshops focus on conceptual or methodological issues, others on particular disease systems. Some of the best workshops had a central theme combined with data in hand and time to analyze it.

• Keep workshops short (typically 1.5-3 days) and maximize the value of in-person meetings. Pre-workshop teleconferences (among leaders or the whole group) help to reach consensus on goals and scope, so talks are on target and preliminary work can get done. Avoid overloading the schedule with formal talks, to ensure ample time for discussion and breakout groups; some participants can act as synthesizers or reactors rather than giving talks. Relaxed interactions on a hike or at a pub usually pay off.

• Designed for follow-through. Before the meeting, specify deliverable and timelines, and identify who will lead these efforts (often RAPIDD postdocs or workshop organizers). Get key pieces such as data sets and foundational analyses in place beforehand. Require workshop reports within a few months, summarizing the scientific content of the meeting, progress toward deliverables, and new opportunities that have emerged. Support follow-up when warranted, including 'working meetings' of project leaders, but also be prepared to stop threads that are not productive or not aligned with program priorities.

Since 2005, MISMS has convened 18 training workshops on all 5 continents and trained several hundreds of scientists on computational methods for infectious diseases. The unique characteristics of these workshops include:

• The primary output of these workshops is technology transfer; i.e. capacity building in infectious disease analytics. Participants are not expected to become modeling experts after a short workshop, but instead they should understand principles of data analysis and modeling (to the point that they are able to understand a disease modeling article).

• MISMS workshops are typically regionally-focused and organized in tandem with a local university, research institute, or ministry of health.

Regional participants are identified based on an exhaustive PubMed search (especially in early years of the MISMS program), word-ofmouth, and existing MISMS contacts (particularly as the program matured). Public health experts from WHO, CDC, and local institutions, are always invited.

• Workshops are sometimes organized in conjunction with a larger influenza scientific meeting to decrease travel costs and optimize participation.

• Weeklong workshops include 2 days of general scientific session and 3 days of hands-on training. The general session is meant to expose the state of the art of influenza epidemiology in the region and highlight success stories in influenza modeling and MISMS collaborations. The hands-on training session demonstrate methods in time series analysis, transmission models, and evolutionary analyses.

• Workshops are open to scientists and public health experts who do not have prior training in infectious disease modeling, e.g. veterinarians, clinicians, virologists, lab technicians, policy makers, etc.

• Flexible travel support for participants based on abstract selection process; focus on participants bringing data that may be amenable to modeling, those who have a clear training plan, and those from underserved areas.

• Training portion of the workshop is limited to 30 participants, for 5-8 faculty, to facilitate one-on-one interactions. • Participants are encouraged to bring their own data; at least a day is devoted to small-group analyses of these data. Sample datasets are always available for demonstration purposes and for those unable to bring data (or with sparser datasets).

• While the workshops are focused on influenza, participation is open to scientists working on related infectious diseases, recognizing that relevant (or specific) influenza data may be scarce in some countries, and that computational skills are cross-cutting.

• Workshop deliverables include joint publications and supplementary issues that spur collaborative work (for instance, Supplementary issues on Big Data for Infectious Disease Surveillance or Historical Pandemics Bansal, Chowell et al. 2016; Simonsen et al., 2016)) • Similarly to RAPIDD, there is opportunity for follow-up beyond the workshop, primarily via longer visits to NIH for more extensive training with MISMS staff, and occasionally, through remote collaborations. Workshops can also be organized in the same region a few years later to gauge progress. MISMS staff provides support for completion of analytical work and manuscripts, where workshop participants are recognized as first authors.

programs targeted at children who are important foci of transmission (Simonsen et al., 2003; Simonsen, Reichert et al. 2005; Rizzo, Viboud et al. 2006; Simonsen et al., 2009) . Global vaccine strategies also require an understanding of how viruses migrate long distances between countries, and how viruses evolve antigenically to require vaccine strain updates (Ferro, Budke et al. 2010; Adler, Eames et al. 2014) . Active areas of MISMS research include a deeper understanding of influenza migration pathways at different spatial scales, climatic and demographic drivers of influenza seasonality, and the mapping between vaccine match and population-level protection, all of which can affect vaccine policy (Box 2).

The global network of collaborators that grew out of the MISMS program is a unique resource to leverage in the event of the next pandemic influenza or another infectious disease threat. As a case in point, in 2009, the presence of MISMS-trained researchers in Mexico facilitated early reporting of severity and striking age-shifts in mortality patterns that are signatures of new pandemic viruses (Chowell et al., 2009a , 2009c ). During the critical early stages of the pandemic, rapid reporting of these epidemiological patterns informed use of limited resources, including vaccine and M.I. Nelson et al. Epidemics 26 (2019) 116-127 therapeutics, as well as interventions such as school closures (Chowell et al., 2009a , 2009c , Chowell, Echevarria-Zuno et al. 2011 . MISMS collaborations also focus on "archeo-epidemiology" studies, or studies of past influenza pandemics and other historic scourges . Such studies have provided estimates of influenza transmission intensity, age-structured mortality patterns, spatial dynamics, and the distribution of mortality across multiple pandemic waves as far back as the 1880s (Andreasen, Viboud et al. 2008; Bloom-Feshbach et al., 2011; Simonsen et al., 2011) . Quantitative studies of past patterns can help predict the course of future pandemics and inform the range of plausible scenarios for preparedness, including elevated mortality occurring several years after the initial 

Policy impact of RAPIDD and MISMS: select success stories.

• Vaccination: MISMS studies in large and climatologically-diverse countries have identified geographic differences that affect annual influenza vaccination programs. Because of the timing of influenza epidemics and composition of circulating strains in Southern China and Southern Brazil, Southern provinces should use the Southern Hemisphere influenza vaccine, while the Northern part of these countries should use the Northern Hemisphere vaccine (Miller, Viboud et al. 2008; Chowell, Viboud et al. 2009; Yu, Alonso et al. 2013; Alonso et al., 2015a,b ).

• Surveillance: MISMS work on the phylodynamics of swine influenza viruses highlight the importance of undersampled "source" locations with large pig populations and intense swine flows to other regions, such as Russia. Surveillance efforts should target these regions that are expected to generate important global diversity of swine viruses ).

• Pandemic control measures: Mexico implemented nationwide social distancing measures to control the 2009 influenza pandemic (combination of mandatory school closure, restaurant closure, and cancellation of large gatherings) during an 18-day period in late April and early May 2009. MISMS researchers were among the first to work with the ministry of health to assess the dynamics of the outbreak and the impact of interventions. By fitting transmission models to surveillance data, they found that social distancing reduced influenza transmission by one-third, lending support to these costly measures. Beyond the 2009 pandemic, this works suggests that social distancing interventions could be implemented to mitigate severe outbreaks and alleviate the pressure on healthcare, for limited periods of time and until other interventions (such as vaccination) can kick in (Chowell, Bertozzi et al. 2009; Chowell, Viboud et al. 2009; Chowell, Echevarria-Zuno et al. 2011 ).

• Addressing data gaps: On the RAPIDD front, modelers thought creatively to find ways to fill data gaps that were essential obstacles to resolving policy-relevant health problems. For example, a common problem in livestock diseases is the lack of information on farm locations and animal movements, especially in the US. These are key ingredients to design transmission models, evaluate interventions, and help prepare for potential outbreaks such as Foot-and-Mouth disease. The RAPIDD network designed a method to leverage veterinary records and mandatory licenses filed by farmers. Further, much simulation work was done based on the richer UK dataset informing farm locations, animal movements, and observed Foot-and-Mouth disease outbreaks, to strengthen US models and remedy data gaps.

• Gain-of-function studies and pandemic risk prediction: As the debate on gain-of-function studies intensified, RAPIDD organized a twoworkshop series on "Assessing the outbreak potential of nonhuman influenza viruses using sequence-based risk approaches'" and "Modeling and Predicting Influenza Phenotypes" to explore modeling of pandemic potential. Gain-of-function studies indicated that an avian influenza virus undergies a series of genetic changes for human adaptation, which are required for efficient transmission among mammals. The workshops assembled global experts to synthesize current knowledge about the three key steps of the adaptation process: switch of the hemagglutinin surface protein to mammalian alpha-2,6 sialic acid binding, enhancement of pH-and temperature-dependent stability of the hemagglutinin, and adaptation of the viral polymerase to function in mammalian cells. These workshops spurred modeling work to infer pandemic potential based on ferret transmission experiments, and further work is underway to explore whether mapping of genotype-tophenotype traits is possible, despite rampant epistatic interactions across the influenza genome (Buhnerkempe et al., 2015a) . Other avenues for policy-oriented research include the optimization of global influenza surveillance strategies to identifies early precursors of pandemic strains.

• Vaccination decline in the aftermath of Ebola: Another example of the link between RAPIDD research and policy is a study on the precipitous decline in childhood immunization during the 2014-2015 Ebola outbreak (Takahashi et al., 2015) . RAPIDD modelers estimated that around one million children in Liberia, Sierra Leone and Guinea were vulnerable to measles following the suspension of vaccination campaigns during the Ebola outbreak. The study concluded on the urgent need to mount an aggressive vaccination campaign as soon as the Ebola outbreak subsided. This advice was heeded by public health authorities, as measles vaccination campaigns targeting several million children were launched in Sierra Leone and Guinea in October 2015 and in Liberia a few months earlier.

emergence of a pandemic virus (Box 2). MISMS also has worked actively to bridge human and veterinary research communities and understand the emergence of pandemic viruses at the animal-human interface. Two workshops held in Ames, Iowa, USA, in collaboration with US Department of Agriculture, focused on swine influenza, while another held in Padua, Italy, in collaboration with Istituto Zooprofilattico Sperimentale delle Venezie, focused on avian influenza, providing training opportunities in an active area of influenza research.

Improvements to genetic sequencing technologies have reduced costs worldwide and advanced the field of 'phylodynamics', in which epidemiological dynamics are inferred from pathogen sequence data. In the early 2000s, new computational packages, including Bayesian Evolutionary Analysis Sampling Trees (BEAST), provided the capacity to infer detailed evolutionary, demographic and spatial patterns from sequence data (Lemey et al., 2009a,b; Lemey, Rambaut et al. 2010) . However, in-depth studies of influenza virus evolution require large numbers of viral sequences carefully sampled over time and space, which are unavailable in many low-and middle-income countries. To address this gap, Fogarty's MISMS staff partnered with the National Institute of Allergy and Infectious Diseases (NIAID) and National Library of Medicine (NLM), to provide whole-genome sequencing of influenza virus collections free of cost via the Influenza Genome Sequencing Program (Box 3). When this sequencing project began in 2005, there were fewer than 100 complete influenza virus genomes available on GenBank. As of 2017, almost 20,000 influenza virus genomes have been sequenced through this project and made publicly available. A strong linkage between the Influenza Genome Sequencing Program and Fogarty's MISMS network of global researchers, and their viral collections, has helped address important gaps in influenza sequence availability and in turn transmission dynamics. The program targeted understudied aspects of influenza virus evolution, including large regions of the viral genome not typically sequenced, under-sampled populations in tropical settings, influenza B virus, and non-human mammalian hosts (Box 3). The program fostered a global culture of data-sharing that facilitates large-scale comparative studies across subtypes, regions, and species, advancing our fundamental understanding of influenza virus evolution and ecology.

Despite increased availability of influenza genetic and epidemiological data, there are still important gaps, particularly regarding surveillance in Africa and South America and at the animal-human interface (Viboud, Nelson et al. 2013) , and poor integration of different data streams (e.g., epidemiological, antigenic and genetic information), representing target areas of future MISMS efforts. Because influenza is unique in combining long-term pandemic risk with a continued need for annual vaccine updates to reduce the burden of seasonal influenza, the field has traditionally enjoyed far better attention than other respiratory infections. As genomic and epidemiological data for other respiratory pathogens increases, a natural extension for MISMS would be to broaden its scope beyond that of influenza. Recognizing this potential, MISMS collaborators have embarked on modeling studies of pneumococcus and respiratory syncytial virus, and their interactions with influenza (Weinberger, Simonsen et al. 2012; Shrestha, Foxman et al. 2013; Weinberger, Harboe et al. 2014; Shrestha, Foxman et al. 2015; Weinberger, Klugman et al. 2015) . With additional resources, a more systematic broadening of the scope of the MISMS study to other important human respiratory pathogens would provide fresh opportunities to enhance understanding of multiple disease systems while drawing on an established global research network and a wealth of computational tools.

Development of infectious disease models deeply rooted in empirical data can improve planning for, and response to, infectious disease threats; however, the link between modeling groups and policy makers remains tenuous in many countries (Heesterbeek, Anderson

The Influenza Genome Sequencing Project, a large-scale collaboration between NIH institutes and J. Craig Venter Institute, transforms our understanding of influenza virus evolution. et al. 2015 ; Metcalf, Edmunds et al. 2015) . Fogarty played an important role in promoting the field of modeling population-based impact of biological threats within the U.S. government. Immediately following the events of September 11 th , 2001 and before the anthrax-laced letters that began just one week later, DIEPS helped establish intergovernmental networks to model and address biological threats. These efforts contributed to launch an extramural modeling funding program on Models of Infectious Disease Agents (MIDAS) managed by the NIH National Institute of General Medical Studies and nascent funding to NIH Fogarty to study bioterror threats. In 2007, the RAPIDD program was established with funds from the Department of Homeland Security to build a hub-and-spokes network of infectious disease modelers working at the interface of policy and academic research, in close collaboration with government. The program's mandate was to advance US capacity in infectious disease modeling by pioneering new methodologies, studying key emerging disease threats, and training the next generation of scientists (Box 4). RAPIDD departed from traditional grant-based programs in its flexibility and concentration on working group and workshop activities, and on developing research capacity. A key success of RAPIDD since 2007 has been to promote collaborative research and synergies between modeling groups beyond core participants of the program (Box 1).

RAPIDD research has been at the forefront of responding to the greatest infectious disease crises of recent years, particularly the 2013-16 Ebola epidemic in West Africa and the Zika epidemic in the Americas. Modeling work helped to identify key routes of Ebola transmission and characterize effectiveness of intervention policies (Team, Aylward et al. 2014 , Camacho, Kucharski et al., 2015a , 2015b , Kucharski, Camacho et al., 2015 Team, Agua-Agum et al. 2015) . Further work, in collaboration with WHO's consortium for Ebola modeling, identified risk factors for transmission including severe symptoms, death, non-hospitalization, older age, and travel history (International Ebola Response Team et al., 2016) . Modeling work further demonstrated the unintended consequences of the Ebola-associated healthcare disruption on childhood immunization, and stressed the need to promote supplemental vaccination campaigns, particularly for measles (Takahashi et al., 2015) . Further, RAPIDD collaborators have developed predictive maps of the spread of the Zika virus in the Americas, driven by environmental conditions and population mobility (Bogoch, Brady et al. 2016) . Another RAPIDD modeling study analyzed factors shaping the efficacy of screening air travelers for emerging pathogens, including Ebola and MERS-coronavirus . Taken together, careful literature surveys of prior modeling work, in addition to primary research done by RAPIDD, form a substantial body of "case law" for infectious disease modeling, which can be used as reference to understand and model outbreaks caused by new pathogens with similar properties. As an important demonstration of this principle, prior modeling work on dengue provided the groundwork for rapid development of regional and global models for the spread of Zika virus (Bogoch, Brady et al. 2016 ).

During 2007-2015, RAPIDD functioned primarily via parallel working groups focused on four areas identified as having important research gaps: (i) methodological and data issues, (ii) zoonoses and pathogen emergence, (iii) vector-borne infections, and (iv) disease dynamics in small mammals. The working groups conducted primary research and led multi-disciplinary workshops on emerging research frontiers (Box 1). Since 2007, RAPIDD organized 118 workshops across a wide range of topics, with participation from thousands of scientists and government employees globally. Workshops produced large-scale reviews of the state-of-the-art of disease modeling in particular areas, and opinion pieces discussing new frontiers of model-data synthesis (Katz, Plowden et al. 2004; Buhnerkempe et al., 2015b; Frost, Pybus et al. 2015; Gog, Pellis et al. 2015; Heesterbeek, Anderson et al. 2015; Lloyd-Smith et al., 2015a,b; Metcalf et al., 2015a, b) . Working group members then pursued follow-up research to address the identified gaps by developing new modeling approaches or new applications of models to understudied but important diseases. The direct link between RA-PIDD research and policy was ensured by periodic presentations to the US White House scientific offices. Overall, since 2007, RAPIDD has made substantive contributions across a remarkable range of public health challenges (Box 2), from strategies to deploy current and future vaccines to the unique role of bats as zoonotic reservoirs. While a comprehensive review of the more than 1000 RAPIDD publications is beyond the scope of this review (Fig. 3, Table 1 ), below we single out a few achievements of the program.

Many of the most important emerging disease threats involve zoonotic pathogens originating from animal reservoirs. By conducting a seminal review of 442 published modeling studies of 85 zoonotic pathogens, RAPIDD researchers identified critical areas in need of further attention ). The review highlighted the marked predominance of studies of directly transmitted infections with simple life cycle (e.g., influenza, SARS), over infections involving multiple hosts with complex life cycles (vector-borne diseases, protozoans, food-borne infections). It identified a dearth of modeling research on looming threats including Ebola, chikungunya and yellow fever viruses, which was regrettably prescient in light of subsequent epidemics around the world (WHO, 2017). Crucially, the review noted that very few studies linked dynamics across the animal-human interface, where the defining process of zoonotic spillover occurs, and few Box 4 : Training the next generation of scientists: RAPIDD post-doctoral fellows.

• Outstanding fellows handpicked and hired for 2 years minimum, and up to 4. • Comfortable prestigious fellowship, with independent and flexible funding for travel. • Strong impetus to work with the larger RAPIDD network of outstanding scientists, particularly by organizing workshops and working groups, monthly Webex presentations, periodic policy-oriented seminars at the White House, and annual RAPIDD network meeting.

• Post-doctoral fellows helped drive major collaborative reviews of modeling gaps (eg, zoonoses and emerging infections, vector-borne diseases) -a superb bonding exercise leading to high-profile publications in their chosen fields.

• Community of RAPIDD fellows learned from each other on topics ranging from communication skills to methodological issues. • Of the 13 RAPIDD fellows, 11 are in tenured or tenure-track academic positions, 1 works for the US government and 1 for a private company (as of July 2018).

M.I. Nelson et al. Epidemics 26 (2019) 116-127 models incorporated evolutionary processes. To address these gaps, the zoonosis working group launched a series of methodological developments to link models across the animal-human interface, analyze the stuttering chains of transmission that foreshadow emergence events, and describe the evolutionary processes linked to pathogen emergence, with key applications in monkeypox, MERS-Coronavirus and influenza H5N1 and H7N9 viruses Rimoin, Mulembakani et al. 2010; Blumberg and Lloyd-Smith, 2013a, b; Park, Loverdo et al. 2013; Blumberg, Enanoria et al. 2014; Chowell, Blumberg et al., 2014; Kucharski and Althaus, 2015; Kucharski, Mills et al. 2015; Lloyd-Smith et al., 2015a,b; Plowright, Parrish et al. 2017) . Similarly, the RAPIDD vector-borne disease working group's review of 388 studies of mosquito-borne pathogen transmission determined that most models were still primarily derived from mid-20th century methods pioneered by Ross and Macdonald Smith et al., 2014) . In response, the group developed more refined models to capture important features of mosquito-borne infections, including mosquito life history, heterogeneous biting and fine-scale spatio-temporal variation in transmission. Applying these methods to malaria and dengue highlighted the impact of incorporating realistic mosquito biology and biting heterogeneities on control efforts Perkins, Garcia et al. 2014; Reiner, Le Menach et al. 2015) .

In recent years, a major focus of RAPIDD research has been infectious disease forecasting, responding to a rising need in the public health community, particularly for influenza and emerging infections Yang, Cowling et al. 2015; Yang et al., 2015a; Yang, Olson et al. 2016 ). In addition to RAPIDD's work on evaluating the impact of interventions in the midst of the West African Ebola epidemic, RAPIDD launched an after-the-fact forecasting challenge using synthetic epidemiological data sets to assess model performances in a controlled environment (Merler, Ajelli et al. 2017; Viboud et al., 2017a, b) . This unprecedented group effort revealed that the most accurate disease forecasts stemmed from ensemble predictions combining outputs from different models, since even the best models have weaknesses, and that prediction accuracy does not scale with model complexity. The forecasting challenge also strengthened communication and collaboration between modeling groups, laying the foundation for more effective response for the next public health crisis (Viboud et al., 2017a (Viboud et al., , 2017b .

Influenza forecasting represents an area of rich synergy between the RAPIDD and MISMS programs. A major aim of ongoing influenza research is to predict the emergence and trajectory of new seasonal strains, and hence improve the imperfect semi-annual vaccine strain selection process. Forecasting work in this area was greatly enhanced by a 2016 workshop, organized in collaboration with WHO, that highlighted the need for better flow of genetic and antigenic information from public health agencies to the institutions and mathematical modelers to improve prediction accuracy and horizon (Morris, Gostic et al. 2017) . Further, RAPIDD and MISMS collaborators were the first to analyze the potential benefits of broadly cross-protective influenza vaccination programs (Arinaminpathy, Ratmann et al. 2012; Subramanian, Graham et al. 2016 ). This is a particularly fruitful area of overlap between RAPIDD and MISMS, as universal influenza vaccine candidates enter pre-clinical trials, and an ideal opportunity to integrate network members' expertise on relevant evolutionary and immunological concepts with the global source-sink dynamics so unique to influenza Russell, Jones et al. 2008 ).

Although the four RAPIDD working groups had separate areas of focus, several synergistic themes rapidly emerged (Box 5). One dominant theme was the spatiotemporal dynamics of pathogens, encompassing major strides in spatial dynamics and mapping of vector-borne infections, a broad effort to advance data-driven models for foot-andmouth disease in the US and abroad, and cutting-edge syntheses of novel and traditional data streams informing human demographics and mobility to produce a new generation of spatio-temporal epidemic models. Another major RAPIDD theme on pathogen evolution -entailing phylodynamics, virulence evolution, host jumps and drug resistance -has generated methodological breakthroughs and important applications to influenza, dengue, malaria, and more. Focused workshops re-examined current thinking on optimal drug treatment to minimize resistance evolution (Read, Day et al. 2011; Kouyos, Metcalf et al. 2014) , and the potential to predict influenza pandemic risk from viral sequence data Lipsitch, Barclay et al. 2016) . 

Bibliometric analysis of the RAPIDD and MISMS programs. RAPIDD publications are based on any mention of the RAPIDD program in acknowledgments; this includes fellows and faculty funded by RAPIDD and/or participation to RAPIDD workshops that led to publications. MISMS publications are based on influenza research articles published by staff of the Fogarty International Center, as listed in affiliations. Data collection performed on July 1, 2018. 

Since 2007 RAPIDD has advanced the methods and applications of data-driven modeling of infectious diseases, with a clear focus on policy-relevant research, and has fostered a new generation of talented researchers in this field (Box 4). The program's remarkable success arises from strong collaborations developed among the many participants over the years, its flexibility and agility in convening workshops on emerging research frontiers, and from providing freedom and unparalleled networking opportunities for talented post-doctoral scientists. Looking forward, pending availability of funds, RAPIDD will continue its research emphases on emerging infections and vectorborne diseases, while growing new areas of focus on topical challenges such as on modeling the dynamics of anti-microbial resistance, and predicting the impact of next-generation vaccines such as for respiratory syncytial virus, malaria, typhoid, dengue, and the universal influenza vaccine.

The Fogarty-led MISMS and RAPIDD programs have been catalysts in advancing state-of-the-art global infectious disease modeling and training with tangible impacts in building capacity globally and guiding policies, such as defining age priority groups for epidemic and pandemic influenza vaccines, addressing optimal delivery of influenza vaccines in tropical and low-income countries, and highlighting gaps in measles vaccine coverage following the Ebola outbreak in West Africa. In addition to influencing policy, a major success of the MISMS and RAPIDD programs is reflected by the outstanding professional trajectories of their alumni, domestically and internationally, and the excellence and international scope of their publications (Table 1 , Box 4). Of the more than 1400 publications of the MISMS and RAPIDD programs, which have been cited over 50,000 times, 28% include authors from LMICs. Many of the RAPIDD and MISMS trainees and collaborators are now established as senior researchers.

Both programs rely on global multi-disciplinary networks to identify and address research questions at the forefront of infectious disease transmission and evolution. Both programs build off the "big data" revolution that provides increasingly detailed and abundant information on disease patterns and host behavior, leveraging a surge of pathogen genetic sequence data, exquisitely detailed epidemiological information derived from digital and social media, and human mobility and demographic proxies derived from mobile phones or remote sensing. New methodological approaches are needed to handle such a vast and diverse amount of information. Further, both programs have long histories of promoting data sharing and development of publicly available methodological tools, with key contributions to the Influenza Genome Sequencing Program and Database, the BEAST package for phylogenetic and associated analyses, and development of new transmission modeling packages (POMP, TSIR) and spatial analysis software (Epipoi) (Alonso and McCormick, 2012; Becker and Morris, 2016; King, 2018) The MISMS and RAPIDD programs are also well connected with parallel efforts at NIH and CDC and in the broader scientific community. For instance, within NIH, there are strong synergies between RAPIDD and MIDAS, a funding program in infectious disease modeling managed by the National Institute for General Medical Sciences, and between MISMS and the Centers of Excellence in Influenza Research and Surveillance (CEIRS), funded by the National Institute of Allergy and Infectious Diseases. In parallel, there are strong ties between RAPIDD and MISMS and the CDC's influenza division, involving joint workshops, forecasting challenges (e.g., (Merler, Ajelli et al. 2017; Viboud et al., 2017a Viboud et al., , 2017b ) and post-doctoral exchanges. Further, there is a tight link between MISMS and RAPIDD and national and global public health agencies, with regular participation of collaborators in the White House pandemic preparedness working group, US government modeling coordination groups, and WHO modeling groups. Several of the MISMS workshops have been co-organized with, and in support of, foreign government agencies (Fig. 1) , while many international government representatives have taken part in MISMS and RA-PIDD workshops.

Despite these successes, some challenges are worth noting. The most

Key aspects of diseases dynamics investigated by RAPIDD.

problematic is the lack of sustainable long-term funding for these programs more than 10 years after they were created, even though their combined annual costs are under 2M$, which is negligible relative to the costs of public health crises (see (Office of Science, 2016; World Bank and Ecohealth Alliance, 2018) for a discussion of the value of infectious disease preparedness and modeling). It is worth nothing however that the low cost of computational modeling does not include funding for generation of biomedical, epidemiological and surveillance data, which are essential to support modeling work. And with their extended duration and distributed network structure, the RAPIDD and MISMS programs fall outside standard funding streams, despite their extraordinary efficiency in propelling leading-edge science and addressing problems of national interest (Box 2). As the Ebola and Zika outbreaks clearly demonstrate, infectious disease threats are not in retreat Advances in multi-pathogen diagnostics and sequencing will facilitate real-time molecular epidemiology during future outbreaks, but will require people on the ground and international partners who can rapidly interpret large amounts of data. Development of new vaccines (e.g., respiratory syncytial virus, universal influenza) and the growing challenge of anti-microbial resistance will require improved models to optimize public health policies and understand pathogen evolutionary responses.

In conclusion, the MISMS and RAPIDD programs illustrate the power of scientific diplomacy and collaborative research networks, with demonstrable successes in improving research and control of infectious diseases. We see no shortage of policy-relevant research themes for these networks to explore, including the emergence of new threats at the animal-human interface, microbial interactions within and between hosts, integration of traditional and novel data streams into ever more sophisticated models, short-and long-term disease forecasts, projections of the impact of novel vaccines, and anti-microbial resistance. Fogarty-led disease modeling programs will continue to strengthen capacity in LMICs for outbreak analysis, and in turn help control emerging infectious disease threats domestically and internationally.

This article is dedicated to our Fogarty colleague, Dr. Ellis McKenzie, who was instrumental in launching, growing and shepherding the RAPIDD program from 2007 until his death in 2016. Ellis played a key role in promoting use of infectious disease models in government and mentored countless infectious disease modelers. The establishment and success of the program would have been impossible without Ellis's gentle and inspired guidance.

The RAPIDD program is indebted to the leadership, vision, and guidance of two outstanding and incredibly generous scientists who shepherded this program since 2007, Bryan Grenfell and Ellis McKenzie.

We acknowledge support from the Department of Homeland Security and FIC for the RAPIDD program, and the HHS Pandemic Threat Unit, Office of Global Affairs and FIC for the MISMS program. We are grateful to Felix Wu, Amherst College, for performing a thorough bibliometric analysis of the RAPIDD and MISMS publications and generating maps of co-authors. We are thankful to Kate Skoczdopole for continued administrative support of the RAPIDD and MISMS programs.

The funders did not play a role in study design; collection, analysis, and interpretation of data; or in the writing the manuscript. The authors do not report any financial and personal conflicts of interest.

Spatial dynamics, mobility, and disease incidence mapping

Integration of novel data-streams, including social and behavioral sciences

Eradication and elimination end-game, persistence, and vaccine refusal

New modeling approaches: statistical fitting algorithms, disease and strain forecasting, and next-generation models for vector-borne diseases (He, Ionides

Inference of transmission dynamics from pathogen sequence data

Evolution of virulence, adaptation to new host species, and drug resistance

Subcritical outbreak dynamics and pandemic risk assessment

Inference of disease dynamics and exposure history from serology

Disease transmission in agricultural settings including poultry and aquaculture

Incidence and risk factors for influenza-like-illness in the UK: online surveillance using Flusurvey

Influenza seasonality in Madagascar: the mysterious African free-runner. Influenza Other Respir

EPIPOI: a user-friendly analytical tool for the extraction and visualization of temporal parameters from epidemiological time series

Seasonality of influenza in Brazil: a traveling wave from the Amazon to the subtropics

A global map of hemispheric influenza vaccine recommendations based on local patterns of viral circulation

Epidemiologic characterization of the 1918 influenza pandemic summer wave in Copenhagen: implications for pandemic control strategies

Impact of cross-protective vaccines on epidemiological and evolutionary dynamics of influenza

Influenza seasonality goes south in the Yucatan Peninsula: the case for a different influenza vaccine calendar in this Mexican region

Big data for infectious disease surveillance and modeling

tsiR: An Implementation of the TSIR Model

Explaining seasonal fluctuations of measles in Niger using nighttime lights imagery

The global distribution and burden of dengue

Measurably evolving pathogens in the genomic era

Latitudinal variations in seasonal activity of influenza and respiratory syncytial virus (RSV): a global comparative review

Natality decline and miscarriages associated with the 1918 influenza pandemic: the Scandinavian and United States experiences

Identifying postelimination trends for the introduction and transmissibility of measles in the United States

Comparing methods for estimating R0 from the size distribution of subcritical transmission chains

Inference of R(0) and transmission heterogeneity from the size distribution of stuttering chains

Anticipating the international spread of Zika virus from Brazil

Mapping influenza transmission in the ferret model to transmission in humans

Eight challenges in modelling disease ecology in multi-host, multi-agent systems

Temporal changes in ebola transmission in Sierra Leone and implications for control requirements: a real-time modelling study

Human mobility and the spatial transmission of influenza in the United States

Severe respiratory disease concurrent with the circulation of H1N1 influenza

Synthesizing data and models for the spread of MERS-CoV, 2013: key role of index cases and hospital transmission

Recrudescent wave of pandemic A/H1N1 influenza in Mexico, winter 2011-2012: age shift and severity

Characterizing the epidemiology of the 2009 influenza A/ H1N1 pandemic in Mexico

Adaptive vaccination strategies to mitigate pandemic influenza: mexico as a case study

Mathematical modeling of the west africa ebola epidemic

The evolutionary emergence of pandemic influenza

The dilemma of influenza vaccine recommendations when applied to the tropics: the Brazilian case examined under alternative scenarios

Virus genomes reveal factors that spread and sustained the Ebola epidemic

The evolutionary genetics and emergence of avian influenza viruses in wild birds

Establishment and cryptic transmission of Zika virus in Brazil and the Americas

Multiyear surveillance for avian influenza virus in waterfowl from wintering grounds

Eight challenges in phylodynamic inference

Mixed infection and the genesis of influenza virus diversity

Spatial transmission of 2009 pandemic influenza in the US

Seven challenges in modeling pathogen dynamics within-host and across scales

Potent protection against H5N1 and H7N9 influenza via childhood hemagglutinin imprinting

Effectiveness of traveller screening for emerging pathogens is shaped by epidemiology and natural history of infection

Mortality attributable to influenza in England and Wales prior to, during and after the 2009 pandemic

Plug-and-play inference for disease dynamics: measles in large and small populations as a case study

Modeling infectious disease dynamics in the complex landscape of global health

Extensive geographical mixing of 2009 human H1N1 influenza A virus in a single university community

Whole-genome analysis of human influenza A virus reveals multiple persistent lineages and reassortment among recent H3N2 viruses

Transmission of equine influenza virus during an outbreak is characterized by frequent mixed infections and loose transmission bottlenecks

Exposure patterns driving ebola transmission in West Africa: a retrospective observational study

Immunity to influenza: the challenges of protecting an aging population

Modelling vaccination strategies against foot-and-mouth disease

Potential drivers of virulence evolution in aquaculture

Evolutionary history and attenuation of myxoma virus on two continents

Simulation-based Inference for Epidemiological Dynamics

Six challenges in the eradication of infectious diseases

The path of least resistance: aggressive or moderate treatment?

The role of superspreading in Middle East respiratory syndrome coronavirus (MERS-CoV) transmission

Measuring the impact of Ebola control measures in Sierra Leone

Transmission potential of influenza a(H7N9) virus, China

Big data. The parable of Google Flu: traps in big data analysis

Bayesian phylogeography finds its roots

Phylogeography takes a relaxed random walk in continuous space and time

Reconstructing the initial global spread of a human influenza pandemic: a Bayesian spatial-temporal model for the global spread of H1N1pdm

Evidence for antigenic seniority in influenza A (H3N2) antibody responses in southern China

Nine challenges in modelling the emergence of novel pathogens

Epidemic dynamics at the human-animal interface

Challenges in modelling infectious disease dynamics: preface

Synthetic model for the RAPIDD Ebola Challenge

Five challenges in evolution and infectious diseases

Six challenges in modelling for public health policy

The signature features of influenza pandemics-implications for policy

Prioritization of influenza pandemic vaccination to minimize years of life lost

Predictive modeling of influenza shows the promise of applied evolutionary biology

Molecular epidemiology of A/H3N2 and A/H1N1 influenza virus during a single epidemic season in the United States

Spatial dynamics of humanorigin h1 influenza a virus in north american Swine

Phylogenetic analysis reveals the global migration of seasonal influenza A viruses

The origin and global emergence of adamantane resistant A/H3N2 influenza viruses

Multiple reassortment events in the evolutionary history of H1N1 influenza A virus since 1918

Towards Epidemic Predition: Federal Efforts and Opportinities in Outbreak Modeling. Report of the Pandemic Prediction and Forecasting Science and Technology Working Group

Multiple scales of selection influence the evolutionary emergence of novel pathogens

Inferring infection hazard in wildlife populations by linking data across individual and population scales

Identifying genetic markers of adaptation for surveillance of viral host jumps

Minimizing the threat of pandemic emergence from avian influenza in poultry systems

Theory and data for simulating fine-scale human movement in an urban environment

Pathways to zoonotic spillover

Real-time, portable genome sequencing for Ebola surveillance

The genomic and epidemiological dynamics of human influenza A virus

Phylodynamic inference for structured epidemiological models

Imperfect vaccination can enhance the transmission of highly virulent pathogens

The evolution of drug resistance and the curious orthodoxy of aggressive chemotherapy

A systematic review of mathematical models of mosquito-borne pathogen transmission

Socially structured human movement shapes dengue transmission despite the diffusive effect of mosquito dispersal

Mapping residual transmission for malaria elimination

Major increase in human monkeypox incidence 30 years after smallpox vaccination campaigns cease in the Democratic Republic of Congo

Influenza-related mortality in the Italian elderly: no decline associated with increasing vaccination coverage

The global circulation of seasonal influenza A (H3N2) viruses

Improving pandemic influenza risk assessment

Real-time influenza forecasts during the 2012-2013 season

Inference and forecast of the current west african ebola outbreak in Guinea, sierra leone and liberia

The role of influenza in the epidemiology of pneumonia

Identifying the interaction between influenza and pneumococcal pneumonia using incidence data

Infectious disease surveillance in the big data era: towards faster and locally relevant systems

Benefits of influenza vaccination on influenza-related mortality among elderly in the US: an unexpected finding. Options for the Control of Influenza V

Impact of influenza vaccination on seasonal mortality in the US elderly population

The need for interdisciplinary studies of historic pandemics

The genesis and spread of reassortment human influenza A/H3N2 viruses conferring adamantane resistance

Influenza vaccination and mortality benefits: new insights, new opportunities

Recasting the theory of mosquitoborne pathogen transmission dynamics and control

House-to-house human movement drives dengue virus transmission

Universal or Specific? A Modeling-Based Comparison of Broad-Spectrum Influenza Vaccines against Conventional, Strain-Matched Vaccines

Reduced vaccination and the risk of measles and other childhood infections post-Ebola

Global influenza seasonality: reconciling patterns across temperate and tropical regions

West African Ebola epidemic after one year-slowing but not yet under control

Ebola virus disease in West Africa-the first 9 months of the epidemic and forward projections

Contrasting the epidemiological and evolutionary dynamics of influenza spatial transmission

The RAPIDD Ebola forecasting challenge special issue: preface. Epidemics

The RAPIDD ebola forecasting challenge: synthesis and lessons learnt

Pneumococcal disease seasonality: incidence, severity and the role of influenza activity

Association between respiratory syncytial virus activity and pneumococcal disease in infants: a time series analysis of US hospitalization data

Impact of the 2009 influenza pandemic on pneumococcal pneumonia hospitalizations in the United States

Connecting mobility to infectious diseases: the promise and limits of mobile phone data

Connecting mobility to infectious diseases: the promise and limits of mobile phone data

Quantifying the impact of human mobility on malaria

Yellow Fever in Brazil: Update

Epidemiology. Foot-and-mouth disease under control in the UK

Lessons from Ebola: improving infectious disease surveillance to inform outbreak management

A synchronized global sweep of the internal genes of modern avian influenza virus

Forecasting influenza epidemics in Hong Kong

Forecasting influenza outbreaks in boroughs and neighborhoods of New York City

Transmission network of the 2014-2015 Ebola epidemic in Sierra Leone

Characterization of regional influenza seasonality patterns in China and implications for vaccination strategies: spatio-temporal modeling of surveillance data