key: cord-0008158-m389kfgr authors: Wang, Yanbai Andrea; Barry, Michele title: Making Online Outbreak Surveillance Work for all date: 2017-11-06 journal: Ann Glob Health DOI: 10.1016/j.aogh.2017.09.002 sha: ebf8b03fb40e95fbc9c4e3be78bf8ceed0e84ed3 doc_id: 8158 cord_uid: m389kfgr nan Delayed detection and reporting in recent epidemics such as Zika and Ebola have brought renewed attention to outbreak surveillance. In particular, the use of digital technologies to crowd-source and interpret volumes of public information from the Internet for signs of outbreaks-so-called event-based Internet surveillance (EBIS)-has received great interest because of its potential for early detection. The National Academy of Medicine suggests that new information technology, which has increased surveillance capacity even in lowresource settings, be fully exploited. 1 The United Nations recommends that the World Health Organization (WHO) create an open platform to manage and analyze public data on unusual health events globally. 2 Developing the capacity for event-based surveillance is also required under the 2005 International Health Regulations (2005 IHR), 3 an international law that binds 196 states worldwide. The growing significance of EBIS in global health raises questions concerning its efficacy, accessibility, and impact. Technical challenges, unreliable funding, institutional exclusivity, and legal shortcomings combine to generate uneven effects across the globe. Emerging in the 1990s with the proliferation of the Internet, EBIS has since become a patchwork of overlapping tools created by a multiplicity of actors and institutions principally in the developed world. Whereas traditional indicator-based surveillance relies on structured information such as case reports and laboratory results, EBIS uses diverse sources of public information from the Internet such as news articles, messages on forums, information on health agency websites, individual investigator reports, blogs, search engine queries, and social media. These disorganized raw signals, containing information of varied quality, are gathered in select languages. They are processed through machine translation and automated textual analysis that discards irrelevant documents and extracts key data about potentially relevant events. Human analysis may then be employed to determine which events warrant further investigation. The most widely used EBIS tools today are Canada's Global Public Health Information Network (GPHIN), the International Society for Infectious Diseases' Program for Monitoring Emerging Diseases (ProMED-mail), the European Commission's Medical Information System (MedISys), and Harvard University's HealthMap. Each has unique strengths and weaknesses as a result of different information acquisition protocol, subject matter scope, geographic and linguistic coverage, funding base, and access rules (Table 1) . GPHIN is only accessible to organizations with a public health mandate, whereas the others are freely available to the public. GPHIN and ProMEDmail are the most human-driven systems, with expert analysts that moderate content. HealthMap has some human input, whereas MedISys is fully automated. Many more systems exist, 4 and the landscape remains volatile with substantial turnover. By looking at data from outside the public health system, EBIS can detect illness before or in the absence of medical treatment, monitor for unknown contagions that may elude traditional surveillance, and overcome transparency problems when outbreaks are covered up by governments. Severe acute respiratory syndrome, for example, was first detected in November 2002 when GPHIN picked up rumors of an outbreak affecting large numbers of schoolchildren in mainland China. 5 A 2011 WHO report noted that 35% of outbreak events were first reported through open sources. 6 Yet extracting epidemiologic intelligence from voluminous data of varying reliability is complicated and the field remains in its infancy. Existing tools pick up information quickly, but they are limited in their linguistic capabilities and suffer from low specificity. The bulk of detected signals-97% according to one study-are in 7 major languages (English, French, Spanish, Portuguese, Arabic, Russian, and Chinese), 7 raising questions about the representativeness of the information captured. Using a prospective dataset that approximates real-life surveillance conditions, another study found that the pooled data from 6 EBIS tools detected confirmed human cases of influenza A/H5N1 earlier than official reporting 93% of the time. 8 However, those timely true positives constituted only 3% of all signals, whereas the remaining were false positives and late detects. EBIS tools have increasingly sought to incorporate insights from data sources such as Twitter, Facebook, and search engine queries. These sources come with their own pitfalls, as exemplified by Google Flu Trends, which used the frequency of search terms to track influenza activity. After some initial success, 9 Google Flu Trends missed the influenza A/H1N1 pandemic in 2009 and drastically overestimated seasonal flu levels in winter 2012-2013. 10 One problem was overfitting: 50 million candidate Google search queries were tested for fit with historic data on influenza activity, so the odds were high that some of the queries would match by chance. 11 Another problem was that Google's search algorithm as well as user behavior changed over time, so the system needed to be continuously recalibrated. 12 These challenges are not insurmountable, but they underscore the complexity of the task and the resources needed. Real-time feedback from public health practitioners can refine data analysis. The sensitivity, specificity, and positive predictive value of MedISys were raised by revising search strategies in light of practitioner input. 13 Moreover, EBIS tools work better with human moderation, 7 and they are more effective when used together rather than in isolation. The detection rate approximately doubled when 6 EBIS tools were combined, and overall timeliness improved. 8 Another study found that none of the examined tools could individually detect all studied events before official reporting. 7 Integration is likewise needed to reduce duplication from cross feeding (ie, when one tool draws data from another) and to standardize rumor verification protocol. Integrating EBIS tools owned and paid for by different countries and organizations poses challenging institutional questions. One question is who should finance integration efforts. In many cases, individual tools lack stable funding to begin with, thus the aforementioned high turnover rate. Prominent EBIS tools such as Japan's BioCaster, the University of Helsinki's PULS, and Georgetown University's Argus have each become inactive over time or are no longer maintained for this reason. Funding an integrated international platform is even more complicated. Another question is how to work across institutional and national boundaries. For the time being, that question has been answered by framing infectious disease as a national security priority. The most promising integration project so far is the Early Alerting and Reporting (EAR) portal launched in 2009 by the Global Health Security Initiative-an international partnership formed by the G7 countries plus Mexico to address chemical, biological, and radionuclear terrorism as well as other security threats. The portal currently pools data from GPHIN, HealthMap, MedISys, and ProMED-mail. Reports concerning more than a million events around the world are automatically sorted every week for public health relevance. A network of bioanalysts from participating countries rotate to conduct a standardized risk analysis that assesses the event's impact, the trustworthiness of the source, and whether the event is suspected to be a deliberate act. 14 Based on this analysis, bioanalysts can alert portal users, namely the G7 countries, Mexico, the European Commission, and the WHO. The WHO has its own outbreak verification process, whereas the other users have a collaborative information-sharing process built on trust. The portal is supported through voluntary and in-kind contributions from the countries involved. Although casting outbreaks as a national security issue has attracted political attention and funding, doing so comes with its perils. Information gathered by EBIS is publicly available on the Internet, yet it can subsequently become classified because of security clearance procedures. The involvement of military actors not accustomed to the rapid and open sharing of data and risk assessments across borders can impair the trust and institutional culture needed for efficient global health collaboration. The EAR portal is currently only accessible to members of the Global Health Security Initiative, although discussions are ongoing to expand its availability. If and when the portal becomes open to a more diverse group of countries-including those with divergent security agendas-its emphasis on terrorism may prove problematic. The inclusion of terrorism within the scope of the 2005 IHR was hotly contested during the law's negotiation, 15 with opposition from countries across Europe, the Middle East, and Asia that worried that focus on biological weapons would detract from the WHO's core public health mandate. Foreign policy interests may conflict with public health objectives, distort the values at stake, and undermine the legitimacy of the initiative. EBIS has been touted by medical and policy experts to hold particular value for developing countries. Because EBIS does not rely on people seeking medical treatment, the percentage of the population assessable via Internet-based surveillance is far higher than that via traditional surveillance given the Internet usage of an average low-income country. 16 Retrospective studies suggest that data gathered from Internet sources correlated highly with epidemic curves during the 2014 Ebola outbreak in West Africa 17 and would have been available up to 2 weeks earlier than official case data during the 2010 cholera outbreak in Haiti. 18 However, in addition to expanding access to the EAR portal, there is much to be done to make EBIS more valuable in low-income settings. EBIS tools have been primarily built and tested for users in developed countries. Existing studies on EBIS focus mainly on a handful of diseases that are of interest to the developed world, particularly influenza. They also use official reporting by the WHO or industrialized countries as the gold standard for verified outbreaks. Studies in a wider range of contexts are needed to gain more generalizable insights and to devise more broadly applicable metrics for evaluating EBIS performance. The low specificity of EBIS poses particular difficulties for low-income countries. This is because the 2005 IHR's requirement that signatory states develop event-based surveillance capability is one aspect of a broader strategy, other parts of which have encountered implementation problems. The 2005 IHR requires that signatory states build the capacity to promptly and effectively respond to outbreaks by 2012, with possible extensions to 2014 and 2016. According to the latest detailed WHO report, only a third of countries worldwide had attained the law's national public health capacity requirements as of late 2014. 19 The 2005 IHR requires the WHO to help countries respond to outbreaks when they request it. Yet recent publications caution that neither the WHO nor the international community at large has the capacity to provide the needed support and coordination. 1, 2, 6, 20 The 2005 IHR limits the travel and commercial restrictions that signatory states may implement to protect themselves from public health risks abroad. But noncompliance has been rife. 20 In short, it is not enough to detect early warnings of outbreaks, especially because the level of false positive signals is high. Many developing countries lack the infrastructure to conduct followup verification and response efforts, and the WHO currently lacks the resources to assist them. Without an effective verification and response system, the media attention resulting from an early warning can cause unnecessary economic havoc as other countries impose excessive restrictions to international trade and travel, or can lead to a diversion of scarce resources from basic health and nutrition programs to infectious disease control that may not make sense for the local population. An international funding mechanism is needed to safeguard the continued existence and advancement of EBIS, to provide access to all countries, and to strengthen outbreak verification and response capacity at national and international levels. EBIS is here to stay and has the potential to more evenly benefit the entire global community. We must ensure that the tools as well as the super system that is now taking shape are inclusive and stably funded, foster openness and trust, and meet the varied needs of countries worldwide. The Neglected Dimension of Global Security: A Framework to Counter Infectious Disease Crises. Washington, DC: Commission on a Global Health Risk Framework for the Future Protecting Humanity from Future Health Crises Core Capacity Workbook: A Series of Exercises to Assist the Validation of Core Capacity Implementation Levels New technologies in predicting, preventing and controlling emerging infectious diseases Coordinating the global response Implementation of the International Health Regulations World Health Organization Factors influencing performance of internet-based biosurveillance systems used in epidemic intelligence for early detection of infectious diseases outbreaks Evaluation of epidemic intelligence systems integrated in the early alerting and reporting project for the detection of A/H5N1 influenza events Detecting influenza epidemics using search engine query data Reassessing Google Flu Trends data for detection of seasonal and pandemic influenza: a comparative epidemiological study at three geographic scales The parable of Google flu: traps in big data analysis When Google got flu wrong How to Maximise Event-Based Surveillance Web-Systems: the Example of ECDC/ JRC Collaboration to Improve the Performance of MedISys Interfacing a biosurveillance portal and an international network of institutional analysts to detect biological threats Coincident polio and ebola crises expose similar fault lines in the current global health regime Internet-based surveillance systems for monitoring emerging infectious diseases Role of big data in the early detection of Ebola and other emerging infectious diseases Social and news media enable estimation of epidemiological patterns early in the 2010 Haitian cholera outbreak Report of the Review Committee on Second Extensions for Establishing National Public Health Capacities and on IHR Implementation Add1-en_IHR_RC_Second_exten sions Report of the Ebola Interim Assessment Panel All authors had access to the data and a role in writing the manuscript. The authors confirm that they have no conflicts of interest to declare. From the Stanford Law School, Stanford, CA (YAW); and Center for Innovation in Global Health, Stanford School of Medicine, Stanford, CA (MB). The authors thank Louise Boily, Clark Freifeld, Ann Marie Kimball, Jens Linge, Larry Madoff, and Abla Mawudeku for reviewing and commenting on drafts of the manuscript.