key: cord-0740269-9q5zckir
authors: Becker, Daniel J.; Albery, Gregory F.; Sjodin, Anna R.; Poisot, Timothée; Bergner, Laura M.; Dallas, Tad A.; Eskew, Evan A.; Farrell, Maxwell J.; Guth, Sarah; Han, Barbara A.; Simmons, Nancy B.; Stock, Michiel; Teeling, Emma C.; Carlson, Colin J.
title: Optimizing predictive models to prioritize viral discovery in zoonotic reservoirs
date: 2021-08-05
journal: bioRxiv
DOI: 10.1101/2020.05.22.111344
sha: bc5fbe843fdf3d2bd304b59a0cfcac63661c2843
doc_id: 740269
cord_uid: 9q5zckir

Despite global investment in One Health disease surveillance, it remains difficult—and often very costly—to identify and monitor the wildlife reservoirs of novel zoonotic viruses. Statistical models can be used to guide sampling prioritization, but predictions from any given model may be highly uncertain; moreover, systematic model validation is rare, and the drivers of model performance are consequently under-documented. Here, we use bat hosts of betacoronaviruses as a case study for the data-driven process of comparing and validating predictive models of likely reservoir hosts. In the first quarter of 2020, we generated an ensemble of eight statistical models that predict host-virus associations and developed priority sampling recommendations for potential bat reservoirs and potential bridge hosts for SARS-CoV-2. Over more than a year, we tracked the discovery of 40 new bat hosts of betacoronaviruses, validated initial predictions, and dynamically updated our analytic pipeline. We find that ecological trait-based models perform extremely well at predicting these novel hosts, whereas network methods consistently perform roughly as well or worse than expected at random. These findings illustrate the importance of ensembling as a buffer against variation in model quality and highlight the value of including host ecology in predictive models. Our revised models show improved performance and predict over 400 bat species globally that could be undetected hosts of betacoronaviruses. Although 20 species of horseshoe bats (Rhinolophus spp.) are known to be the primary reservoir of SARS-like viruses, we find at least three-fourths of plausible betacoronavirus reservoirs in this bat genus might still be undetected. Our study is the first to demonstrate through systematic validation that machine learning models can help optimize wildlife sampling for undiscovered viruses and illustrates how such approaches are best implemented through a dynamic process of prediction, data collection, validation, and updating.

Introduction Identifying likely reservoirs of zoonotic pathogens is challenging 1 . Sampling wildlife for the presence of active or recent infection (i.e., seropositivity) represents the first stage of a pipeline for proper inference of host species 2 , but sampling is often limited in phylogenetic, temporal, and spatial scale by logistical constraints 3 . Given such restrictions, statistical models can play a critical role in helping to prioritize pathogen surveillance by narrowing the set of plausible sampling targets, by either ruling out clades of low-likelihood hosts 4, 5 or predicting high-risk clades 6 . For example, machine learning approaches have generated candidate lists of likely, but unsampled, primate reservoirs of Zika virus, bat reservoirs of filoviruses, and avian reservoirs of Borrelia burgdorferi [7] [8] [9] .

At the same time, host predictions are rarely validated empirically 10 . Occasional case studies suggest both success and failures. For example, models predicted Eonycteris spelaea as an undetected bat host of filoviruses 7 , which was later confirmed through field sampling in southeast Asia 11, 12 . Similarly, models of mosquito-Zika virus interactions predicted Culex quinquefasciatus as a likely vector 13 , which was rapidly validated through experimental competence trials 14, 15 . A recent model of Nipah virus in India also predicted several bat species as undetected hosts 2 . However, experimental infection of the predicted Rousettus aegyptiacus demonstrated that this species cannot support virus replication 16 . Further, Nipah virus was recently found circulating in Pipistrellus pipistrellus, a species with a low predicted probability of being a host 17 . More generally, predictions from most models remain either untested or opportunistically validated, limiting insight into which approaches have greatest predictive accuracy. Systematically validating predictions would provide critical insights into the broader utility (or inefficacy) of different models in zoonosis research. Moreover, these modeling approaches are generally developed in isolation; implementation of multiple modeling approaches collaboratively and simultaneously, as part of a model-to-validation workflow, could reduce redundancy and apparent disagreement at the earliest stages of pathogen tracing while advancing predictive analytics by addressing inter-model reliability.

Coronaviruses (CoVs) are an ideal family of viruses with which to compare and validate predictive models of likely zoonotic reservoirs. CoVs are positive-sense, single-stranded RNA viruses that are detected across mammals and birds 18 . They have a broad host range, a high mutation rate, and the largest genomes of any RNA viruses, but they have also evolved mechanisms for RNA proofreading and repair to mitigate the deleterious effects of a high recombination rate acting over a large genome 19 . Consequently, CoVs fit the profile of viruses with high zoonotic potential. There are eight human CoVs (three and five in the genera Alphacoronavirus and Betacoronavirus, respectively), of which three are highly pathogenic in humans: SARS-CoV, MERS-CoV, and SARS-CoV-2. These are zoonotic and widely agreed to have evolutionary origins in bats [20] [21] [22] [23] .

Our collective experience with both SARS-CoV and MERS-CoV illustrates the difficulty of tracing specific animal hosts of emerging viruses. During the 2002-2003 SARS epidemic, SARS-CoV was traced to the masked palm civet (Paguma larvata) 24 , but the ultimate origin remained unknown for several years. Horseshoe bats (family Rhinolophidae: genus Rhinolophus) were implicated as reservoir hosts in 2005, but their SARS-like CoVs were not identical to circulating human strains 21 Coronavirus disease 2019 (COVID-19) is caused by SARS-CoV-2, a novel virus with presumed evolutionary origins in bats. Although the earliest cases were linked to a wildlife market 23 , contact tracing was limited, and there has been no definitive identification of the wildlife contact that resulted in spillover nor a true "index case." The divergence time between SARS-CoV-2 and two of the closest-related bat viruses (RaTG13 from Rhinolophus affinis and RmYN02 from Rhinolophus malayanus) has been estimated as 40-50 years 33 , suggesting that the main host(s) involved in spillover remain unknown.

Evidence of viral recombination in pangolins has been proposed but is unresolved 33 . SARS-like betacoronaviruses have been recently isolated from Sunda pangolins (Manis javanica) traded in wildlife markets 34, 35 , and these viruses have a very high amino acid identity to SARS-CoV-2, but only show a ~90% nucleotide identity with SARS-CoV-2 or Bat-CoV RaTG13 36 . None of these host species are universally accepted as the origin of SARS-CoV-2 nor are any of the viruses a clear SARS-CoV-2 progenitor, and a "better fit" wildlife reservoir could likely still be identified. However, substantial gaps in betacoronavirus sampling across wildlife limit actionable inference about plausible reservoir hosts and bridge hosts for SARS-CoV-2 37 .

Here, we use betacoronaviruses in bats as a case study for the data-driven process of comparing and validating predictive models of likely reservoir hosts, with the ultimate aim to help prioritize surveillance for known and future zoonotic viruses. We focused on betacoronaviruses rather than SARS-like CoVs (subgenus Sarbecovirus) specifically, as the latter are only characterized from a very small number of bat species in publicly available data. This sparsity makes current modeling methods poorly suited to more precisely infer potential reservoir hosts of sarbecoviruses specifically. Instead, we used predictive models to (1) identify bats (and other mammals) that may broadly host any betacoronavirus and (2) identify species with a high viral sharing probability with the two Rhinolophus species carrying the earliest known close viral relatives of SARS-CoV-2. In mid-2020, at the early stages of the COVID-19 pandemic, we developed a standardized dataset of mammal-virus associations by integrating a previously published edgelist 38 with a targeted scrape of all GenBank accessions for Coronaviridae and their associated hosts. Our final dataset spanned 710 host species and 359 virus genera, including 107 mammal hosts of betacoronaviruses as well as hundreds of other (non-CoV) association records. We integrated our hostvirus data with a mammal phylogenetic supertree 39 and over 60 standardized ecological traits of bat species 7, 40, 41 .

We then used these data to generate an ensemble of predictive models and drew on two popular approaches to identify candidate bat reservoir hosts of betacoronaviruses (Table 1) . Network-based methods estimate a full set of "true" unobserved host-virus interactions based on a recorded network of associations (here, pairs of host species and associated viral genera). These methods are increasingly popular as a way to identify latent processes structuring ecological networks [42] [43] [44] , but they are often confounded by sampling bias and often can only make predictions for species within the observed network (i.e., those that have available virus data; in-sample prediction). In contrast, trait-based methods use observed relationships concerning host traits to identify species that fit the morphological, ecological, or phylogenetic profile of known host species of a given pathogen, and rank the suitability of unknown hosts based on these trait characteristics 8, 45 . These methods may be more likely to recapitulate patterns in observed host-pathogen association data (e.g., geographic biases in sampling, phylogenetic similarity in host morphology), but they more easily correct for sampling bias and can predict host species without known viral associations (i.e., out-of-sample prediction).

In total, we implemented eight different predictive models of host-virus associations, including four network-based approaches, three trait-based approaches, and one hybrid approach using trait and phylogenetic information to make network predictions. These efforts generated eight ranked lists of suspected bat hosts of betacoronaviruses. Each ranked list was then scaled proportionally and consolidated in an ensemble of recommendations for betacoronavirus sampling and broader ecoevolutionary research. Next, approximately one year after our initial model ensemble, we reran our entire analytic pipeline with new bat betacoronavirus detections, taking advantage of the recent proliferation of published research on bat CoVs. This provided an unprecedented opportunity to rapidly compare model performance, provide up-to-date predictions of likely but unsampled bat hosts, and critically assess model accuracy in the context of ongoing sampling for bat CoVs.

Our initial ensemble found wide variation in model performance; individual models explained 0-69% of the variance in betacoronavirus positivity (mean of 25%), whereas the ensemble generally had improved predictive capacity (R 2 = 42%; Supplemental Figure 1 ). The predictions of bat betacoronavirus hosts derived from network-and trait-based modeling approaches displayed strong inter-model agreement within each group but largely differed between groups ( Figure 1 ). Of the 1,037 included bat species not known to be infected by betacoronaviruses during our initial analysis in 2020 (against 79 known positives), our models identified between 7 and 722 potential hosts based on a 10% omission threshold (90% sensitivity). Applying this same threshold to our ensemble predictions, our initial models identified 371 bat species as likely undetected hosts. Importantly, only 48 suspect hosts were identified in-sample, whereas we identified 323 suspect hosts out-of-sample, highlighting that most undiscovered hosts-and in turn undiscovered betacoronaviruses-should be in unsampled bat species. This multi-model ensemble predicted undiscovered betacoronavirus bat hosts with striking geographic and taxonomic patterning ( Figure 2 ). In-sample, predicted hosts were globally distributed and recapitulated geographic patterns of known bat betacoronavirus hosts in Europe, the Neotropics, and southeast Asia; however, our models also predicted high richness of likely bat reservoirs in North America. Applying a graph partitioning algorithm (phylogenetic factorization) to the bat phylogeny 46 , we similarly found that both betacoronavirus positivity and in-sample predictions were, on average, lowest for the superfamilies Noctilionoidea and Vespertilionoidea in the Yangochiroptera. This makes intuitive sense, as these taxa do not include the groups known to harbor the vast majority of betacoronaviruses detected in bats (e.g., Rhinolophus, Hipposideridae). In contrast, our out-of-sample predicted hosts were more starkly clustered in much of sub-Saharan Africa and southeast Asia (e.g., Vietnam, Myanmar, and southern China), with no representation in the western hemisphere. Likewise, out-of-sample predictions were lower in Neotropical bat families (e.g., Noctilionidae, Mormoopidae, Phyllostomidae), most emballonurids, and primarily Neotropical molossids, whereas the Rhinolophus genus and most of the Old World subfamily Pteropodinae were predicted to be more likely to host betacoronaviruses (Supplemental Table 1 ).

Because only trait-based models were capable of out-of-sample prediction, the differences in geographic and taxonomic patterns of our predictions likely reflect distinctions between the network-and trait-based modeling approaches. We suggest these should be considered as qualitatively different lines of evidence. Network approaches proportionally upweight species with high observed viral diversity, recapitulating sampling biases largely unrelated to coronaviruses (e.g., frequent screening for rabies lyssaviruses in the common vampire bat Desmodus rotundus, which has been sampled in a comparatively limited capacity for coronaviruses 31, [47] [48] [49] ). Highly ranked species may also have been previously sampled without evidence of betacoronavirus presence; for example, Rhinolophus luctus and Macroglossus sobrinus from China and Thailand, respectively, tested negative for betacoronaviruses, but detection probability was limited by small sample sizes [50] [51] [52] . In contrast, trait-based approaches are constrained by their reliance on phylogeny, ecological traits, and geographic covariates, all of which made models more likely to recapitulate existing spatial (i.e., clustering in southeast Asia) and taxonomic (i.e., the Rhinolophus genus) patterns. However, out-of-sample predictions are, by definition, inclusive of unsampled bat hosts 53 , which potentially offer greater return on viral discovery investment.

Following this initial 2020 model ensemble, we used broad literature searches to systematically track betacoronavirus-positive bat species that were missed in our initial data compilation (e.g., CoV sequences that were not annotated to genus on GenBank 52,54 ). These searches also tracked the exponential increase in data on bat CoVs stemming from the emergence of SARS-CoV-2 that were published after our first model ensemble. A year after our initial data compilation (June 2021), we also reran our initial scrape of GenBank to identify new betacoronavirus-positive bats, limiting our search to matches to Betacoronavirus (taxid: 694002) and Chiroptera (taxid: 9397); however, this did not recover any additional positive host species not already recorded as positives in our updated data. We also mined publically available metagenomic and transcriptomic datasets for evidence of betacoronavirus infection [55] [56] [57] . However, no published libraries contained evidence of betacoronaviruses (see Supplement). Lastly, we analyzed the wildlife testing data from the USAID Emerging Pandemic Threats PREDICT program collected from 2009-2019, which was publicly released in June 2021 and includes a number of betacoronaviruses that were discovered during the program's run (2009 to 2019) but have only published and identified down to the genus level in the full release.

In total, we uncovered 40 novel bat hosts of betacoronaviruses that were either absent from our original dataset or newly discovered after our initial analyses. This data update resulted in a total of 119 positive bat species, and we continue to collate these records (https://www.viralemergence.org/betacov). Of these 40 new hosts, the original ensemble correctly predicted only 24 (60% success rate), but some submodels performed significantly better than others, including both network-and trait-based methods: Network-1 identified 20 of the 21 in-sample novel hosts (95%), while Trait-1 identified 37 of the 40 novel hosts (92.5%). The high performance of both of these top models, and their high performance on the training data (Supplemental Figure 1) , suggest both approaches contributed usefully to the initial ensemble.

The 40 newly discovered hosts also enabled us to develop a new kind of performance metric for machine learning tasks with presence-only validation data (i.e., new "positives" can be collected, while "negatives" are substantially more difficult to prove). If a model makes predictions at random, the predicted prevalence of positives in the training data should be roughly the same as the success rate with novel test data. For example, a "coin toss" model will estimate that approximately 50% of species are betacoronavirus hosts and would likewise successfully identify approximately 50% of newly discovered hosts. A high-performing model, on the other hand, will identify a higher proportion of newly discovered hosts than expected at random. To evaluate how models perform in this regard, we developed a new diagnostic called the training prevalence-test sensitivity curve (TPTSC) that can be applied to modeling problems where training data are composed of a mix of true positives, true negatives, and false negatives, but test data only include novel true positives ( Figure 3 ). The TPTSC plots the assumed prevalence in the training data against the sensitivity in the test data at every possible threshold from 0 to 1; these curves can be treated like receiver-operator or precision-recall curves, where a higher area under the curve (AUC) indicates better-than-random performance. Using the AUC-TPTSC scores, we found that traitbased and hybrid models consistently performed extremely well (Trait-1: 0. These results have two key implications for future efforts to target sampling for putative reservoir hosts. First, ensembling can be useful as a buffer against variation in model quality, particularly in settings when the underlying drivers of model performance have yet to be identified. Second, and perhaps more importantly, models failed to have better-than-random performance without trait data that characterized bat ecology, even when they included phylogenetic data (e.g., Network-4). Part of this difference may also be attributable to the different scope of prediction: the response variable of trait-based models is betacoronavirus presence, while betacoronavirus-relevant predictions were extracted from a broader set of predictions made by the network models. However, this is contraindicated by the results of Hybrid-1, which performed comparably to the other trait-based models. Therefore, we conclude that making meaningful predictions about likely zoonotic reservoirs is best accomplished by incorporating detailed information on host ecology. The vastly greater performance of trait-based models provides another compelling reason, in addition to other One Health and conservation rationales, to better understand the fundamental ecology and evolution of bats.

Inclusion of these 40 novel bat hosts dramatically improved the performance of our predictive models. When revised with new data, our eight individual models explained 8-73% of the variance in betacoronavirus positivity (mean of 32%), with the ensemble R 2 increasing to 60% (Supplemental Figure  2 ). Using our previously applied 90% sensitivity threshold, our revised ensemble identified a more narrow set of 311 bat species as likely undetected hosts of betacoronaviruses. Predictions from the initial and revised ensembles were strongly correlated (ρ = 0.97). However, after dynamically updating our models, our revised ensemble lost 54 suspect reservoirs and gained 26 new suspect reservoirs ( Figure  4A ). Predicted reservoir species that were lost from the initial ensemble were dominated by members of the Vespertillionidae, whereas new suspect hosts were gained in the Vespertillionidae, Hipposideridae, and Molossidae.

Using the 40 newly discovered hosts, we were also able to tailor the updated ensemble responsively to model performance. To do so, we weighted the rank averaging across models based on their AUC-TPTSC score relative to the lowest performing model (Network-2) . In doing so, we effectively dropped Network-2 from the ensemble, a choice supported by the fact the model's predictions are substantially poorer than expected at random. In the updated ensemble, this correction creates a marginal improvement in model performance (unweighted ensemble: AUC-TPTSC = 0.755; weighted ensemble: AUC-TPTSC = 0.788). Therefore, we applied this weighting to the ensemble of updated predictions in the final copy released with this study.

This weighted revised ensemble identified 423 suspect bat hosts, dramatically expanding the scope of plausible candidates for future virus discovery compared to the two prior unweighted ensembles ( Figure  4B ). Predictions from this final ensemble iteration were slightly less correlated with those from our initial ensemble (ρ = 0.92), and these final predictions retained most suspect hosts from the original ensemble. Figure 4D ).

For Rhinolophus bats specifically, our final ensemble identified 11 new suspect hosts, resulting in likely hosts for over three quarters of the species in this genus that are not currently known to be infected by betacoronaviruses (44 of 57 in our dataset), compared to 20 known hosts. Given the known roles of rhinolophid bats as hosts of SARS-like CoVs 18, 21, 50 , it is notable that our results suggest the diversity of these viruses could be undescribed for around three-fourths of these potential reservoir bat species.

As in our initial ensemble, we lastly evaluated geographic and taxonomic patterns in this finalized set of predicted betacoronavirus hosts. Spatially, undiscovered bat hosts were globally distributed (in not only the eastern but also western hemisphere) and especially concentrated within a more narrow band of equatorial sub-Saharan Africa and more starkly in Malaysia and Borneo ( Figure 5A) . Importantly, the geography of these predicted hosts contrasted with the distributions of both known bat hosts and likely hosts from our initial ensemble, each of which instead showed a stronger hotspot in southern China. We also identified distinct clades of high-risk bat hosts from the weighted revised ensemble ( Figure 5B , Supplemental Table 2 ). Both the Rhinolophus genus and subclades of the Pteropodidae again had greater concentrations of predicted betacoronavirus hosts, although phylogenetic factorization now identified the Old World molossids (i.e., genus Mops and Chaerophon) as particularly likely to host these viruses, even though the Mollosidae as a whole have lower mean probabilities of betacoronavirus hosts.

These geographic hotspots and clade-specific patterns of predictions could be particularly applicable for guiding future viral discovery and surveillance. On the one hand, betacoronavirus sampling in southeast Asian bat taxa (especially the genus Rhinolophus) may have a high success of viral detection (and isolation) of sarbecoviruses specifically but may not substantially improve existing bat sampling gaps 5 .

On the other hand, discovery of novel betacoronaviruses in high-priority pteropodid clades, Old World molossids, and Neotropical bats could significantly revise our understanding of the bat-virus association network relative to the coevolutionary distribution of bat betacoronaviruses 38 . For example, predicted bat hosts in the Neotropics may be unlikely reservoirs of sarbecoviruses (given their known distribution in the eastern hemisphere) but would be expected to carry novel viruses from the subgenus Merbecovirus. Such discoveries could be particularly important for global health security, given the surprising recent identification of MERS-like viruses within the merbecoviruses in Mexican and Belizean bats 31, 58 and the likelihood that post-COVID research efforts will focus disproportionately on Asia despite the near-global presence of bat betacoronavirus hosts.

Our work suggests that over 400 bats may host undiscovered betacoronaviruses and that these species can be prioritized for sampling more efficiently via machine learning. Although our models do not target sarbecoviruses specifically, these efforts may help find more SARS-like viruses in wildlife and may even uncover the direct progenitor of SARS-CoV-2, particularly given that 44 species of horseshoe bats are predicted to host undiscovered betacoronaviruses. However, our models provide otherwise limited insights into the origins of SARS-CoV-2, given the likely role of non-bat bridge hosts in spillover to humans 59,60 . We thus attempted a similar model ensemble in June 2020, using five of our eight models to predict the broader mammal-virus network with a focus on potential betacoronavirus bridge hosts. At the time, only 30 non-bat hosts of betacoronaviruses were available in our data. Among the five models, we found poor concordance in predictions (Supplemental Figure 3 ). Predictions were also heavily biased towards well-studied and domesticated mammals (e.g., Ovis aries, Vulpes vulpes, Capra hircus, Procyon lotor, Rattus rattus), indicating that sampling bias dwarfed biological signals. As such, we evaluated these models as having limited value or consistency for an ensemble. This finding may be relevant given other studies have also modeled susceptibility to SARS-CoV-2 across mammals; however, some have used more detailed trait data and thus likely make better predictions at this broader taxonomic scale 61, 62 Instead of further calibrating this mammal-wide ensemble, we focused on the outputs of Trait-3, which predicts how species should share viruses in nature based on their evolutionary history and geography. In June 2020, we predicted the mammals expected to share viruses with Rhinolophus affinis and R. malayanus, which hosted the two viruses (RaTG13 and RmYN02) most relevant to SARS-CoV-2's origins known at that time 23, 63 (a closer related virus, RpYN06, has since been discovered in R. pusillus 64 ). We predicted that these two bat species are disproportionately likely to share viruses with pangolins (Pholidota) and carnivores (Carnivora), including civets (Viverridae), mustelids (Mustelidae), and cats (Felidae; Supplemental Figure 4 ). These predictions have been broadly validated by the role of the masked palm civet (Paguma larvata) in the original SARS-CoV outbreak 65, 66 , the discovery of SARS-CoV-2-like viruses in the Sunda pangolin (Manis javanica) 34 , and extensive "spillback" of SARS-CoV-2 into captive big cats, domestic cats, and both farmed and wild mink 67, 68 . Notably, only the association between palm civets and SARS-CoV was present in the training data used for generating predictions from Trait-3.

Given these successful predictions, we expect there might be potential insights into SARS-CoV-2's emergence when these predictions are paired with data on wildlife supply chains. Out of the top 30 species (Figure 6 ), two are known to have been traded in wildlife markets in Wuhan immediately before the pandemic (the wild boar, Sus scrofa; the palm civet, Paguma larvata), as were two species closely related to those in the top predictions (the Siberian weasel, Mustela siberica; the Northern hog badger, Arctonyx albogularis) 69 . Another top species, the Burmese ferret badger (Melogale personata), was also reportedly of interest in the World Health Organization's origins investigation 70 . Our models indicate that any of these species would be expected to regularly share viruses with relevant Rhinolophus bats in nature. Although bats use habitats differently than most of these likely bridge host species, opportunities for contact exist: a recent study from Gabon found cohabitation among pangolins, bats, and other mammals in burrows 71 . A number of species potentially implicated in the origins of SARS-CoV-2 could therefore have plausibly acquired a progenitor to SARS-CoV-2 in nature, at some point prior to contact with-and spillover into-humans (but see 72 ) . We suggest that this shortlist of species ( Figure 6 ) may therefore be a useful line of evidence for further investigations into the identity of potential bridge hosts, especially in combination with experimental evaluation of susceptibility.

Our study is the first to demonstrate through predictive validation that machine learning models could help optimize wildlife sampling for undiscovered viruses. As such, the growing toolkit of models that predict host-pathogen interactions are likely to aid future efforts both to predict and prevent pandemics and to trace the origins of novel infections after they emerge. However, these tools will work best if they are implemented through a dynamic process of prediction, data collection, validation, and updating, as we have implemented in this study. Although some studies have incidentally tested specific hypotheses (e.g., filovirus models and bat surveys 7,11 , henipavirus models and experimental infections 2,16 , vector-virus models and competence trials [13] [14] [15] ), predictions are almost never subject to systematic verification. Greater dialogue between modelers and empiricists is necessary to confront this research gap. This is particularly important when establishing species' role as viral reservoirs rather than incidental hosts; susceptibility is only one aspect of host competence 1,10 . Future work, including longitudinal tracking of viral shedding over space and time, isolation of live virus from wild animals, and experimental confirmation of viral replication, can support more robust conclusions about whether predicted host species actually play a role in viral maintenance 16 as well as inform related efforts to pinpoint and minimize risk factors for pathogen spillover 73, 74 .

Our study is also the first to benchmark the performance of a set of differently calibrated and designed statistical models all trained for one host prediction task. We found a surprising range of model performance, even among a set of models that all performed well on training data. This finding underscores the need to incorporate long-term validation into similar studies and suggests there may be key lessons about viral ecology to be learned from this kind of exercise. In our study, we found that network models performed mostly at random in validation against new bat hosts, whereas trait-based models were more successful in their predictions. Two explanations may underscore this difference in model performance. First, models that successfully predict the broader mammal-virus network are likely to vary in performance when subset to any given node. This may seem contradictory to the idea that understanding the broader "rules of life" underpinning mammal-virus interactions will improve predictions in specific cases. However, there are ways to combine the strengths of both network-and traitbased approaches. In a closely related study, network predictions of zoonotic risk performed essentially at random, whereas a hybrid approach that embedded network predictions in targeted, trait-based models performed better than any approach in isolation 75 . Future work should aim to develop and benchmark these kinds of hybrid model approaches more extensively and treat exclusively network-driven predictions with caution in the interim.

Second, models that integrate data on host ecology, evolution, and biogeography are likely to make more powerful predictions than those that are mostly agnostic to biology. This has many broader implications. Most importantly, it suggests that filling gaps in the basic biology of bats is a key step towards zoonotic risk assessment and can benefit both pandemic prevention and bat conservation. In particular, highquality host genomes are critical to developing better predictive features, including genome composition bias metrics, improved host phylogenetic trees, and immunological traits 62, [76] [77] [78] [79] . Whole-genome sequencing through initiatives such as the Bat1K Project (https://bat1k.ucd.ie) will expand the sparsely available data on bat genomics and can facilitate other insights into the immune pathways used by bats to harbor virulent viruses [80] [81] [82] . Targeted sequencing could also identify endogenized viral elements in bat genomes, shedding light on bat virus diversity and the evolution of bat immune systems 83, 84 . Large-scale research networks, such as the Global Union of Bat Diversity Networks (https://gbatnet.blogspot.com) and its member networks, will further facilitate efficient sample sharing and ensure proper partnerships and equitable access and benefit sharing of knowledge across countries 85, 86 . Additionally, museum specimens and historical collections offer important opportunities to retrospectively screen samples for betacoronaviruses (thereby testing predictions), sequence tissue for assembly of host genomes, and enhance our understanding of complex host-virus interactions 87 .

Lastly, our iterative modeling of bat betacoronaviruses fits into a broader set of synergies in One Health research on bats, which can create win-win scenarios for conservation and outbreak prevention. For example, North American bats are threatened by an emerging disease, white-nose syndrome 88 90 ), efforts to minimize risks of SARS-CoV-2 "spillback" into novel bat reservoirs [91] [92] [93] [94] [95] , as well as to understand the dynamics of other bat coronaviruses, will both reduce zoonotic risk and help understand and counteract disease-related population declines. Similarly, conservationists have expressed concern that negative framing of bats as the source of SARS-CoV-2 has impacted public and governmental attitudes toward bat conservation 96 ; this can fuel negative responses, including indiscriminate culling (i.e., reduction of populations by slaughter), which has already occured in response to COVID-19 even outside of Asia (where spillover likely occurred) 97 . Evidence demonstrates that culling has numerous negative consequences, not only threatening population viability 98 but also possibly increasing viral transmission within the very species that are targeted 99, 100 . Bat conservation programs and One Health practitioners must continue to work together to find sustainable solutions for humans to live safely alongside wildlife and to communicate with the public about the ecological importance of these highly vulnerable species and the science of pathogen spillover. the Research Foundation -Flanders (FWO17/PDO/067) and the Flemish Government under the "Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen" program. Lastly, we thank three anonymous reviewers for constructive feedback.

The standardized data on betacoronavirus associations, and all associated predictor data, is available from the VERENA consortium's Github (github.com/viralemergence/virionette). All modeling teams contributed an individual repository with their methods, which are available in the organizational directory (github.com/viralemergence). Code for the 2020 analysis, and a working reproduction of each author's contributions, is available from the study repository (github.com/viralemergence/Fresnel); code to produce the updated analyses from 2021 is available in a separate repository (github.com/viralemergence/Fresnel_Jun). A complete list of the 423 predicted betacoronavirus bat hosts from our final ensemble is provided in Supplemental File 1. Figure 1. Broad agreement across an ensemble of predictive modeling approaches. The pairwise Spearman's rank correlations between models' ranked species-level predictions were generally substantial and positive. Models are arranged in decreasing order of their mean correlation with other models. Models that used trait data made more similar predictions to each other than to approaches using network methods with the same data (A); network-based models that used some ecological data made more similar predictions to all other models (Network-4, which uses phylogeny, and Hybrid-1, which uses both phylogeny and trait data). All models that could make out-of-sample predictions used trait data (B) and showed strong agreement.

ise ial ls. rk re th nd

Known hosts of betacoronaviruses (A, B) are found worldwide, but particularly in southern Asia and southern Europe. Taxonomically, betacoronaviruses are less common in two superfamilies of the Yangochiroptera, the Noctilionoidea and Vespertilionoidea (clade 1). The predicted in-sample bat hosts (i.e., those with any viral association records; C, D) tend to recapitulate observed geographic patterns of known hosts but with a higher concentration in the Neotropics. Similarly, taxonomic patterns reflect those of known betacoronavirus hosts. In contrast, the out-of-sample bat host predictions based on phylogeny and ecological traits (E, F) are mostly clustered in Myanmar, Vietnam, and southern China, with none in the Neotropics or North America. Predicted hosts are likewise more common in the Rhinolophidae (clade 2) and subfamilies of Old World bats (clade 5) and are rare in many Neotropical taxa (clades 1 and 7) and emballanurids (clades 3 and 4). In the phylogenies, bar height indicates betacoronavirus positivity (B) or predicted rank (D, F; higher values indicate lower proportional ranks). Colors indicate clades identified through phylogenetic factorization (red indicates clades more likely to contain hosts, blue indicates less likely hosts; see Supplemental Table  1 for more information).

Performance is based on the comparison of total predicted prevalence (i.e., how many species are predicted positives?) with the sensitivity measured from validation data (i.e., how many of the 40 new species are correctly identified?). The null expectation for a model with random performance is these should be equivalent (e.g., a coin toss model will say that 50% of all bats have a betacoronavirus, and will be right 50% of the time), while a model with strong performance will be above that null expectation (thin grey line). (A) The training prevalence-test sensitivity curve (TPTSC) is a novel diagnostic that is conceptually similar to the receiver-operator curve (ROC), in that the model is evaluated at each possible scaled rank threshold between 0 and 1. (B) The same analysis, but only showing the point estimate of positivity created by each model's internallycalibrated threshold. For model-guided sampling, the best model would be one that predicts a low-tomedium positivity rate and has a disproportionately high sensitivity (i.e., in the upper left corner). Both (A) and (B) show that the trait-based models (including the hybrid model) perform well, while the network-only models perform roughly at-random or worse than random (i.e., close to the line); the ensemble model, which includes all eight, performs comparably to the two best trait models and better than six of the eight component models. 

In the weighted revised ensemble predictions, most predicted undiscovered betacoronavirus hosts are found in sub-Saharan Africa and southeast Asia, especially in Malaysia and Borneo (and less in the high-elevation mainland hotspot where most reservoirs of SARS-like viruses are currently known). Predicted hosts from this final ensemble were also most likely in the Rhinolophus genus (clade 7), several subclades of the Pteropodidae (clades 5 and 6), and the Old World molossids (clade 8), even though the Mollosidae as a whole had less likely hosts (clade 3). Bar height in the phylogeny indicates predicted rank, and colors indicate clades identified through phylogenetic factorization (red indicates clades more likely to contain hosts, blue indicates less likely hosts; see Supplemental Table 2 for more information). Uses pseudoabsences for all bats in trait data to predict over all species, including those without known associations

Neutral phylogeographic Yes Trains on a broader network, and predicts sharing probabilities among any mammals in phylogeny and IUCN range map data Hybrid 1 Two-step kernel ridge regression

Uses pseudoabsences for all bats in trait data to predict over all species, including those without known associations

Assembling evidence for identifying reservoirs of infection

Prioritizing surveillance of Nipah virus in India

Temporal and spatial limitations in global surveillance for bat filoviruses and henipaviruses

Taxonomic patterns in the zoonotic potential of mammalian viruses

Identifying Suspect Bat Reservoirs of Emerging Infections

The problem of scale in the prediction and management of pathogen spillover

Undiscovered Bat Hosts of Filoviruses

Confronting data sparsity to identify potential sources of Zika virus spillover infection among primates

The macroecology and evolution of avian competence for Borrelia burgdorferi

Beyond Infection: Integrating Competence into Reservoir Host Prediction

Genetically Diverse Filoviruses in Rousettus and Eonycteris spp. Bats, China

Serologic Evidence of Fruit Bat Exposure to Filoviruses

Data-driven identification of potential Zika virus vectors

Zika virus replication in the mosquito Culex quinquefasciatus in Brazil

Culex quinquefasciatus (Diptera: Culicidae) From Florida Transmitted Zika Virus

Rousettus aegyptiacus Bats Do Not Support Productive Nipah Virus Replication

Detection of possible Nipah virus infection in Rousettus leschenaultii and Pipistrellus pipistrellus bats in Maharashtra, India

Global patterns in coronavirus diversity

Coronaviruses: an RNA proofreading machine regulates replication fidelity and diversity

Full-length genome sequences of two SARS-like coronaviruses in horseshoe bats and genetic variation analysis

Bats are natural reservoirs of SARS-like coronaviruses

Isolation and Characterization of a Novel Bat Coronavirus Closely Related to the Direct Progenitor of Severe Acute Respiratory Syndrome Coronavirus

A pneumonia outbreak associated with a new coronavirus of probable bat origin

Isolation and characterization of viruses related to the SARS coronavirus from animals in southern China

Discovery of a rich gene pool of bat SARS-related coronaviruses provides new insights into the origin of SARS coronavirus

Middle East respiratory syndrome coronavirus in bats, Saudi Arabia

Bat origins of MERS-CoV supported by bat coronavirus HKU4 usage of human receptor CD26

Receptor usage and cell entry of bat coronavirus HKU4 provide insight into bat-to-human transmission of MERS coronavirus

Bat origin of human coronaviruses

Further Evidence for Bats as the Evolutionary Source of Middle East Respiratory Syndrome Coronavirus

Coronaviruses in bats from Mexico

MERS-related betacoronavirus in Vespertilio superans bats

Synonymous mutations and the molecular evolution of SARS-Cov-2 origins

Identifying SARS-CoV-2 related coronaviruses in Malayan pangolins

Isolation of SARS-CoV-2-related coronavirus from Malayan pangolins

Probable Pangolin Origin of SARS-CoV-2 Associated with the COVID-19 Outbreak

The proximal origin of SARS-CoV-2

Host and viral traits predict zoonotic spillover from mammals

Geographical variation in predictors of mammalian extinction risk: big is bad, but only in the tropics

PanTHERIA: a species-level database of life history, ecology, and geography of extant and recently extinct mammals

EltonTraits 1.0: Specieslevel foraging attributes of the world's birds and mammals

Spatio-temporal Bayesian network models with latent variables for revealing trophic dynamics and functional networks in fisheries ecology

Modeling food webs: exploring unexplained structure using latent traits

Predicting cryptic links in host-parasite networks

Rodent reservoirs of future zoonotic diseases

Phylofactorization: a graph partitioning algorithm to identify phylogenetic scales of ecological data

A coronavirus detected in the vampire bat Desmodus rotundus

Highly diversified coronaviruses in neotropical bats

Neotropical bats from Costa Rica harbour diverse coronaviruses

Discovery and genetic analysis of novel coronaviruses in least horseshoe bats in southwestern China

Extensive diversity of coronaviruses in bats from China

Diversity of coronavirus in bats from Eastern Thailand

The influence of bat ecology on viral diversity and reservoir status

Longitudinal study of age-specific pattern of coronavirus infection in Lyle's flying fox (Pteropus lylei) in Thailand

Diversification of mammalian deltaviruses by host shifting

Using noninvasive metagenomics to characterize viral communities from wildlife

Complete Genome Sequence of an Alphacoronavirus from Common Vampire Bats in Peru

Surveying the vampire bat (Desmodus rotundus) serum proteome: a resource for identifying immunological proteins and detecting pathogens

The Potential Intermediate Hosts for SARS-CoV-2. Frontiers in Microbiology

Origin and evolution of pathogenic coronaviruses

Predicting the zoonotic capacity of mammal species for SARS-CoV-2. bioRxiv 2021

Predicting mammalian hosts in which novel coronaviruses can be generated

A novel bat coronavirus reveals natural insertions at the S1/S2 cleavage site of the Spike protein and a possible recombinant origin of HCoV-19

Identification of novel bat coronaviruses sheds light on the evolutionary origins of SARS-CoV-2 and related viruses

SARS-CoV Infection in a Restaurant from Palm Civet

Cross-host evolution of severe acute respiratory syndrome coronavirus in palm civet and human

New Approaches to Anticipate the Risk of Reverse Zoonosis

Spillback in the Anthropocene: the risk of human-to-wildlife pathogen transmission for conservation and public health

Animal sales from Wuhan wet markets immediately prior to the COVID-19 pandemic

In Hunt for Covid-19 Origin, WHO Team Focuses on Two Animal Types in China

Pangolins and bats living together in underground burrows in Lopé National Park

No Evidence of Coronaviruses or Other Potentially Zoonotic Viruses in Sunda pangolins (Manis javanica) Entering the Wildlife Trade via Malaysia

Sampling to elucidate the dynamics of infections in reservoir hosts

Ecological interventions to prevent and manage zoonotic pathogen spillover

Imputing the mammalian virome with linear filtering and singular value decomposition

Predicting reservoir hosts and arthropod vectors from evolutionary signatures in RNA virus genomes

Identifying and prioritizing potential human-infecting viruses from their genome sequences. Cold Spring Harbor Laboratory

Phylogenetic inference using whole genomes

Wild immunology

Bat Biology, Genomes, and the Bat1K Project: To Generate Chromosome-Level Genomes for All Living Bat Species

Six reference-quality genomes reveal evolution of bat adaptations

Decoding bat immunity: the need for a coordinated research approach

The Potential Role of Endogenous Viral Elements in the Evolution of Bats as Reservoirs for Zoonotic Viruses

Six new reference-quality bat genomes illuminate the molecular basis and evolution of bat adaptations

Networking networks for global bat conservation

Bat Research Networks and Viral Surveillance: Gaps and Opportunities in Western Asia

Integrating biodiversity infrastructure into pathogen discovery and mitigation of epidemic infectious diseases

An emerging disease causes regional population collapse of a common North American bat species

White-nose syndrome is associated with increased replication of a naturally persisting coronaviruses in bats

Experimental challenge of a North American bat species, big brown bat (Eptesicus fuscus), with SARS-CoV-2

Possibility for reverse zoonotic transmission of SARS-CoV-2 to free-ranging wildlife: A case study of bats

Risk of SARS-CoV-2 transmission from humans to bats -An Australian assessment

The risk from SARS CoV 2 to bat species in england and mitigation options for conservation field workers. Transboundary and Emerging Diseases

Risks posed by SARS CoV 2 to North American bats during winter fieldwork. Conservation Science and Practice

IUCN SSC Bat Specialist Group (BSG) Recommended Strategy for Researchers to Reduce the Risk of Transmission of SARS-CoV-2 from Humans to Bats (version 2.0)

COVID-19 drives new threat to bats in China

COVID-19 and threats to bats

Do current vampire bat (Desmodus rotundus) population control practices pose a threat to Dekeyser's nectar bat's (Lonchophylla dekeyseri) long-term persistence in the Cerrado?

Ecological and anthropogenic drivers of rabies exposure in vampire bats: implications for transmission and control

Resolving the roles of immunity, pathogenesis, and immigration for rabies persistence in vampire bats

Predicting the global mammalian viral sharing network using phylogeography

We thank Heather Wells for generously sharing thoughtful comments and code. The Viral Emergence Research Initiative (VERENA) consortium is supported by L'Institut de Valorisation de Données (IVADO) through Université de Montreal and by NSF BII 2021909; for more information, see viralemergence.org. LMB was supported by the Wellcome Trust (217221/Z/19/Z). MS was supported by