key: cord-0313584-g5fsod3f
authors: Ramsey, Jolene; McIntosh, Brenley; Renfro, Daniel; Aleksander, Suzanne A.; LaBonte, Sandra; Ross, Curtis; Zweifel, Adrienne E.; Liles, Nathan; Farrar, Shabnam; Gill, Jason J.; Erill, Ivan; Ades, Sarah; Berardini, Tanya Z.; Bennett, Jennifer A.; Brady, Siobhan; Britton, Robert; Carbon, Seth; Caruso, Steven M.; Clements, Dave; Dalia, Ritu; Defelice, Meredith; Doyle, Erin L.; Friedberg, Iddo; Gurney, Susan M.R.; Hughes, Lee; Johnson, Allison; Kowalski, Jason M.; Li, Donghui; Lovering, Ruth C.; Mans, Tamara L.; McCarthy, Fiona; Moore, Sean D.; Murphy, Rebecca; Paustian, Timothy D.; Perdue, Sarah; Peterson, Celeste N.; Prüß, Birgit M.; Saha, Margaret S.; Sheehy, Robert R.; Tansey, John T.; Temple, Louise; Thorman, Alexander William; Trevino, Saul; Vollmer, Amy Cheng; Walbot, Virginia; Willey, Joanne; Siegele, Deborah A.; Hu, James C.
title: Crowdsourcing biocuration: the Community Assessment of Community Annotation with Ontologies (CACAO)
date: 2021-05-01
journal: bioRxiv
DOI: 10.1101/2021.04.30.440339
sha: 1d4d44f6605ab789845541a2f567ebb14edc45da
doc_id: 313584
cord_uid: g5fsod3f

Experimental data about known gene functions curated from the primary literature have enormous value for research scientists in understanding biology. Using the Gene Ontology (GO), manual curation by experts has provided an important resource for studying gene function, especially within model organisms. Unprecedented expansion of the scientific literature and validation of the predicted proteins have increased both data value and the challenges of keeping pace. Capturing literature-based functional annotations is limited by the ability of biocurators to handle the massive and rapidly growing scientific literature. Within the community-oriented wiki framework for GO annotation called the Gene Ontology Normal Usage Tracking System (GONUTS), we describe an approach to expand biocuration through crowdsourcing with undergraduates. This multiplies the number of high-quality annotations in international databases, enriches our coverage of the literature on normal gene function, and pushes the field in new directions. From an intercollegiate competition judged by experienced biocurators, Community Assessment of Community Annotation with Ontologies (CACAO), we have contributed nearly 5000 literature-based annotations. Many of those annotations are to organisms not currently well-represented within GO. Over a ten-year history, our community contributors have spurred changes to the ontology not traditionally covered by professional biocurators. The CACAO principle of relying on community members to participate in and shape the future of biocuration in GO is a powerful and scalable model used to promote the scientific enterprise. It also provides undergraduate students with a unique and enriching introduction to critical reading of primary literature and acquisition of marketable skills. Significance Statement The primary scientific literature catalogs the results from publicly funded scientific research about gene function in human-readable format. Information captured from those studies in a widely adopted, machine-readable standard format comes in the form of Gene Ontology annotations about gene functions from all domains of life. Manual annotations based on inferences directly from the scientific literature, including the evidence used to make such inferences, represents the best return on investment by improving data accessibility across the biological sciences. To supplement professional curation, our CACAO project enabled annotation of the scientific literature by community annotators, in this case undergraduates, which resulted in contribution of thousands of validated entries to public resources. These annotations are now being used by scientists worldwide.

an evolving biocuration resource that provides the framework for capturing attributes of 153 gene products within three aspects or main branches: biological process, molecular 154 function, and cellular component(5, 6). Importantly, connections can be made between 155 model organism genes and human genes with comprehensive GO coverage(7).

Additionally, using GO data generates testable hypotheses in areas with little direct 157 experimentation(8-10). Application to high-throughput and systems biology, for 158 instance, has led to insights and better methods for identification and analysis of the 159 genes involved in cardiac and Alzheimer's disease(11, 12).

Without question GO is a critical scientific resource, but manual annotation is an 161 extremely labor-intensive process (13, 14) . The pace at which the information is generated information about normal gene functions from the paper study subjects, and capture the evidence 576 and conclusions using the Gene Ontology. Those annotations are reviewed by trained judges and 577 marked as unacceptable (red X), requiring changes (yellow !, or purple ? flagged for further 578 review), or acceptable (green check, or blue check after correction) within the GONUTS 579 framework. Competitors challenge entries and engage in peer review until an annotation is 580 corrected or marked unacceptable. Fully vetted annotations are deposited into the public GO 581 database maintained by professional biocurators and used by scientists worldwide. As required, 582

CACAO-submitted annotations will be updated to reflect rearrangements and changes in GO. 583

Importantly, the GONUTS wiki provides a web-References

Building an efficient curation workflow 454 for the Arabidopsis literature corpus

Manual Gene Ontology annotation 456 workflow at the Mouse Genome Informatics Database

Applying the Gene Ontology in 536 microbial annotation

E. approach to primary 538 literature shifts undergraduates' self-assessed ability to read and analyze journal articles, 539 attitudes about science, and epistemological beliefs

Figure facts: encouraging undergraduates to take a 542 data-centered approach to reading primary literature

The Joint Genome Institute's microbial genome annotation program 545 for undergraduates

A broadly implementable research course in phage discovery and 547 genomics for first-year undergraduate students

Figure 2: The GO annotations contributed by CACAO users are diverse and specific. A) 586

The organisms most highly 587 represented in each domain are displayed on the outer ring of the chart divided by the following 588 rank: Phylum for eukaryotes and archaea, Order for bacteria, and Family for viruses. The number 589 of GO annotations in each category is indicated in brackets. B) The distribution of GO terms used 590 for CACAO annotations are graphed by aspect within the ontology. The top three terms within 591 each aspect are labeled on the outer ring. For clarity

The number of GO annotations for 594 each term is indicated in brackets. C) The descendant counts, corresponding to depth within the 595 ontology, for CACAO annotations (n = 4913) and all other manual GO annotations through 2019 596 (n = 255,958) are graphed