There's An App for That

scite: Providing Additional Context for Citation Statements

Elizabeth Sterner
Health and Human Services Librarian
University Library
Governors State University
University Park, IL
esterner@govst.edu

What Is scite.ai?

scite is a startup based in Brooklyn, New York, and funded in part by the National Science Foundation and the National Institute of Drug Abuse of the National Institutes of Health. With scite, researchers can use “smart citations” to see the context, the location, and the classifications of citation statements. Currently, over 800 million citation statements from scientific articles have been included (Citation Coverage). This tool is a work in progress, and its greatest coverage of the literature is in the field of biomedical research (scite 2021). Browser extensions and limited scite reports are free, however, increasing functionality is limited to subscribers. What role do citations play in scholarly communication? “scite combines deep learning with a network of experts to evaluate the veracity of scientific work” (M scite 2020). Citation statistics, e.g. h-index and impact factor, are often viewed as directional proportional to a manuscript’s value and integrity. Background literature and the literature review provide valuable information for both the authors and readers of scientific papers. With proper background research, authors can save time and money by learning from past successes and failures as evidenced in previous research. Citations can help readers trust that authors have reached scientifically sound conclusions based on previously established research and evidence. Even if a scholarly article is retracted, it can still be cited. The reasons for citation statements in a retracted article may vary. scite’s goal is to further clarify the citation statements and the role they play in all scholarly articles.

What Types of Citations Exist?

The developers of scite used a deep learning model to create “smart citations” which differentiate citation statements between three functional categories. Users can look up citation statements to determine if they provide supporting evidence, disputing evidence, or mention the article without providing evidence. Terminology, such as “in support of previous work” and “does not replicate results from previous studies,” is used to classify citations statements as supporting or disputing evidence, respectively. scite classifies citation statements that refer to only a part of paper. An entire paper may support or dispute another paper, but the individual citation statements are used to determine if they support or dispute the other paper. Even if the overall paper does support or dispute a paper, the citation statement will be categorized as mentioning if that citation statement does neither. Retracted articles might not display any disputed citation statements because they may not have any disputing citation statements or those disputing papers are not yet included. Incorrectly classified citation statements can be flagged to be reviewed by two independent experts for correction (scite 2021).

How Do I Use scite.ai?

Users may either use the website or install the browser plugin for Chrome and Firefox available on their website. Basic registration is free and includes one scite report and one visualization per month (Simple and flexible pricing 2021). Paid plans scale up from individual to institutional and include unlimited scite reports and visualizations. To begin, type the DOI, title, or author into the search bar (Figure 1).

Figure 1: View of search bar

Results are loaded and can be ordered by relevance, date, most and least citation volume, supported, disputed, and mentioned citations. Click on a title to select the desired article. Results appear on the loaded page (Figure 2). Filters on the left-hand side allow users to select classification types (supporting, mentioning, disputing), publication dates of citation statements, paper sections (introduction, methods, results, discussion, other sections), and paper types (research articles, reviews). The number of citation statements appears above the section with citation statements. Clicking on the title of an article will take users to the homepage of the article citing the original article. Users can easily identify if a citation statement is marked as supporting, refuting, or mentioning. Click on the flag to highlight potential errors for re-review.

Figure 2: Partial screenshot of results page

Two highly cited retracted articles were analyzed to demonstrate the patterns of types of citation statements that occurred before and after retraction of each article. Article A was published in 2012 and retracted in 2018 (Figure 3). Article B was published in 2005 and retracted in 2007 (Figure 4). Even after papers were retracted, they continue to be cited with the three types of citation statements. This is especially evident in Article B, where the article continues to be cited by other articles thirteen years later.

Figure 3: Numbers of supporting, mentioning, and disputing citation statements of Article A

Figure 4: Numbers of supporting, mentioning, and disputing citation statements of Article B

Conclusion

The ability to differentiate citation statement types is valuable for assessing the veracity of research articles. This tool is not meant to replace database searching, but it does provide additional information related to citation statements. The features provided by scite will be especially important as preprints, which have not yet gone through the time-consuming process of peer-review, become ever more important in times of fast changing scientific discovery, e.g. the COVID-19 pandemic (M scite 2020). scite is a valuable tool for researchers looking to better understand the context, location, and classification of citation statements.

References

Citation Coverage [Internet]. Brooklyn (NY): scite, Inc.; [Cited 2021 May 28] Available from https://help.scite.ai/en-us/article/how-many-citations-does-scite-have-qla34b/.

M scite [Internet]. San Francisco (CA): Medium; 2020, April 7 [Cited 2020 June 21]. Available from https://medium.com/scite/12-preprints-on-covid-19-that-have-been-supported-or-contradicted-by-subsequent-research-2ef1738e01a2.

scite [Internet]. Brooklyn (NY): scite Inc; 2021 [Cited 2020 June 20]. Available from https://scite.ai.

Simple and flexible pricing [Internet]. Brooklyn (NY): scite Inc; 2021 [Cited 2021 May 28]. Available from https://scite.ai/pricing.

This work is licensed under a Creative Commons Attribution 4.0 International License.