key: cord-0705319-4b2jqac4
authors: Ran, Na
title: Association Between Immediacy of Citations and Altmetrics in COVID-19 Research by Artificial Neural Networks
date: 2021-08-31
journal: Disaster Med Public Health Prep
DOI: 10.1017/dmp.2021.277
sha: 5e2297cc5801d45eb66ac3150ed11d994dfe33ae
doc_id: 705319
cord_uid: 4b2jqac4

OBJECTIVES: Both citations and Altmetrics are indexes of influence of a publication, potentially useful, but to what extent that the professional-academic citation and media-dominated Altmetrics are consistent with each other is a topic worthy of being investigated. The objective is to show their correlation. METHODS: DOI and citation information of coronavirus disease 2019 (COVID-19) researches were obtained from the Web of Science, its Altmetric indicators were collected from the Altmetrics. Correlation between the immediacy of citation and Altmetrics of COVID-19 research was studied by artificial neural networks. RESULTS: Pearson coefficients are 0.962, 0.254, 0.222, 0.239, 0.363, 0.218, 0.136, 0.134, and 0.505 (P < 0.01) for dimensions citation, attention score, journal impact factor, news, blogs, Twitter, Facebook, video, and Mendeley correlated with the SCI citation, respectively. The citations from the Web of Science and that from the Altmetrics have deviance large enough in the current. Altmetric score is not precise to describe the immediacy of citations of academic publication in COVID-19 research. CONCLUSIONS: The effects of news, blogs, Twitter, Facebook, video, and Mendeley on SCI citations are similar to that of the journal impact factor. This paper performs a pioneer study for investigating the role of academic topics across Altmetric sources on the dissemination of scholarly publications.

A worse disease result can occur from lack of knowledge. 1 Public health can be promoted by means of the sharing of knowledge and the development of knowledge. 2 Specialized knowledge is necessary to effective preparedness, response, and recovery from disasters. 3 Therefore, people need to master the knowledge related to a disaster, which is usually understood gradually. It is very important to quickly and effectively disseminate the newly developed disaster-related knowledge to the scientific and social communities. This paper studies how newly developed knowledge of the disaster disease (coronavirus disease 2019 ) disseminates in the professional community and social problems in a timely manner. The more quickly the disaster knowledge is disseminated to the whole society, the better off society could be.

When you search for COVID-19 on Google, you can find more than 5,640,000,000 items. Billions of Internet searches have been done worldwide on seeking information on COVID-19. 4 At the same time, scholarly publications become a focus for academic social webs (eg, ResearchGate, Mendeley, Academia.edu, etc.). These webs secure public access to freely exploring the publications. Thus, academic social web-linked bibliometric indicators might be used to evaluate a research impact too. Altmetrics (social web metrics) as a result of the introduction of social media into scholarly practices was first proposed in 2010. 5 It tries to evaluate scientific researches circulating on social webs such as Facebook, Twitter, blogs, news media, etc. And it gives insights into the analysis of research impact complementary to the limitations of traditional and science web-based impact metrics. 6 Altmetric scores were found to be correlated to citation scores. [7] [8] [9] [10] [11] Although further study demonstrated that there were positive correlations between Altmetrics and citations the correlation for publications in the field of social sciences, humanities, and the medical and life sciences are pretty weak. 12 It has ever been reported that the correlations between ResearchGate indicators and the university rankings are unexpectedly moderate. 13 Correlations among a set of critical indicators have also been studied. 9 These results suggest that it is not clear how scientifically academic social web-based indicators can be used to evaluate the impact of research. A principal component analysis was used to study this issue, and it concluded that usage metrics form an opposed component regarding bibliometric counts. 14 Other publications support that "Altmetric indicators seem to measure impact mostly orthogonal to citation", 15 and the numbers of Mendeley readers can "reflect different aspects of the research impact". 16 The initial studies on academic social webs focused on the motivations and preferences of researchers using academic social sites. 17, 18 The study of how to improve research activity with academic social webs have also been performed. 19 For studying their significance in evaluating research, for example, how tweets correlate with a publication citation, a study was conducted on how these new metric measures are correlated with each other. 20 Most academic social web-based bibliometric indices are statistically significantly correlated, but most Altmetric data are to be still low. Altmetric events prefer to focus on publications in Biomedical and Health Sciences, Social Sciences and Humanities, and Life and Earth Sciences. 21 The digital dissemination of a scientific article is hypothesized to correlate with citations and the journal impact factor in pediatric surgery. 22 Altmetric scores of articles reflect the emerging role of social media in research dissemination other than the correlation between citation number and Altmetric scores of a publication. 23 Articles published in journals with higher impact factors were found to draw greater attention to social media. 24 Altmetric scores of articles in implantology are so far insufficient to replace traditional Bibliometrics. 25 Twitter, newsfeeds, and Facebook are the main contributors to the Altmetric. Attention Score was not significantly correlated with the impact factor. 26

It is very important to use data to test and validate generating theories and hypotheses. The Pearson correlation coefficient of data sets X and Y is a key validating factor of this process 27,28

When the Pearson r correlation coefficient is larger than −1 and less than 0, it means that there is a negative (or inverse) linear correlation between the 2 data sets. The artificial neural network is the key to the success of computers solving intelligent tasks. Its attraction comes from its remarkable information processing capability, mainly involving nonlinear, high parallelism, fault tolerance, and anti-noise, as well as learning and generalization ability. 29 Artificial neural networks are increasingly being introduced into various disciplines, such as bibliometrics, linguistics, medicine, etc. 30 Using artificial neural network algorithms, knowledge can be extracted from large heterogeneous data sets. 31 Artificial neural network models have been applied to citations and impact of scholarship, such as it was used to predict long-term citations of a paper 32 ; it was applied to predict 5-y citations of papers, 33 and it was introduced to measure these factors influencing academic citations. 34 In this study, an artificial neural network model was introduced to discuss the correlation between immediate citations and Altmetrics of COVID-19-related papers.

One can note that both citations and Altmetrics are different qualified factors of publication. To what extent these 2 indexes can predict each other is important to validate their roles of evaluating publications precisely from different angles. Two data sources for the analysis of COVID-19 papers were used: (1) the impact factor of the journals that publish COVID-19 papers, the author number, the citations, and the type of COVID-19 papers from Journal Citation Report from the website of the Web of Science (www.webofknowledge.com); (2) the dimensions citation, news, blogs, Twitter, Weibo, Facebook, Google plus, video, Mendeley, Citeulike, and the attention score from Altmetrics from the website of the Altmetrics (www. Altmetric.com). Journal Impact Factor measures the influence or impact of a scientific journal, based on citations received by papers published by this journal. 35 Briefly, the Altmetric score represents a weighted count of the amount of attention for research output from a variety of sources. 36 Note that a multilayer perceptron can map any function to any precision 37 ; a neural network model based on a multilayer perceptron program was used in this study. This multilayer perceptron is composed of several simple processing nodes in several different layers. The linear combination of weighted inputs has been calculated at each node from the link feeding it. 38 Each hidden unit is a function of the input weighted sum. Log sigmoid or hyperbolic tangent function is used to convert network input. In the following example, the hyperbolic tangent function is used to convert a real parameter to a range (-1, 1). The hidden layer contains invisible network nodes. In the output layer, the output unit is the weighted sum of the hidden unit, because the activation function is an identity function. In a word, 1 or more dependent variables (targets) can be predicted based on the values of the predicted variables.

First, the topic word "COVID-19" was used to retrieve all databases in the Web of Science database. The DOI (digital object identifier), citation, and journal impact factor, etc., for each retrieved paper in the topic of "COVID-19" was classified, listed, and used in the next step. Second, with the DOI of each retrieved paper in the topic of "COVID-19" free Hou Yi crawler software was used to crawl and download its information from the Altmetric website. All information generated above was classified, listed, and used in a neural network model based on a multilayer perceptron program. Last, the above classified and listed data have been analyzed by using the neural network model with a multilayer perceptron program in the computer program SPSS. Based on these above results, the correlation between the immediacy of citations and COVID19 research has been discussed

Using the topic word of "COVID-19" searched in all databases on the Web of Science, 575 articles, 19 letters, and 106 reviews are found before September 18, 2020. After duplicated items and items without DOI have been removed, 429 articles, 19 letters, and 98 reviews are used. Table 1 shows the statistical characteristics of the above information. The papers with COVID-19 as a topic were not mentioned in Weibo, Google plus, Mendeley,

or Citeulike before September 18, 2020. Thus, they have been excluded from the final analysis. In the calculation, the number of training samples is 355, the number of valid is 355, and the number of excluded samples is 191. The detail of network information is given in the following. For input layer: 1 factor as article "type," 8 covariates as "journal impact factor," "Author numbers," "news," "blogs," "Twitters," "Facebook," "Video," and "Mendeley," and standardized rescaling method for covariates have been used. For hidden layers, the number of the hidden layer as 1, hyperbolic tangent activation function had been used, and the number of units in the hidden layer has been chosen according to the sum of squares error. Figure 1 shows the percentage of SCI (Science Citation Index) citations and Attention Scores for the papers with COVID-19 as a topic on the website of the Altmetric; 66.8% of the papers have no SCI citations. This figure demonstrates that both the percentage of SCI citations and Attention Scores for the papers have a similar tendency. Figure 1 also illustrates the result of the percentage of the journal impact factor in the Web of Science and dimension citations for the papers with COVID-19 as a topic on the website of the Altmetric; 58.4% of documents have no dimension citations in Altmetric. This figure denotes that there is significant dependent relation between the number of papers with COVID-19 as a topic published in journals indexed by SCI and journal impact factor. It also shows that the number of papers with COVID-19 as a topic published in journals indexed by SCI has a decreasing tendency with dimension citations. Figure 2 denotes the percentage of author numbers and total number in social webs for the papers with COVID-19 as a topic on the website of the Altmetric. A total of 40.8% of documents have not been reported in social nets in Altmetric. This figure shows that the papers with COVID-19 as a topic with 2 authors are the maximum. The percentage of the papers with COVID-19 as a topic, first, increases with author numbers, then decreases with author numbers. It also demonstrates that the number of papers with COVID-19 as a topic published in journals indexed by SCI has a decreasing tendency with the total number on social nets. Figure 2 depicts the percent of the papers with COVID-19 as a topic on the website of the Altmetrics mentioned in news and Twitter; 91.4% and 40.8% of the papers with COVID-19 as a topic did not appear in news and Twitter, respectively. This figure shows that the percentage of the papers with COVID-19 as a topic mentioned in both news and Twitter tends to decrease with oscillations. Figure 3a depicts the percentage of the papers with COVID-19 as a topic on the website of the Altmetrics mentioned in blogs and Facebook. A total of 95.4% and 95.8% of the papers of COVID-19 as a topic have not been mentioned in blogs and Facebook, respectively. This figure shows that the percentage of the papers with COVID-19 as a topic mentioned in both blogs and Facebook decrease with fluctuations. Figure 3b shows the sum of squares errors changes with the number of neurons in the hidden layer. Thus, 26 neurons in the hidden layer have been selected in the following analysis. A relative error in the calculation is 0.445 and the sum of squares error is 2.310 in the following calculation. The calculation of the neural network model with a multilayer perceptron program stops when the relative error does not decrease after 1 step. Table 2 shows that the values of importance of the journal impact factor, news, blogs, video, and Mendeley are close by orders of magnitude. And the values of importance for the number of authors of any paper, Twitter, Facebook are close by orders of magnitude. As noted by the orders of magnitude difference in the listed values, it is clear that choosing the critical variables for some social websites for evaluating their scientific nature is quite important due to their preference for social impact other than scientific purposes.

The data of Twitter, newsfeeds, and Facebook that are shown in Tables 1 and 2 support the finding that Twitter, newsfeeds, and Facebook gave the most contribution to the Altmetrics. 26 The Pearson correlation coefficient between the journal impact factor of papers with COVID-19 as a topic and its Altmetric scores is 0.185 with P < 0.01. It demonstrates that there is a positive correlation between the journal impact factor of articles and attention on social media. 24 The data of Mendeley that are shown in Tables 1 and 2 agree that the research impact can be reflected by the numbers of Mendeley readers. 16 Figure 4 depicts the difference between the predicted and recorded citations on the web of science. This figure demonstrates that the predicted number is much different from the recorded one. It may suggest that some critical parameters in current neural network modeling are missing. In other words, the immediacy of citations of a paper can be affected by other key factors except for Altmetric scores. Such an observation agrees that the Altmetric scores are not scientific enough to replace the traditional Bibliometrics. 25 The Pearson correlation coefficient between the immediacy of citations of the papers with COVID-19 as a topic and their Altmetric scores shown in Table 3 is 0.254 with P < 0.01. It means there is a small significant positive linear correlation between the immediacy of citations of the papers with COVID-19 as a topic and their Altmetric scores. Such results agree well with the conclusion that a correlation between Altmetric scores of articles and citation counts for articles was drawn in reference. 23 The Pearson correlation coefficient between the immediacy of citations of the papers with COVID-19 as a topic and their dimensions citation count in Altmetrics is 0.962, with P < 0.01. Such a conclusion is consistent with the former finding that there is a strong correlation between the dimensions citation count in Altmetrics and the SCI citation. 39 Table 3 also demonstrates that the number of authors and the type of article are not significantly linearly correlated to the immediacy of citation. In summary, most Pearson correlation coefficients in Table 3 are small except for 1 Pearson correlation coefficient that is between Dimensions citations count in Altmetrics and the citations. A larger Pearson correlation coefficient denotes a strong correlation. This conclusion supports the finding that there were weak positive correlations between Altmetrics and citations. 12, 39 Conclusions A neural network model with a multilayer perceptron program has been used to evaluate the effects of social media on the immediacy of citations in this article. Through the case study of the papers with a topic word of COVID-19 from the Web of Science and the Altmetric, it is found that Twitter, newsfeeds, and Facebook are the main contributors to the Altmetric; the journal impacts factor of articles is positively correlated to social media's attention; the numbers of Mendeley reader gives an important contribution to the SCI citations; while the number of authors and the type of article are not correlated to the immediacy of citation; the strong correlation between Dimensions citation count in Altmetrics a nd the SCI citations have been validated. Last, the predicted immediacy of citation of a paper with COVID-19 as a topic has large deviance from the SCI citations. Our results show that, although Altmetric scores cannot replace the traditional Bibliometrics, their potential cannot be underestimated, because it may underlie the social roles of a scientific publication. In a word, both citations and Altmetrics should be considered, because either can reflect the underlying different features of scholarly publishing. Disaster Medicine and Public Health Preparedness 5

Time intervals from first symptom to treatment of cancer: a cohort study of 2,212 newly diagnosed cancer patients

Knowledge and health promotion practice of Family Health Strategy nurses

Core competencies for disaster medicine and public health

COVID-19-related web search behaviors and infodemic attitudes in Italy: infodemiological study

Altmetrics: rethinking the way we measure

On the relation between Altmetric s and citations in medicine (RIP). presented at: Proceedings at

Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature

Do Altmetric s work? Twitter and ten other social web services

How well developed are Altmetric s? cross-disciplinary analysis of the presence of 'alternative metrics' in scientific publications (RIP)

How well developed are Altmetric s? A cross-disciplinary analysis of the presence of 'alternative metrics' in scientific publications

Altmetrics" correlate with citations? Extensive comparison of Altmetric indicators with citations from a multidisciplinary perspective

ResearchGate: disseminating, communicating, and measuring scholarship?

A principal component analysis of 39 scientific impact measures

Altmetrics in the wild: using social media to explore scholarly impact

Mendeley readership Altmetric s for the social sciences and humanities: research evaluation and knowledge flows

Users' classification and usage-pattern identification in academic social networks

User participation in an academic social networking service: a survey of open group users on Mendeley

Mendeley group as a new source of interdisciplinarity study: how do disciplines interact on mendeley?

Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact

An extensive analysis of the presence of Altmetric data for Web of Science publications across subject fields and research topics

Correlation between Altmetric score and citations in pediatric surgery core journals

Top-cited articles in medical professionalism: a bibliometric analysis versus Altmetric scores

Social attention and scientific articles on stroke: Altmetric analysis of top-50 articles

Analyzing the relationship between Altmetric score and literature citations in the Implantology literature

The top 100 spine surgery articles on social media: an Altmetric study

Mathematical Modeling for Business Analytics

Statistical Methods in Social Science Research

Artificial neural networks: fundamentals, computing, design, and application

The opportunities and limitations of using artificial neural networks in social science research

Machine learning, artificial neural networks and social research

Predicting citation counts based on deep neural network learning techniques

Predicting the citation counts of individual papers via a BP neural network

Measuring the impact of novelty, bibliometric, and academic-network factors on citation count using a neural network

Elsevier's journal metrics for the identification of a mainstream journals core: a case study on Mexico

The Altmetric attention score: what does it mean and why should i care?

Multi-criteria validation of artificial neural network rainfallrunoff modeling

Simulation of daily temperatures for climate change scenarios over Portugal: a neural network model approach

Measuring the social impact of research in paediatric dentistry: an Altmetric study