key: cord-0964133-vtkw3iex
authors: nan
title: COVID-19 Sensing: Negative Sentiment Analysis on Social Media in China via BERT Model
date: 2020-07-28
journal: IEEE Access
DOI: 10.1109/access.2020.3012595
sha: 34e22eeed4eb78c371f959570c1ba1976777f99e
doc_id: 964133
cord_uid: vtkw3iex

Coronavirus disease 2019 (COVID-19) poses massive challenges for the world. Public sentiment analysis during the outbreak provides insightful information in making appropriate public health responses. On Sina Weibo, a popular Chinese social media, posts with negative sentiment are valuable in analyzing public concerns. 999,978 randomly selected COVID-19 related Weibo posts from 1 January 2020 to 18 February 2020 are analyzed. Specifically, the unsupervised BERT (Bidirectional Encoder Representations from Transformers) model is adopted to classify sentiment categories (positive, neutral, and negative) and TF-IDF (term frequency-inverse document frequency) model is used to summarize the topics of posts. Trend analysis and thematic analysis are conducted to identify characteristics of negative sentiment. In general, the fine-tuned BERT conducts sentiment classification with considerable accuracy. Besides, topics extracted by TF-IDF precisely convey characteristics of posts regarding COVID-19. As a result, we observed that people concern four aspects regarding COVID-19, the virus Origin (Gamey Food, 3.08%; Bat, 2.70%; Conspiracy Theory, 1.43%), Symptom (Fever, 2.13%; Cough, 1.19%), Production Activity (Go to Work, 1.94%; Resume Work, 1.12%; School New Semester Beginning, 1.06%) and Public Health Control (Temperature Taking, 1.39%; Coronavirus Cover-up, 1.26%; City Shutdown, 1.09%). Results from Weibo posts provide constructive instructions on public health responses, that transparent information sharing and scientific guidance might help alleviate public concerns.

Coronavirus Disease 2019 (COVID- 19) , caused by a new coronavirus with higher reproductivity than SARS [1] , first emerged in the People's Republic of China in December 2019 [2] . Early outbreak data grew rapidly at an exponential rate [3] , and human-to-human transmission also occurred [4] , [5] , which brought severe challenges to China and the whole world. Soon vicarious traumatization caused by COVID-19 was found spreading in members of medical teams united in aiding the COVID-19 control and general public [6] . Social media has been found as a key platform for the public on information gathering and social learning to manage uncertainty and risks during a public health crisis. Gui et al. [7] investigated public concerns to the Zika virus crisis and reported mechanisms of personal risk assessment and travel-related decision making during the The associate editor coordinating the review of this manuscript and approving it for publication was Derek Abbott . crisis. Meanwhile, social media have been widely used by public health professionals for epidemiological monitoring and understanding public reactions to urgent public health issues. Pei et al. [8] developed methods to detect the intensity of social reaction with word-to-vector technique and context analysis. Tibebu et al. [9] analyzed real-time information on Twitter about opioid use and perceptions in Canada, which facilitated public health practice and opioid crisis addressing.

To explain and predict public emotional responses, especially the sentiment of distress and grief towards COVID-19, we analyze 999,978 microblogging posts from January 1, 2020 to February 18, 2020 on Sina Weibo 1 (Weibo for short), one of the most popular social media platforms in China with 550 million monthly active users in Quarter one, 2020. Weibo enjoys its traits of instant messaging, transparent sharing and publicly accessibility. In this work, the Deep Natural Language Processing (NLP) model and topic modelling method are utilized. To be specific, we fine-tune BERT for sentiment classification upon posts with three potential categories of sentiment, positive, neutral and negative, achieving a 75.65% of high accuracy, which surpasses many NLP baseline algorithms. The number of posts on each date is analyzed based on sentiment classification. Thereafter, TF-IDF model is adopted to extract central topics of posts. As the public sentiment on social media reflects people's psychological well-being and the spread of posts with negative sentiment may lead to social disruption and challenges for infection preventions [10] , we analyze 11 dominant and distinctive topics extracted from Weibo posts with negative sentiment and investigate the trends of sentiment development and the underlying major themes to understand public concerns.

Outbreaks are now taking place in many countries around the world, especially Europe and North America [11] . For instance, as of 25 July 2020, there have been more than 4 million people affected in the U.S. Under this circumstance, main contributions of this study include:

• We fine-tune BERT model for sentiment classification on Chinese Weibo posts about COVID-19 and achieve considerable accuracy that beats all baseline NLP algorithms.

• The study demonstrates how public sentiment on social media evolves as COVID-19 spreads.

• We extract representative topics and discuss the dominant discourse of public distress about COVID-19 caused by related social events. Findings of this study could assist governments worldwide in making efficient and effective public health protection decisions.

Coronavirus Disease 2019 (COVID-19) is a newly occurred disease that related research has barely been published by the time of conducting our study. However, there has been some studies on text sentiment classification, which to some extent relate to our work. Ye et al. [12] applied Machine Learning SVM [13] model on Chinese product reviews for sentiment (positive or negative) classification and achieved better performance than the classical Semantic Orientation approach. Narayanan et al. [14] built a fast sentiment classifier using Naïve Bayes, which achieved 88.80% accuracy on popular IMDB movie reviews dataset. In recent years, researchers have used more deep learning neural network techniques on sentiment classification. Ren et al. [15] enhanced word representation with character embeddings and mainly applied CNN for a context-sensitive sentiment classification on Twitter contents. Tang et al. [16] proposed sentiment classification upon documents by a combination of LSTM/CNN embedding and gated RNN.

Our sentiment analysis model consists of two parts, as shown in the workflow of Fig. 1 . In particular, we first use fine-tuned BERT [17] to classify the sentiment of Weibo posts into positive, neutral and negative categories and analyze the trends of posts. Then we apply TF-IDF [18] algorithm to extract topics of posts with different sentiment. Specifically, 11 topics for negative posts are generalized, and we then analyze the underlying patterns. VOLUME 8, 2020

Bidirectional Encoder Representations from Transformers (BERT), a neural network-based technique for natural language processing pre-training, has been largely applied in sentiment analysis [19] . The BERT model can be fine-tuned with proper input and output layers to create state-of-the-art models in a wide range of text analysis tasks [17] . The core of BERT is the adoption of transformer technique [20] , which perfectly applies encoder-decoder model [21] and attention mechanism [22] - [24] on NLP tasks. In particular, attention mechanism allows the model to focus on the relevant parts of the input sequence as needed when input sequence is too long for typical NLP models to memorize all input features. In our work, attention mechanism in BERT model helps determine the relevance weight of each word token in input of Weibo posts and generate corresponding hidden states that can best describe the characteristics of different sentiment of Weibo posts during the training process by (1), where Q, K , and V are three vectors, query, key and value, created based on input embedding, and d k is the dimension of key vectors.

After sentiment classification, the percentage of each sentiment category is calculated. In public health crisis, people's responses evolve over time [25] . To gain the insights into people's reactions, Weibo posts with positive, neutral and negative sentiment of each day from 1 January 2020 to 18 February 2020 are compared. Furthermore, in order to identify dates with rapid changes in a number of posts with different sentiment categories, the increasing rate is calculated by (2) and further analyzed according to critical events happened on the corresponding dates.

numberof posts on aday−numberof posts on theday before number of posts on theday before (2)

Term frequency-inverse document frequency (TF-IDF) is a numerical statistic reflecting how important a word is to a document in a collection or corpus [26] . The TF-IDF method is able to catch words that occur frequently by calculating term frequency and avoid insignificant words that occur in every document as important by the ability of inverse document frequency. Da Silva and Lopes [27] used TF-IDF to find the most informative Relevant Expression in each document in the corpus of their research. We perform the well-trained automatic sentiment classification model upon the unlabeled Weibo posts, and apply the built-in TF-IDF function in jieba [28] , a Chinese word segmentation tool that operates based on its huge pre-trained corpus, to posts in each labeled sentiment class for topic extraction by (3) for word t in a Weibo post d from the entire 999,978 Weibo posts dataset D.

In detail, we treat each Weibo post as a document and each segmented Chinese token as a potential topic. Due to the fact that most Weibo posts are not too long (length distribution without an outlier 884 is as shown in Fig. 2 ), topics with top 5 TF-IDF scores in each post are extracted and analyzed. Topics in each sentiment category represent public focus and concerns regarding COVID-19. Further analysis based on the extracted topics is performed. Thematic analysis is a common tool to understand the perceptions and reasons for people's posts with negative sentiment [29] . In our case, topics appearing in more than 1% of total posts in each category of sentiment (positive, neutral and negative) are collected. Five topics are excluded, including Pneumonia ( ), Outbreak ( ), Virus ( ), Coronavirus ( ), and COVID-19 ( ), because they directly represent COVID-19 and couldn't provide any valuable information in interpreting public sentiment. As a result, there are 38 key topics for positive sentiment, 19 for neutral sentiment, and 19 for negative sentiment, respectively. The 19 key topics for negative sentiment are compared with those of neutral sentiment and positive sentiment. Thereinto, 11 topics are found distinctive for negative sentiment and 8 topics are shared among posts with neutral or positive sentiment, where the latter are Masks ( ), Wuhan ( ), Definite Diagnosis ( ), Doctor ( ), Case ( ), Infection ( ), Quarantine ( ), and Hospital ( ). In addition, by exploring the semantics of topics, we divide the 11 negative-distinctive topics to four themes for further analysis. The four themes are Origin, Symptom, Production Activity, and Public Health Control. Moreover, we analyze the frequency of the 11 topics from 1 January 2020 to 18 February 2020 to visualize their trends. Manual sentiment labelling (positive, neutral and negative) is accomplished by CCIR 2020 organizer. In particular, open source Chinese sentiment analysis tools are adopted for preprocessing, and 12 volunteers were invited to finish the manual labeling on 120 thousand randomly selected Weibo posts from the dataset based on the preprocessing results and human decisions upon sentiment of Weibo contents. In detail, label of each Weibo post is decided by majority voting method of 3 volunteers who didn't know each other.

As a result, one million randomly selected posts from the whole dataset are shared to the public and 10% of them are manually labeled with three sentiment categories (positive, neutral and negative) by CCIR 2020 organizer.

Manually labeled Weibo posts are randomly split into training and testing sets with a ratio of 5 to 5. In our experiment, we fine-tuned the Chinese BERT-BASE model with 12 layers and hidden dimension 768. According to the suggestion on hyperparameters selection by the original BERT paper [17] , we set up parameters of 4 epochs, learning rate of 2e-5 and batch size of 32, and applied softmax neural network layer to train a three-category (positive, neutral and negative) sentiment classifier using the training dataset.

After training, the sentiment classification model achieves a 75.65% accuracy upon testing set. F1-scores on testing set along with precision and recall metrics for each sentiment category are summarized in Table 1 . An overall weighted F1-score 0.7458 for classification model is obtained on data in Table 1 by (4), where w is the weight of each sentiment category. The sentiment classification is then performed upon the rest unlabeled Weibo posts.

We performed comparative tests on the same dataset using different baseline sentiment classification algorithms with the same data split ratios and random state values. Results are presented in Table 2 . COVID-19 is first noticed in China in December 2019 [1] , but the number of Weibo posts remains stable and relatively low during the early period. On 20 January 2020, there is a surge in total Weibo posts and posts with negative sentiment as well, and the number of posts is kept at that level ever since. One important event might be related to this. On the night of 20 January Dr. Zhong Nanshan confirmed the human-to-human transmission of COVID-19 on China Central Television [31] . This surge of Weibo posts indicates the tremendous influence of information revealed by government-related media and it is therefore important for government to discreetly handle the public depression about COVID-19. Table 4 , the 11 key topics of Weibo posts with negative sentiment fall into four themes: Origin, Symptom, Production Activity, and Public Health Control. Typically, the number of posts in each theme for each day from 1 January 2020 to 18 February 2020 are plotted in Fig. 4-7 for further analysis.

There are three topics about the Origin of COVID-19 as shown in Fig. 4 . In particular, ''Gamey Food'' (3.08%) and ''Bat'' (2.70%) are two primary assumptions for the origins of COVID-19 [5] , [32] . They remain low frequency in the early January. However, there is a sharp increase in posts of ''Gamey Food'' on 20 and ''Bat'' on 22, January 2020, both then keeping a high frequency thereafter and reaching the highest number of posts with negative sentiment per day among 11 topics. And the ''Conspiracy Theory'' (1.43%) suggesting that COVID-19 does not have a natural origin is condemned by scientists [33] but widespread on social media. However, it remains in a relatively low frequency in comparison with ''Gamey Food'' and ''Bat''. The discussions of the origins of virus are deeply correlated to negative sentiment and trigger the largest amount of posts with negative sentiment as demonstrated in Fig. 4 . However, as discussed, rumors and unconfirmed information may overwhelm the discussion [33] . Consequently, it is important for the government to release the transparent progress about the investigation of origins.

''Fever'' (2.13%) and ''Cough'' (1.19% ) are identified as the representative symptoms of COVID-19 [34] . For posts about Symptom as shown in Fig. 5 , posts with ''Fever'' as topic outnumbered that with ''Cough'', and the gap is quite large in early January but declines in February as COVID-19 spread. Symptoms about COVID-19 might lead to negative sentiment but could also be beneficial for self-detection of infection. Therefore, typical symptoms of disease should be revealed to the public clearly and timely, which would benefit early detection of the disease.

Production Activity as pictured in Fig. 6 summarizes topics about work-life arrangements under the threat of COVID-19. ''Go to Work'' (1.94%) and ''Resume Work'' (1.12%) portray the public concern over work, and ''School New Semester Beginning'' (1.06%) indicates people's worries about students going back to school. The concern on ''Go to Work'' starts from early January and remains relatively high while the one on ''School New Semester Beginning'' starts to grow from 20 January 2020 and the worries for ''Resume work'' starts to grow from 26 January 2020. The arrangements for production activity might be an important driving force for the general depression. The peak of concern for ''School New Semester Beginning'' in Fig. 6 comes earlier than that for ''Resume Work'', thus earlier arrangements are necessary in easing people's tension. Three topics about Public Health Control (Fig. 7) depict different aspects. ''Temperature Taking'' (1.39%) is an important method for COVID-19 diagnosis. ''Coronavirus Cover-up'' (1.26%) is the misbehavior or fault in public health control which might lead to risk of more infections [35] . And ''City Shutdown'' (1.09%) describes the strict public health control taken by the Chinese government [36] .

There are almost no negative posts about Public Health Control in the early January. The peak discussions are ''City Shutdown'' the first on January 23, ''Coronavirus Cover-up'' the second on January 26, and ''Temperature taking'' the third on January 27, and the three topics reach the same level of attention around 18 February 2020. This trend demonstrates the focuses on different control measures as situation develops. Towards this end, the public health control and opacity of information may also lead to people's depression. Therefore, the corresponding public health control measures at a certain time period should be elaborated for public acceptance.

The spread of COVID-19 has turned to a worldwide pandemic thus far [37] . Public health concerns not only relate to the infection prevention but also the psychological status of people experiencing the disaster [38] . Therefore, analyzing posts with negative sentiment from social media could contribute to understanding the experiences of Chinese general public during the outbreak of COVID-19 and offers examples for other countries. Our analyses provide insights on the evolution of social sentiment over time and the topic themes connected to negative sentiment of Weibo posts. Fig. 3 illustrates the clear outbreak dates for public attention about COVID-19. Moreover, concerns about Origin, Symptom, Production Activity, and Public Health Control are deeply intertwined with the public sentiment.

This study collects data on social media from early stage of COVID-19 transmission in China. Based on the data analysis and discussion, several advantages emerge. First, state-ofthe-art fine-tuned BERT classification model and TF-IDF topic extraction model deliver results with considerable accuracy. Second, it can further be implemented as an online platform for real-time monitoring on public sentiment during other crises in the future. Third, this study reveals important topic themes which are deeply connected to sentiment of depression. As the infection of COVID-19 keeps spreading worldwide now, insights from this study may contribute to public administration and prevention of social disruptions.

Despite of informative results found in this study, further improvements are expected on the classification model to achieve a higher accuracy. Furthermore, only information on Sina Weibo is used in this study, which may lead to bias by neglecting posts on other social media platforms. Finally, in order to focus on the centrality of topics, only topics appearing in more than 1% of total posts are selected in each category of sentiment. This may lead to the overlook of important topics with less percentage. Future studies by incorporating information in empirical data from different social media platforms and different countries may contribute to a more solid conclusion.

New outbreaks are taking places in many other countries all around the world. The sentiment classification model and findings of this study would provide constructive instructions for governments worldwide on making efficient and effective public health protection decisions. VOLUME 8, 2020 TIANYI WANG is currently pursuing the Ph.D. degree with the Department of Computer Science, The University of Hong Kong. His major research interests include machine learning, deep learning, and cyber security.

KE LU is currently pursuing the Ph.D. degree with the Department of Social Work and Social Administration, The University of Hong Kong. His major research interests include nonprofits, organization development, social enterprise, social innovation, crowdfunding, and philanthropy. Recently, he has been involved in studying the organization development of education nonprofits in China.

KAM PUI CHOW received the Ph.D. degree in computer science from the University of California, Santa Barbara, USA.

He is currently an Associate Professor with the Department of Computer Science, and the Director of the Center for Information Security and Cryptography (CISC), The University of Hong Kong (HKU). He is also the Leader of the Computer Forensics Research Group (CFRG). In the years 1994-1997, he together with other professionals and a team of software engineers developed the search engine for Hong Kong Telecom's 108 Telephone Directory Enquiry System using state of the art technology in main memory database and distributed computing. In the recent years, his research interests include migrated to digital forensics and computer security. His research interests include computer forensics, digital investigation, data privacy, cryptography, and computer security.

QING ZHU received the Ph.D. degree in cardiology from Shandong University, Jinan, China, in 2003.

She is currently a Cardiologist and a Chief Physician with the Qilu Hospital of Shandong University, and also an Associate Professor and the Master Tutor of the Medical College of Shandong University. Her research interests include the clinical prevention treatment and basic research of lipid metabolism abnormality and atherosclerosis, electrophysiological mechanism of arrhythmia and radiofrequency ablation, and chronic disease management of cardiovascular disease. VOLUME 8, 2020 

The reproductive number of COVID-19 is higher compared to SARS coronavirus

Novel Coronavirus (2019-nCoV) Situation Report-1. World Health Organization

Preliminary estimation of the basic reproduction number of novel coronavirus (2019-nCoV) in China, from 2019 to 2020: A data-driven analysis in the early phase of the outbreak

COVID-19) Situation Report-45. World Health Organization

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and coronavirus disease-2019 (COVID-19): The epidemic and the challenges

Vicarious traumatization in the general public, members, and non-members of medical teams aiding in COVID-19 control

Managing uncertainty: Using social media for risk assessment during a public health crisis

A new method for early detection of mass concern about public health issues

Ata-glance-What can social media tell us about the opioid crisis in canada?'' Health Promotion Chronic Disease Prevention Canada

Twitter sentiment classification for measuring public health concerns

Centers for Disease Control and Prevention

Sentiment classification for chinese reviews: A comparison between SVM and semantic approaches

SVM and logistic regression,'' in The Nature of Statistical Learning Theory

Fast and accurate sentiment classification using an enhanced naive Bayes model

Context-sensitive twitter sentiment classification using neural network

Document modeling with gated recurrent neural network for sentiment classification

BERT: Pre-training of deep bidirectional transformers for language Understanding

TF-IDF,'' in Encyclopedia of Machine Learning

Target-dependent sentiment classification with BERT

Attention is all you need

Sequence to sequence learning with neural networks

Effective approaches to attentionbased neural machine translation

Neural machine translation by jointly learning to align and translate,'' presented at the 6th Int

Structured attention networks,'' presented at the 5th Int

Evolution of complex disasters

Data mining,'' in Mining Massive Datasets

A document descriptor extractor based on relevant expressions

Disease detection or public opinion reflection? Content analysis of tweets, other social media, and online newspapers during the measles outbreak in The Netherlands in 2013

Zhong Nanshan Confirmed the Human-to-Human Transmission of COVID-19

Public exposure to live animals, behavioural change, and support in containment measures in response to COVID-19 outbreak: A populationbased cross sectional survey in China

Statement in support of the scientists, public health professionals, and medical professionals of China combatting COVID-19

Pathological findings of COVID-19 associated with acute respiratory distress syndrome

Full spectrum of COVID-19 severity still being depicted

Taking the right measures to control COVID-19

COVID-19)-Events as They Happen. World Health Organization

Social media messages in an emerging health crisis: Tweeting bird flu

The authors would like to thank J. Ma from Shandong University for cogent advice on data analysis viewpoint and format issues.