key: cord-0882920-pmjo6set
authors: Liu, Jun; Liu, Liyi; Tu, Yan; Li, Shixuan; Li, Zongmin
title: Multi-stage Internet public opinion risk grading analysis of public health emergencies: An empirical study on Microblog in COVID-19
date: 2021-10-26
journal: Inf Process Manag
DOI: 10.1016/j.ipm.2021.102796
sha: a0e1ea8ebb32eb7c8ee619f823565ea892b5c7a3
doc_id: 882920
cord_uid: pmjo6set

In the period of Corona Virus Disease 2019 (COVID-19), millions of people participate in the discussion of COVID-19 on the Internet, which can easily trigger public opinion and threaten social stability. This paper creatively proposes a multi-stage risk grading model of Internet public opinion for public health emergencies. On the basis of general public opinion risk grading analysis, the model continuously pays attention to the risk level of Internet public opinion based on the time scale of regular or major information updates. This model combines Analytic Hierarchy Process Sort II (AHPSort II) and Swing Weighting (SW) methods and proposes a new Multi-Criteria Decision Making (MCDM) method – AHPSort II-SW. Intuitionistic fuzzy number and linguistic fuzzy number are introduced into the model to evaluate the criteria that cannot be quantified. The multi-stage model is tested using more than 2,000 textual data about COVID-19 collected from Microblog, a leading social media platform in China. Seven public opinion risk assessments were conducted from January 23 to April 8, 2020. The empirical results show that in the early COVID-19 outbreak, the risk of public opinion is more serious on macroscopic view. In details, the risk of public opinion decreases slowly with time, but the emergence of important events may still increase the risk of public opinion. The analysis results are in line with the actual situation and verify the effectiveness of the method. Comparative analysis indicates the improved method is proved to be superior and effective, sensitivity analysis confirms its stability. Finally, management suggestions was provided, this study contributes to the literature on public opinion risk assessment and provides implications for practice.

In the period of Corona Virus Disease 2019 , millions of people participate in the discussion of COVID-19 on the Internet, which can easily trigger public opinion and threaten social stability. This paper creatively proposes a multi-stage risk grading model of Internet public opinion for public health emergencies. On the basis of general public opinion risk grading analysis, the model continuously pays attention to the risk level of Internet public opinion based on the time scale of regular or major information updates. This model combines Analytic Hierarchy Process Sort II (AHPSort II) and Swing Weighting (SW) methods and proposes a new Multi-Criteria Decision Making (MCDM) method -AHPSort II-SW. Intuitionistic fuzzy number and linguistic fuzzy number are introduced into the model to evaluate the criteria that cannot be quantified. The multi-stage model is tested using more than 2,000 textual data about COVID-19 collected from Microblog, a leading social media platform in China. Seven public opinion risk assessments were conducted from January 23 to April 8, 2020. The empirical results show that in the early COVID-19 outbreak, the risk of public opinion is more serious on macroscopic view. In details, the risk of public opinion decreases slowly with time, but the emergence of important events may still increase the risk of public opinion. The analysis results are in line with the actual situation and verify the effectiveness of the method. Comparative analysis indicates the improved method is proved to be superior and effective, sensitivity analysis confirms its stability. Finally, management suggestions was provided, this study contributes to the literature on public opinion risk assessment and provides implications for practice.

On December 8, 2019, the first case of unexplained pneumonia was officially reported. On January 22, 2020, more than 100 cases were confirmed in a single day in Hubei Province. At 10 a.m. on January 23, 2020, Wuhan announced the closure of the city, suspended the operation of the subway, ferry, bus, and other public transport, temporarily closed the airport and railway station . In emergencies, because people are worried about their health and safety, Internet users will search more frequently for potential information that may threaten their health on the Internet (Li, Wang et al., 2020; McMullan et al., 2019) . Some mainstream social platforms, such as QQ, Weibo, Wechat, Facebook, Corona Virus Disease 2019 (COVID-19) related topics are full of discussions and very hot. According to ''The report on public cognition and information dissemination of COVID-19 1 '', during the survey period (from January 20, 2020 to February 13th), 90% of the public maintained a high degree of concern about the epidemic related information, 47% of the public closed the epidemic information every day, and 43% of the public was concerned about the epidemic at all times.

Through the platform on the Internet, people can fully and freely express their views and exchange views with others. At the same time, false information will also spread among people. Vosoughi et al. (2018) assume that social media expands the scope of Internet users' communication, which will accelerate the spread of false information on the Internet. Influenced by COVID-19, most people communicate with others only by using social platforms, and the phenomenon of false information dissemination becomes more serious (Apuke & Omar, 2021) . For example, in the early COVID-19, there were rumors that alcohol, Shuanghuanglian (a drug for treating cold) and Indigowoad Root (an antiviral drug) could prevent COVID-19 on Chinese social media platform, which led to the online and offline stores of Shuanghuanglian and Indigowoad Root being snapped up.

People tend to express their thoughts and attitudes towards emergencies on the Internet (Svensson, 2014) . With the advent of ''we media era'', it means that anyone can become a speaker on the Internet, personal views are particularly prominent on the Internet (Wang et al., 2019) . Internet public opinion is the sum of people's views, attitudes, and emotions about public health emergencies . The different views and opinions of the users on emergencies are essential factors for the formation of Internet public opinion. As the Internet provides a rapid and widespread public opinion, uncontrolled dissemination of false information can easily lead to helplessness and panic of the public, and also reduce the credibility of the government (Saltzman et al., 2020) . The unstable Internet public opinion may affect the stability and development of society . Government intervention is one of the effective means to reduce the negative influence of rumors and public opinion (Huo & Ma, 2017) . The cooperation between the media and the government can reduce the damage caused by public health emergencies (Li, Liu et al., 2020) . In the context of the spread of public health emergencies, to ensure the flawless operation and development of Internet space, it is necessary to study the governance methods of Internet public opinion and determine the risk level. Surprisingly, there are few or no literatures that can be retrieved, which consider the multi-stage model when judging the Internet public opinion level. Most of the literatures only considered static public opinion risk assessment, while public health emergencies generally last for a long time and the information is updated frequently. Therefore, it is necessary to conduct multiple risk assessments on the same platform in a certain period of time.

To address this research gap, this study aims to establish a multi-stage Internet public opinion risk grading evaluation model (more suitable for public health emergencies), and obtain the results of Internet public opinion risk level of Microblog platform during the early period of COVID-19, and provide management suggestions for the government, relevant enterprises and users. The final results have reference value for the government to control public opinion and reduce the negative impact of public opinion, and are of great significance in guiding users to avoid the risk of Internet public opinion.

In summary, to explore Internet social media platform's public opinion risk level during COVID-19 and to reduce the negative effects of Internet public opinion, the following four questions are studied in this paper:

(1) what are the main differences between public health emergencies (COVID-19) and general emergencies in Internet public opinion? (2) How to establish a risk grading evaluation model that can keep a long-term monitoring of Internet platform public opinion risk? (3) How to get a more realistic results of Internet public opinion rating with the help of Analytic Hierarchy Process Sort II (AHPSort II) and Swing Weighting (SW) methods? (4) How can the government, Internet companies and users reduce the risk of public opinion and avoid the negative harm? The results can contribute to the government, Internet enterprises and the public from both theoretical and practical aspects. For example, these results are expected to help relevant Internet enterprises find public opinion risk in time, clarify rumors, limit users to speak with negative emotions, so that they can restrict the spread of negative news and false news among the public, and help the government better guide users to implement policies. Multi-Criteria Decision Making (MCDM) is desirable in supporting the solution of complex problems (Stewart, 2005) . AHPSort II and SW methods provide reliable help for evaluating the risk level of Internet public opinion. This study also broadens the research field for MCDM methods and provides a reference for future scholars to study the risk level of Internet public opinion.

The rest of the structure is as follows. Section 2 reviews the literature on Internet public opinion, rumor detection and MCDM methods in the application of Internet public opinion. Section 3 introduces the background knowledge of two types of fuzzy numbers. Section 4 explains the interpretation mechanism of Internet public opinion and proposes a multi-stage Internet public opinion risk classification model. Section 5 introduces the AHPSort II-SW method, which is used to analyze and grade the Internet public opinion of public health emergencies. The COVID-19 risk rating is demonstrated in Section 6, the AHPSort II-SW method is applied to Microblog's public opinion risk rating in the period of COVID-19, and some management suggestions are given. Finally, Section 7 summarizes the conclusions of this paper and points out the future direction of improvement.

Internet public opinion has always been the research content concerned by many scholars. After the outbreak of COVID-19, the research on Internet public opinion under the epidemic has sprung up. These studies play a positive role in controlling the Internet public opinion risk. Moreno et al. (2020) evaluated the impact of different information sources on public information behavior. COVID-19 will affect people's attitudes towards infectious diseases. The influence of weather on the spread ability of COVID-19 has always been a controversial topic. Gupta et al. (2021) collected the views of Twitter users on the influence of weather and used machine learning/natural language processing technology to draw the conclusion: 40.4% of users indicated uncertainty about weather's impact, 33.5% indicated no effect, and 26.1% indicated some effect. Zhong (2020) used Python to search relevant data on ''Baidu Post Bar'' (the world's largest Chinese forum). According to Latent Dirichlet Allocation (LDA) analysis results, most public had negative emotional performance. Similarly, Yao et al. (2021) collected 334 questionnaires from ''Baidu Post Bar'' to analyze the relationship between public cognition, emotion and belief, and concluded that the disclosure of information has a great impact on the public's emotions. COVID-19 has been developed worldwide. Governments need to take into account their national conditions when formulating policies. Wu et al. (2021) analyzed the policy changes of Chinese government in the period of COVID-19 from the perspective of China. The special multi-agency, joint epidemic prevention and control mechanism has been effectively helped to implement the relevant policies and control the virus. In Italy, Vezzoni et al. (2020) applied rolling cross-section (RSC) to monitor the change trend of Internet public opinion during COVID-19. Xing et al. (2021) analyzed the difference of Internet public opinion between the largest social media platform in China (i.e., Microblog) and the largest social media platform in the United States (i.e., Twitter) from the perspective of national culture. By analyzing the comments of Singapore media, Shorey et al. (2020) found that the main feeling of the Singapore public about the epidemic virus was fear and worry. Bates et al. (2020) suggested that health education should not only pay attention to the relevant health knowledge but also promote the optimistic mood through the research on the related knowledge of COVID-19 of Erdogan people. Through an online survey of the Turkish public, Geçer et al. (2020) found that online news and social media were the most accepted sources of information.

The COVID-19 topic on Facebook has been discussed around 3 billion times worldwide, most users will actively or passively receive false information or rumors when they search for the information they want on Twitter (Cato et al., 2021) . The spread of rumors is an integral part of the formation of Internet public opinion, so how to detect and identify Internet rumors has become an essential direction to control the development of public opinion. Some researchers provided a variety of rumor detection methods and analyzed the effectiveness of rumor detection (Cao et al., 2018) . Most researchers on rumors prefer to study on Microblog, Twitter and YouTube (Liu & Liu, 2018; Obadimu et al., 2021) , both of which are among the largest social platforms in the world. Li et al. (2021) developed a rumor repudiation effectiveness index (REI) for the actual rumor repudiation effect of social media and put forward decision suggestions for the rumor repudiation of Microblog. In the aspect of false image identification, Xue et al. (2021) proposed a Multimodal Consistency Neural Network (MCNN), which solved the problem of mismatch between image and text in false news. Compared with traditional methods, its detection accuracy was also significantly improved.

In summary, the existing researches on internet public opinion mainly focus on the emotional analysis on social media (Shorey et al., 2020; Yao et al., 2021) , the cause analysis of public opinion and rumor detection Xue et al., 2021) . These studies have played a positive role in the prevention and control of rumors and the governance of public opinion.

In the above articles, there are many pieces of research on Internet public opinion. However, few articles talk about the combination of Internet public opinion and MCDM method. MCDM is often used in the research of supplier problems (Alidoosti et al., 2021; Ar et al., 2020; Feng & Gong, 2020; Hong & Mwakalonge, 2020; Mina et al., 2021) . However, a few scholars apply MCDM to the field of Internet public opinion. Lu et al. (2019) innovatively combined decision-making trial and evaluation laboratory (DEMATEL) with analytical network process (ANP) to analyze the critical factor that affect user's diffusion behavior in an emergency. Ishizaka et al. (2012) extends the AHP method from the ranking problem of MCDM to the sorting problem, and proposed the AHPSort method. AHPSort II method is a development of AHPSort method, it was proposed by Miccoli and Ishizaka (2017) , which was first used in Umbria according to the risk of wolf attacks. Subsequently, the AHPSort II has been evolved by considering group decisionmaking (Assumma et al., 2021; López & Ishizaka, 2017) . Mei et al. (2019) used AHPSort II and DEMATEL method to analyze the earthquake public opinion level of Yibin City, Sichuan Province. As an extension of the AHPSort method, AHPSort II has fewer comparison times, more convenient usage, more accurate results, and its effectiveness has been verified . Labella et al. (2020) used AHPSort II to assess the performance of EU countries to reduce inequality. Xu et al. (2019) combined interval type-2 fuzzy sets (IT2FSs) with AHPSort II to show a new supply chain management method, which is proved to have higher priority accuracy than traditional methods. This paper selects the SW method for the criteria system weighting to get more consistent with the actual weights of criteria data. In the SW method, the idea of turning the worst into the best is in line with the original intention of Internet public opinion governance, and the data obtained is more appropriate to the reality and people's expectations. SW method has been proved to be effective in drug review (Bonomo et al., 2019) , medical decision-making (Broekhuizen et al., 2017; Németh et al., 2019; Tervonen et al., 2017) , infrastructure (Jayasooriya et al., 2019) , supply chain risk management (Qazi et al., 2018) and other fields (Zheng & Lienert, 2018) .

With the development of time, many MCDM problems in different fields have been solved by experts and scholars. MCDM methods are applied to more and more different fields (Li et al., 2019; Pelissari et al., 2019) . AHPSort II and SW method are among the mature MCDM methods. Based on this, this paper will provide a new Internet public opinion risk grading model combined AHPSort II and SW method. 

In this paper, intuitionistic fuzzy number and linguistic fuzzy number are used to evaluate the criteria that cannot be expressed by specific values. The introduction and related calculation formulas of intuitionistic fuzzy number and linguistic fuzzy number are as follows.

Let be a non-empty universe, and the intuitionistic fuzzy set on is defined as

where ( In the following, an improved scoring function is proposed. Let = {( , ( ), ( )) | ∈ } be an intuitionistic fuzzy number. The score of can be evaluated by the modified score function ( ), as follows (Wang & Xin, 2005) :

where ( ) ∈ [0, 1]. The proportion of neutral people who tend to vote in favor is assigned as

. That means, the proportion of neutrals who may vote for is considered to be 0.5 at first, and then the proportion of assignment is corrected by half of the difference between the affirmative and negative votes, to reflect that the more people vote for, the greater the proportion of neutrals who tend to vote for, and vice versa.

Linguistic fuzzy number is mainly used in the evaluation information, which cannot be used in real number, interval number, triangular fuzzy number, and intuitionistic fuzzy number. Linguistic fuzzy number is generally expressed in natural language, which is a qualitative description of the evaluation object. It is used to evaluate the performance of alternatives, such as ''Poor'', ''Medium Poor'', ''Medium'', ''Medium Good'', ''Good'', etc. Decision Makers (DMs) use familiar and straightforward linguistic terms to evaluate alternatives, which helps DMs get rid of the shortage of having to evaluate alternatives in digital form, it is more in line with the subjective will of DMs. In order to accurately distinguish the advantages and disadvantages of the attributes of things, it is generally divided into 5-9 grades. Linguistic fuzzy number is evaluation information expressed in words, and it cannot be directly used for calculation. Therefore, it needs to be converted into triangular fuzzy number or intuitionistic fuzzy number in advance. In this paper, linguistic fuzzy numbers are transformed into intuitionistic fuzzy number, and 9-level linguistic fuzzy numbers are used, which are converted into intuitionistic fuzzy numbers. The evaluation scales and corresponding intuitionistic fuzzy number are shown in Table 1 .

It has become the most common way for the masses to express their opinions and ideas in public health emergencies through the Internet. The trend of Internet public opinion will seriously affect the work of the government and people's daily life. Public opinion risk control will help the government and Internet users deal with public health emergencies more calmly. In the following, the formation mechanism and multi-stage risk evaluation model of Internet public opinion in public health emergencies will be introduced.

Compared with general emergencies, public health emergencies have a distinct feature. Public health emergencies are often persistent in time. The occurrence time of general emergencies is very short, such as earthquakes, traffic accidents. The period of public health emergencies is more extended than general emergencies, it can even last up to several years. With the passage of time, the event will produce many new changes, which will not only affect the prevention and control of the epidemic, but also affect the Internet public opinion.

Internet public opinion is often formed after the occurrence of general emergencies, when people have got rid of the direct impact of the event, and they are in a non-threatening environment. They exchange and express opinions about the relevant information of emergencies on the Internet. In the process of forming online public opinions on public health emergencies, people often voice on the Internet when the event is not over due to the long duration of the event, and their security is still threatened by the event. A pandemic such as COVID-19 does not only have connotations from the point of view of physical health. The economic and financial situation of families as a consequence of such a pandemic means that we must take mental health also into consideration. There are three differences between public opinion generated by public health emergencies and general emergencies.

(1) Users pay more attention to the event. Different from general emergencies, the update speed of public health emergency information is very fast. The Internet is the best platform to display these information updates. People generally pay attention to the changes of information on the Internet. At the beginning, only the virus was of interest, and later the consequences and the direct impact of the virus on daily life. In fact, people need to get the latest information from the Internet to adjust their work and living arrangements. Therefore, users are forced to pay attention to the information related to the event. The attention of public health emergencies on the Internet will be significantly greater than that of general emergencies.

(2) Users tend to have more extreme emotions. Because of the characteristics of public health emergencies, people will pay more attention to their safety and health. Suppose some things are subconsciously thought to be harmful to their safety and health. Fear and anxiety about a new disease and what might happen can be overwhelming and provoke strong emotions in the population. Tracking people's emotions and general sentiment through social media about the COVID-19 epidemic reveals this: The current COVID-19 pandemic is creating additional stress on our emotional well-being. In that case, people will be more extreme in dealing with the problems, and the possibility of irrational behavior will be greater.

(3) The possibility and risk of public opinion formation are more serious. Due to the above two reasons, the proportion of irrational words and behavior in the overall Internet environment will increase significantly. This not only provides the foundation for the formation of negative Internet public opinion but also brings challenges to the governance of Internet public opinion.

Generally speaking, the public opinion risk caused by public health emergencies is higher than that of general emergencies, and the difficulty of public opinion control and governance will be more serious, which requires the government and relevant institutions to take efficient measures for public opinion risk management. Public opinion risk management can reduce the negative impact of public health emergencies on the Internet and improve the credibility of the government. Moderately controlling and scientifically guiding negative public opinion is conducive to social harmony and stability, and also helps the government to eliminate the harm of public health emergencies to the society and the public. Therefore, the governance, guidance and restraint of Internet public opinion should become an important part of the overall public health emergencies management.

The Internet is not only a platform for media to release event information, but also an important carrier for users to obtain information and express their opinions. The Internet is a multi-party communication platform. Therefore, it is of great significance to study the formation process of Internet public opinion as shown in Fig. 1 .

When public health emergencies occur, the media will release the mainstream information on the Internet platform, which is generally objective facts without personal emotions. This information will be known to users who follow these media. Furthermore, it spreads to each other through other social platforms or chat tools. The scope of these mainstream information gatherers has been expanded. At the same time, the behavior of some Non-Governmental Organizations may be exposed on the Internet. Some of the information will encourage the masses. For example, on February 6, 2020, SGMW (an automobile production company) announced that it would convert joint suppliers to produce masks by rebuilding production lines. Another part may aggravate people's panic. For instance, some supermarkets greatly increased the price of vegetables during the epidemic. Affected by these things, users choose to express their views on the Internet to meet their moral needs according to the harmful degree of the information they get. All kinds of different opinions collide with each other on the Internet, which will produce public opinion. With the passage of time and the continuous development of events, the actual situation of public health emergencies will produce many changes. These new changes will bring a lot of new information. One of the responsibilities of the media and the government is to timely disclose the latest relevant information to the public on the Internet, and the latest information will also form Internet public opinion through the above process. The public opinion generated by new information will overlap and interact with the public opinion generated by old information. At this time, the development direction and influence scope of public opinion begin to become uncertain. If there is no timely and correct guidance and control of public opinion, these public opinions will have a huge negative impact on the country and society. It will also hinder the handling of public health emergencies in real life. Therefore, the main task of the government is to analyze and manage the negative public opinion on the Internet, and try to eliminate the negative impact that public opinion may bring, and ease the user's mood. The user's emotions directly affect the user's views on the Internet, extreme user's emotions will lead the Internet public opinion to the wrong side. Therefore, it is imperative to classify the public opinion risk level of public health emergencies in time. 

This paper proposes a model of Internet public opinion detection, classification and governance of public health emergencies based on a multi-stage risk classification model, which can be seen in Fig. 2 . The model is mainly composed of analysis process and governance process. The multi-stage describes that in addition to the first public opinion level analysis after the occurrence of public health emergencies, it can quickly start the public opinion level analysis program again after the event changes or major information updates. This is particularly important in the Internet public opinion risk classification of public health emergencies, which is also the core content of this paper.

In the initial stage of public health emergencies, Internet public opinion is mainly affected by the scale and type of event, government response speed and other information. In order to understand the risk level of Internet public opinion, it is necessary to analyze Internet public opinion. For the first time, a criterion system needs to be established, which is usually completed by experts brainstorming and learning relevant literature. After that, it is also necessary to collect public opinion information of public health emergencies related to the Internet platform, and select experts with specific experience to use this method to analyze the risk level of Internet public opinion on the platform. According to the analysis results, the corresponding treatment method is selected to reduce the risk level of public opinion. With the development of the emergencies, more and more new information has been added to the Internet, such as the spread of rumors, the expansion of the scope, the introduction of new government policies. At this time, the public opinion risk level of the Internet platform will change to a certain extent. Therefore, it is necessary to analyze and evaluate the public opinion risk level of the platform again. Multi-stage Internet public opinion risk analysis can continuously evaluate the public opinion risk of Internet platform, so that the government can take measures to reduce the negative impact of public opinion. Compared with the first start of public opinion grade analysis, this process reduces the process of experts brainstorming to establish the criterion system, dramatically reduces the evaluation time, and can reflect the risk of Internet public opinion at that time more timely, which is conducive to the government to formulate public opinion governance methods faster. The risk levels are divided into ''High risk'', ''Medium risk'', ''Low risk'' and ''No risk''. When the risk levels are ''High risk'', ''Medium risk'' and ''Low risk'', corresponding management measures should be taken to reduce the risk. The ''No risk'' level means that the platform has no Internet public opinion risk at present, and no intervention measures need to be taken. However, it is necessary to continue to pay attention to the trend of public opinion and respond to the possible changes of public opinion risk in the future. In order to continuously monitor and manage Internet public opinion and improve the effectiveness of public opinion governance, we suggest that after taking corresponding measures to reduce the risk of Internet public opinion, the government should test the effect of public opinion intervention and governance, and analyze the risk level of Internet public opinion again. If the risk is reduced, it proves that the public opinion intervention has a significant effect. If the risk is constant or increased, it proves that the effect of public opinion intervention is not significant. Adjustment of intervention strategies and control measures should be considered to eliminate the negative impact of Internet public opinion as soon as possible. It is worth mentioning that the model is also applicable to Internet public opinion risk research in other countries except China. However, due to different national conditions around the world, it is necessary to analyze all stakeholders who have a direct or indirect impact on citizens when considering the risk of Internet public opinion. Therefore, when trying to apply the model to other countries, we should make some flexible adjustments to the model to meet the specific situation of the studied object. For example, in religious countries such as India and Pakistan, the impact of religion on people's thought and speech must be included in the analysis of the model; In the United States, Japan and other Multiparty system countries, whether the ideology of different parties will increase public uncertainty still needs to be analyzed in detail. The national conditions of different countries vary greatly, and there are barriers between Chinese social platforms and foreign social platforms. Therefore, this paper only analyzes Microblog, the largest social media platform in China. It should be emphasized that the multi-stage Internet public opinion risk grading model proposed in this paper is still applicable to other countries and regions.

The risk classification of platform Internet public opinion is vital for the risk control and governance of Internet public opinion. This paper combines AHPSort II and SW and proposes a extended sorting method called AHPSort II-SW method. The detailed steps are as follows.

Step 1. Define the goal and criteria. The goal of classification of public opinion risk on Internet platform is to select the risk level of public opinion spread by different Internet platforms in a special dangerous period, and manage and control the public opinion in the platforms. Define the goal, the criteria , = 1, 2, … , , and the experts , = 1, 2, … , with respect to the problem. There are three different data types: real, intuitionistic fuzzy and linguistic fuzzy. Specially, real data is called , = 1, 2, … , , this kind of data is obtained through data collection. The other two data types are called , = 1, 2, … , , and = + , obtained through experts evaluation.

Step 2. Define the risk levels of Internet public opinions. The risk levels are defined as High risk (H), Medium risk (M), Low risk (L) and No risk (N). High risk means that Internet public opinion has had or will have a negative impact, mandatory management measures need to be taken immediately; Medium risk means that Internet public opinion is likely to have a negative impact, appropriate management measures need to be taken; Low risk means that Internet public opinion may have a negative impact, moderate management measures can be considered; No risk means that the Internet public opinion risk has not been found for the time being, management measures may not be taken, but it is necessary to continue to pay attention to the public opinion risk. Each risk level is called a class, and the class is defined as , = 1, 2, … , . Different risk levels represent the risk degree of public health emergencies in the Internet platform. The higher the risk level, the more attention should be paid to the platform.

Step 3. Determine the evaluation of experts, who are usually composed of local government officials, managers of social media platforms, researchers in the field of MCDM, etc. Invite experts to evaluate according to the in criterion system and get the initial evaluation matrix = :

During the COVID-19, we propose to use the form of ''online meeting'' to carry out this process, so as to reduce personnel gathering and some unnecessary risks. ''Online meeting'' is very popular in China and has been adopted by many companies and units, which provides a new way for people to work during the epidemic (Seery & Flaherty, 2020) .

Step 4. Determine the expert weights. Each evaluation standard is assigned by the experts who are familiar with this field. The linguistic fuzzy numbers given by experts are transformed into intuitionistic fuzzy number. So a complete evaluation matrix on fuzzy criteria for each expert ′ = ′ can be obtained as follows:

⎦ Expert weights can be obtained by intuitionistic fuzzy number and linguistic fuzzy number transformed into intuitionistic fuzzy number. Let = ( , , ) be an intuitionistic fuzzy number for rating of the th expert. Then the weight of the ℎ expert can be obtained (Zhang & Liu, 2011) as:

where ∑ =1 = 1.

Step 5. Determine the weights of criteria. In this paper, the SW method is used to determine the weights of criteria. The weights of criteria value determined by the SW method indicates the impact of the criteria on the risk of Microblog public opinion. The greater the weight value is, the higher the impact is. The results accord with the objective facts. Suppose a worst case: in this case, the performance of all criteria are the worst. Its criteria performance is described as ( 1 , 2 , … , ), and the risk of public opinion is the highest. Suppose a best case: in this case, the performance of all criteria are the best. Its criteria performance is described as ( 1 , 2 , … , ), and the risk of public opinion is the lowest. Experts, same as Step 3, will choose the criterion 1 , = 1, 2, … , that most need to be improved from the worst assumptions to the best. The 1 is given a score of 100 as the initial weight of the criterion. Similarly, ask experts to select other criterion , = 1, 2, … , that are most important to improve from the worst to the best in order, and assign scores to the criteria through group negotiation, with the score range of 0 − 100, marked . Especially, 1 = 100, ≤ 100 and ≥ +1 ( ≥ 2). According to the importance ranking selected by the experts, the score can be reduced in turn until the last criterion is scored.

Firstly, the total score is calculated and the initial weight is normalized according to the proportion of each item in the total score. The calculation formula is

The final weight can be adjusted according to the actual situation, and the adjusted weights are the weights of criteria.

Step 6. Score function and data aggregation. Through the aggregation formula of intuitionistic fuzzy number, the evaluation values of many experts are aggregated into an intuitionistic fuzzy number according to the expert weights. Numerical data is the objective data collected, and does not need experts evaluation, so it does not need to be aggregated. Aggregation of intuitionistic fuzzy data. The expert weights and individual subjective evaluation information are collected by the above formula, and the group evaluation matrix is obtained by the aggregation operator. Let ( , , ) be the intuitionistic fuzzy number and the linguistic variable type subjective evaluation information transformed into intuitionistic fuzzy number

In order to facilitate the calculation of the following steps, the score function will convert different types of evaluation data into a unified data form.

For , the score function iŝ = − min max − min For , the score function is used to convert the evaluation value into the number between the intervals (0,1). Hence, a complete score matrix =̂,̂∈ (0, 1) is obtained.

Step 7. Determine reference points and limiting profiles. In AHPSort method, the limiting profiles or the central profiles are allowed to be used to achieve the same effect (Ishizaka et al., 2012) . The limiting profiles represent the lowest value belonging to a risk class, and the central profiles represent the central value in a risk class. In order to show the classification more clearly, this paper chooses the limiting profiles for research. This step is the core of the whole evaluation system. Different reference points can have a great influence on drawing the judgment curve. The interval of the score matrix is (0,1), therefore, the (0,1) interval will be divided into 10 parts (0.1, 0.2, … , 1). The reference points can be determined by equal distance and there are 10 reference points. Define the limiting profiles of each risk level. This can be done with a local limiting profile , = 1, 2, … , − 1 which indicates the minimum risk to belong to a class +1 . Totality requires ⋅ ( − 1) limiting profiles to define , which can be seen in Fig. 3 .

Step 8. Select reference points for each criterion. According to the actual situation, the reference point denoted by , = 1, 2, … , will average the score of the criteria from the minimum to the maximum as some equal share. Some of the reference points will coincide with the limit profiles.

Then, the pairwise comparison between the reference point and the limiting profile should be calculated, which is completed by MATLAB, a matrix calculation software (Taha & Rostam, 2012) . If the final score of the Internet platform belongs to the interval of two consecutive reference points and +1 , the local priority is calculated by:

where:

• and +1 are two consecutive reference points on criterion ;

• and +1 are the local priorities of the two consecutive reference points;

• is the score on criterion ; • is the local priority.

For negative criteria, =̂, for positive criteria, = 1 −̂. Negative criteria: criteria that have a negative effect on public opinion. The higher the value of such criteria, the higher the risk of public opinion. Positive criteria: criteria that play a positive role in public opinion. The higher the value of such criteria, the lower the risk of public opinion.

Step 9. Derive the global priority, after obtaining the local priority for every criterion and the weight of each criterion, then multiply them and the global priority of public opinion risk of Internet platform can be easily attained.

Step 10. Derive the risk level of public opinion, if limiting profiles have been defined, the public opinion risk of Internet platform is assigned to the class which has an just below the global priority .

In this section, AHPSort II-SW method to analyze the risk of public health emergency Internet public opinion is introduced (see Fig. 4 for details). The classification is completed in nine steps. Finally, the result of the Internet public opinion risk of this platform is obtained. The analysis step will be enabled again according to information update and event development.

The occurrence of public health emergencies every time will cause people's hot discussion and cause great public opinion on the Internet. At the same time, some rumors will be generated on the Internet. Moreover, these rumors are usually spread on some mainstream social platforms. Therefore, we propose a case of public opinion risk rating for Microblog platform. In this section, we first give a case to illustrate the performance of AHPSort II-SW, which is proposed to solve the public opinion classification problem of Microblog platform. Then the proposed method is further discussed.

According to the data of the 40th statistical report on the development of China's Internet, as of June 2017, the number of Internet users in China has reached 751 million. The number of mobile Internet users reached 724 million, and the mobile Internet has penetrated into all aspects of people's lives. Microblog is the leading platform for people to exchange opinions and ideas in their daily life. As of September 2017, there were 376 million active users in the Microblog. On December 26, 2019, doctors found the first patient with unexplained pneumonia. This matter has already aroused people's attention on.

At 10:00 a.m. on January 23, 2020, the Wuhan government announced the ''closure of Wuhan''. This news has caused a great stir on the Internet, and a large number of Internet users and media exchanging and discussing on the Internet. With the joint efforts of the Chinese government and many other parties, Wuhan officially lifted its state of closure on April 8. Therefore, this paper will select the relevant data of Microblog platform from January 23 to April 8 for analysis. In order to demonstrate the continuous assessment method proposed in this paper, we will conduct about one assessment every two weeks from January 23 to April 8 (i.e., January 23 (1/23), February 6 (2/6), February 20 (2/20), March 5 (3/5), March 19 (3/19), April 2 (4/2), April 8 (4/8)). January 23, 2020 is the time for Wuhan to announce the closure of the city. Therefore, January 23 will be the time for the first risk assessment. After that, keep the frequency of public opinion risk assessment every two weeks until April 2 (Six times). April 8, 2020 will be the time for the seventh risk assessment, when Wuhan officially announced the lifting of the closure. The top right of Fig. 5 is Chinese most influential media, ''People's Daily'', which issued the proportion of Microblog, WeChat and TikTok in January 23 (Ning & Lu, 2020) . The lower right corner of Fig. 5 shows the average number of likes, comments and forwards of the top 100 microblogs in the first three hot searches in the seven days. Therefore, Microblog is the main platform for people to express their opinions in response to public health emergencies. Next, the Internet public opinion risk analysis case under Microblog platform in COVID-19 using this method on January 23 will be shown.

Step 1. Four experts in different fields are asked to establish an Internet public opinion risk evaluation criterion system of Microblog platform. After consulting the relevant literature, the four experts will discuss on ''Tencent Meeting'' platform, the most widely used online meeting platform in China, to get the final results. Online meeting can effectively reduce the possibility of virus transmission and get rid of the constraints of participants' geographical location. Through experts brainstorming and consulting relevant literature, the criterion system of Microblog platform is determined. It can be seen in Table 2 . (Ning & Lu, 2020) Real Negative

Number of likes ( The data under the ''Spread'' criterion are all completed by web crawlers, and the rule is to crawl and analyze the microblogs in the ''hot columns'' under the top three hot search topics (related to emergencies) of microblogs on that day. The number of microblogs refers to the average number of microblogs on one topic, while the number of likes, comments and forwarding refer to the corresponding value of each microblog on each topic.

The number of microblogs is calculated as 1 3 ∑ 3 =1 , = 1, 2, 3, where means the number of microblogs in the top three hot topics. The number of microblogs is calculated as

where means the number of like in the th microblog in the th hot topic. The number of comments and the number of forwards are calculated in the same way as the number of likes. Fig. 6 shows a total of 21 hot topics, the number of likes, comments and forwarding of each topic. Balls of the same color indicate that these topics belong to the same day.

Step 2. Determine the evaluation of experts. There are four experts participating in the evaluation = ( 1 , 2 , 3 , 4 ) , the identities of these experts are: a microblog platform operation administrator ( 1 ), a local government staff ( 2 ), a researcher in the Table 4 Intuitionistic fuzzy evaluation matrix. field of decision-making ( 3 ) and a scholar in the field of public opinion ( 4 ). In order to comply with the government's epidemic prevention policy and avoid the risks brought by COVID-19, the form of ''online meeting'' was adopted to provide communication and discussion for experts. The evaluation result matrix is shown in Table 3 :

Step 3. According to Table 1 , the linguistic fuzzy number in the evaluation matrix of four experts are transformed into intuitionistic fuzzy number to form a new intuitionistic fuzzy evaluation matrix. The transformed matrix is shown in Table 4 .

Using Eq.

(3), according to the hesitant set of experts , the trust function is constructed by using the principle of information entropy ( ), finally, the expert weights are obtained as 0.23, 0.26, 0.25 and 0.26, respectively.

Step 4. To obtain the weights of criteria, experts believe that the first important criterion to improve the worst-case situation to the best is ''The number of microblogs'', so the criterion is given a preliminary weight of 100. The second important criterion is ''The number of forwards'', which is given a preliminary weight of 94. By analogy, the initial weights of all criteria will be obtained, and the weights of criteria can be obtained after the initial weights are standardized. The final weighting results are shown in Table 5 .

Step 5. Firstly, the evaluation values of the four experts are gathered, and then the results are transformed by the score function. Intuitionistic fuzzy number is transformed into score values by using data normalization method. Select the reference points by an equal distance. Determine the minimum and maximum value of each criterion and then divide it by ten reference points, in which Step 6. The reference points are divided into three different clusters. Take '' 1 '' as an example. The relationship between the reference points and clusters and how the results are calculated are shown in Fig. 7 . Three clusters of ''Educational level ( 1 )'' are shown in Table 6 . The results of standardization are shown in Table 7 .

Step 7. Utilize Eq. (6) to get the local priority of each criterion. The global priority is calculated by weighting with the weights of criteria.

Step 8. Derive the local priority of each criterion and then obtain the global priority. The local priority curve is shown in Fig. 7 . Step 9. According to the relationship between the global priority of limiting profiles and the global priority of alternatives, public opinion risk of Microblog are divided into different classes defined in advance (High risk, Medium risk, Low risk, No risk). According to the above steps, the risk level of Microblog in COVID-19 on January 23, 2020 is ''High risk''.

Step 10. By repeating steps 2-9, the criterion system and weights of criteria are not changed. The results and details of the seven evaluations from January 23 to April 8, 2020 are shown in Fig. 8 .

In this paper, comparative analysis and sensitivity analysis are used to explain the advantages and stability of AHPSort II-SW method. The results put forward management suggestions for the government, enterprises and users to reduce the risk of Internet public opinion, the details are as following.

According to the analysis results, the conclusion is obvious: the risk level of Microblog Internet public opinion of public health emergencies is the highest on January 23, that is, the day of ''Wuhan City closure''. Among them, the criteria that contribute to the risk of public opinion are ( 2 , 6 , 7 , 8 ), which is confirmed by the results in Fig. 6 . This proves that in the early stage of COVID-19, the discussion of COVID-19 in Microblog is very hot, and users spend a lot of time browsing information about the epidemic on the Internet. 14 days later, the risk of the second risk assessment (i.e., 2/6) is moderate, which is significantly lower than that of the first one. The reason is that the users began to understand and accept the epidemic from being fear and panic at the early stage. The popularity of related topics in Microblog is greatly reduced. However, China's epidemic situation has come to the worst, the number of new cases and deaths officially announced has gradually increased, and the real-world epidemic situation has brought potential risks to Internet public opinion. Therefore, the criterion contributing the most in the second risk level evaluation are ( 4 ). In the third public opinion risk assessment (i.e., 2/20), we also found a similar situation, but the criterion with the greatest contribution became ( 5 ).

With the vigorous struggle of the Chinese government and people, the epidemic situation in China has gradually improved. During the subsequent Internet public opinion risk assessment (i.e., 3/5, 3/19, 4/2), the number of new cases and deaths decreased significantly, which also greatly reduced the public opinion risk of the Microblog platform. The main factors affecting public opinion gradually changed from the epidemic situation in the real world and the popularity of Internet discussion to the psychological pressure of users. With the extension of the duration of the epidemic, many users are required to stay at home for a long time, resulting in great psychological pressure. Furthermore, the loss of jobs and the death of relatives and friends will increase the psychological pressure. After that, the risk level of several evaluations decreased slowly until April 8, when the Wuhan government J. Liu et al. announced ''Wuhan's lockdown was lifted''. This topic had a great impact on the Microblog and attracted the attention of many people. Many non Wuhan City users expressed their concerns on the Internet about whether the virus will spread to other cities after Wuhan's lockdown was lifted. Such statements have driven some negative voices on the Internet. The risk level of public opinion increased slightly compared with the previous one. But it is much lower than January 23.

In terms of survey modality, this paper uses ''online meeting'' to provide a way for experts to discuss and evaluate. In the context of COVID-19, this survey modality meets the needs of experts for policy provisions (i.e., the evaluation can be completed at home). Due to the flexibility of ''online meeting'' and the characteristics of being able to participate without being limited by geographical location, this model has received high praise from participating experts. Therefore, we still suggest using ''online meeting'' to evaluate public opinion risk in follow-up work.

Combining AHPSort II and SW, this paper proposes AHPSort II-SW method to analyze the risk level of Internet public opinion in COVID-19 period. In order to show the utility of using SW method to improve AHPSort II, we will use the characteristic root method (the same as AHPSort and AHPSort II) of evaluation criteria weights for comparison.

The AHPSort method will be used to analyze the case, and the decision results of the two methods will be compared. The result of criteria weights using AHPSort method is (0.014, 0.039 , 0.074, 0.057, 0.028, 0.237, 0.048, 0.115, 0.099, 0.174, 0.058, 0.035, 0.021) .

As it can be seen in Fig. 9 , the trends of AHPSort method and AHPSort II-SW method are basically the same, which it proves the effectiveness of SW method. The advantages of the SW method are summarized as follows:

(1) Methodological aspect: the number of comparisons of the SW method are fewer than that of AHPSort. SW method only has 12 times of comparison, with 13 criteria, while the eigenvalue method has 23 times. The comparison times of the SW method is only 52% of the eigenvalue method.

(2) Realistic aspect: from the evaluation process of criteria weights, the improvement of the worst assumption into the best assumption in SW method is in line with people's subjective expectation of public opinion risk. Therefore, the results obtained by the SW method are more practical than the eigenvalue method, which is in line with people's subjective psychology.

(3) Managerial aspect: the SW method can find the most important criterion to adjust the worst level to the best level, which has a certain reference value for target public opinion governance.

According to the original evaluation information, the subjective method is used to weight the evaluation criteria. When the weights change, does the public opinion risk level change? In order to illustrate the stability of the proposed method, this section analyzes the stability of scheme classification by analyzing the sensitivity of weight setting changes.

Based on the evaluation results on January 23, sensitivity analysis is carried out. In the SW method, the most important criterion should be assigned 100, and the other criteria should be assigned (9,100) -The interval is the minimum and maximum value of experts assignment in case analysis. The value of 1 will be set to 100, and use Excel to generate a random number of (9,100) for other criteria. This number will be used as the initial weight of the corresponding criteria, get the weights of criteria through calculation, and finally get the new grading result. The random number is generated 20 times. Then we assign 2 to 100, and use Excel to generate random numbers (9,100) for other criteria, as above, to get the grading results, and repeat 20 times. By analogy, 260 new grading results are obtained from 13 criteria, and the results are shown in Fig. 10 . Based on the characteristics of SW method and AHPSort II method, the global priority of alternatives is obtained by the multiply accumulate of weights of criteria and local priority. Therefore, the higher the local priority is, the greater the weight is, the higher the global priority will be. The local priority is closely related to ''criteria score'' and ''pairwise comparison of reference points''.

The results of sensitivity analysis most of them are ''High risk'', which is the same as the analysis results in this paper. It is obvious that most of the classification results are not sensitive to weights change. It can be considered that this method is effective and reliable. 

The results of the study provide some suggestions for managers and the government to deal with the risk of Internet public opinion of public health emergencies, the details are listed as following:

(1) Based on ''The report on public cognition and information dissemination of COVID-19'', the forwarding rates and attention degrees of top 10 epidemic related topics can be concluded shown in Fig. 11. From Fig. 11 , on the Internet, the most concerned topic of the public is the epidemic data (for example, the number of new cases and deaths, etc.), and its attention and forwarding rate has reached 55% and 41%. Therefore, the correct and timely update of epidemic information is a necessity. Making the information as open and transparent as possible will help to reduce people's panic. In addition, ''verification of true and false information'' is also one of the topics that people are most concerned about, the mainstream media also need to undertake the task of rumor refutation. In order to prevent the spread of rumors and reduce the harm of rumors, the false rumors spread on the Internet should be announced in time.

(2) In the early period of COVID-19, the situation in China was not optimistic. Fig. 12 shows the daily epidemic data officially reported by the Chinese government from November 23 to April 8, 2020, which is from the ''website of the National Health J. Liu et al. Commission of the people's Republic of China 2 ''. From the perspective of Internet public opinion, the number of reported ''Diagnosis cases'' and ''Death cases'' will directly affect the risk of public opinion. Therefore, when the epidemic situation is severe, the government should improve the intensity of public opinion control and increase management measures to ensure the normal operation of the Internet.

The operator and manager of the Internet platform is the primary force of Internet public opinion governance. Its attitude towards public opinion directly affects the development and change of Internet public opinion. Therefore, the following suggestions are put forward to the operators and managers of the platform:

(1) Enhance the sensitivity of the platform or software to public opinion, and improve the prevention and resistance ability of the platform to harmful public opinion. In the evaluation results on January 23, the criterion ''spread'' (i.e., 6 , 7 , 8 , 9 ) contributed more than 60% to the risk of public opinion. This phenomenon is mainly due to the sudden emergence of a lot of microblogs about the discussion of COVID-19, accompanied by a large number of likes, comments and forwards. In emergencies, social platforms should be well prepared for the discussion that may suddenly increase. Internet platform should quickly identify and accurately locate rumors and Internet public opinion, and find out the root cause of public opinion. Timely and effective measures should be taken to control public opinion, such as issuing rumor refuting tips to users through technical means, and ordering rumor initiators to publicly apologize on the platform and explain the truth, etc.

(2) As can be seen from Fig. 11 , many users are very concerned about the topic of ''touching story of anti-epidemic'' on the Internet (ranking 5-7). Social platforms can intentionally push to users some moving things about doctors, patients or volunteers in the process of anti epidemic, which can help people improve their belief in fighting the epidemic (Yao et al., 2021) , and also promote the positive emotions of the Internet.

(3) Promote users' self-monitoring and improve their ability to accept users' reports. Any Internet platform has a large number of users, which may cause difficulties in the management of Internet public opinion. Promoting users' self-monitoring and improving the ability to accept users' reports will help managers control many users. Let Internet users become a part of Internet public opinion governance. With their participation, the power of Internet public opinion governance will be even greater.

As an important part of the Internet, Internet users are also the main body of Internet public opinion, which plays a decisive role in the emergence and development of public opinion. Therefore, it is necessary to provide suggestions to Internet users, as follows:

(1) In the fourth (i.e., 3/5) and fifth (i.e., 3/19) public opinion risk assessment, criterion ''Psychological pressure'' (i.e., 12 ) provides the largest contribution rates (i.e., 30%, 32%). Therefore, it is necessary for users to learn to adjust their own pressure and learn to reduce pressure appropriately. When personal pressure is too high to handle, it is a good choice to turn to a professional psychologist.

(2) In all the evaluation results, the criterion ''Frequently'' (i.e., 2 ) has always maintained a large contribution rate. Therefore, users need to properly control the online time. The Internet is full of a lot of false information. Browsing for a long time may be deceived by these false information. Similarly, browsing negative news for a long time will also increase their psychological pressure. In the face of public health emergencies, the masses should face them calmly and actively cooperate with the relevant actions of government staff to avoid spreading false statements on the Internet and harming the Internet Environment.

(3) Improve the judgment awareness of Internet information, learn to identify the information on the Internet, do not blindly believe other people's opinions, use scientific and reasonable methods to identify the true and false, do not accept the false information, and timely inform the people around by Microblog, circle of friends, Wechat group and other social means to stop the spread of rumors.

Internet has become an inseparable part of people's lives. Every day, millions of people participate in discussions about COVID-19 on Twitter, Microblog or other social media. Compared with general emergencies, COVID-19, as a public health emergency, has the characteristics of long duration. Therefore, it is particularly important to build a long-term and sustainable Internet public opinion risk evaluation model. The main contributions of this study are as follows: (1) This paper emphasizes the particularity of public health emergencies compared with general emergencies, and points out that the Internet public opinion risk of public health emergencies is generally greater than that of general emergencies. On this basis, this paper analyzes the generation mechanism of Internet public opinion on public health emergencies. (2) A multi-stage risk classification model of Internet public opinion is established, which can effectively focus on and monitor the risk levels of Internet public opinion for public health emergencies with long time extension.

(3) AHPSort II-SW method is proposed to grade the risk levels of Internet public opinion in public health emergencies, and applied to the public opinion risk rating of Microblog platform. (4) Seven risk assessments are conducted from January 23 to April 8, among them, the first Microblog public opinion risk evaluation result is High risk, and those of the other six assessments are Medium risk. In general, the degree of risk shows a slow downward trend over time, which verifies the effectiveness of the method. (5) On the basis of case analysis, this paper puts forward some practical suggestions for the three main bodies (government, enterprises, users) participating in Internet public opinion, in order to reduce the risk and harm of Internet public opinion in public health emergencies.

In this study, a multi-stage Internet public opinion risk grading model is established by combining AHPSort II and SW methods, with the help of intuitionistic fuzzy number and linguistic fuzzy number, which provides a solution for how to know the public opinion risk of a specific social platform in public health emergencies.

In theory, MCDM is applied to the risk assessment of Internet public opinion, which enriches the application field of MCDM research. Then, the improved method of AHPSort II (i.e., AHPSort II-SW method) is put forward. When dealing with the problem of criteria weights, the AHPSort II-SW method is more in line with the characteristics of Internet public opinion, reduces the number of comparisons, and reduces the decision-making pressure for experts. Moreover, AHPSort II-SW method is applicable to other fields of MCDM. In a word, this method provides more choices for researchers in the field of MCDM. There are many researches on the risk of Internet public opinion in general emergencies. However, there are few studies that propose a multi-stage risk grading analysis model based on the long duration characteristics of public health emergencies. Therefore, our research has an important theoretical contribution to identify the difference of public opinion risk between general emergencies and public health emergencies, and continuously evaluate the public opinion risk of social media in the long term in public health emergencies. It also provides a theoretical reference for the government and enterprises on how to timely understand the degree of public opinion risk and quickly formulate management measures. Moreover, concerning the impact of public emotion on the risk of Internet public opinion provides theoretical support for the authorities to prevent the formation of public opinion and reduce the negative risk of public opinion. Finally, our case study analysis also provides certain implications for the prevention and control of Internet public opinion risk of social media in other countries.

In practice, our research provides important implications for the government on how to maintain the order of the Internet in the context of public health emergencies, ensure the normal operation of the Internet space and eliminate the negative public opinion that endangers social stability. For the government, our research results also provide implications on how to prevent the formation of public opinion and deal with the harm of public opinion in public health emergencies. Specifically, first full disclosure and timely update of epidemic information and data should be guaranteed. To let the public know the latest information related to the epidemic in time is not only an important means to reduce the spread of the virus in the medical perspective, but also the main method to prevent Internet public opinion. In public health emergencies, most of the public access to health information through social media platform (Dias da Silva & Walmsley, 2019). The latest information may directly affect their life and work arrangements. For example, workers need to know the latest epidemic prevention and control policies of the destination (whether nucleic acid testing reports or quarantine measures are required) to make their own travel plans. Additionally, the spread of rumors should also be concerned. Potential harms of rumors can cause economic effects directly or indirectly. For example, in April 2020, the rumor that ''the edible salt produced in Hubei Province has COVID-19'' spread widely on the Internet, which has had an economic impact on the local salt production enterprises. Setting up a special rumor refuting website and reminding the public to ''not believe in rumors, not spread rumors'' can effectively alleviate the spread of rumor and reduce the harm. Finally, it will be helpful to reduce the risk of public opinion by restraining the speeches of people who tend to make negative comments on the Internet, especially those who have a large number of fans. The government should take advantage of these people who receive a lot of attention on the Internet to guide their fans to regard the epidemic rationally and cooperate with the relevant departments in epidemic prevention and control work.

For the future research, we aim to expand the research object and consider the Internet public opinion risk of more social media platforms. Then, we will consider continuing to improve the AHPSort II method, such as reference point, limiting profile, fuzzy number. Moreover, we will continue to expand the application fields of the AHPSort II method. 

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Analysis of the impact of education rate on the rumor spreading mechanism

Social sustainability of treatment technologies for bioenergy generation from the municipal solid waste using best worst method

Fake news and COVID-19: modelling the predictors of fake news sharing among social media users

Evaluating the feasibility of blockchain in logistics operations: A decision framework

Group analytic hierarchy process sorting II method: An application to evaluate the economic value of a wine region landscape

Knowledge, attitudes, and practices towards COVID-19 among ecuadorians during the outbreak: An online cross-sectional survey

The Australian drug harms ranking study

Public preferences for lung cancer screening policies

Automatic rumor detection on microblogs: A survey

The bright and dark sides of social media usage during the COVID-19 pandemic: Survey evidence from Japan

Fake news and dental education

Integrated linguistic entropy weight method and multi-objective programming model for supplier selection and order allocation in a circular economy: A case study

Sources of information in times of health crisis: evidence from Turkey during COVID-19

Whether the weather will help us weather the COVID-19 pandemic: Using machine learning to measure twitter users' perceptions

Biofuel logistics network scheme design with combined data envelopment analysis approach

The interaction evolution model of mass incidents with delay in a social network

AHPSort: An AHP-based method for sorting problems

Multi criteria decision making in selecting stormwater management green infrastructure for industrial areas Part 1: Stakeholder preference elicitation

An AHPSort II based analysis of the inequality reduction within European union

Construction of index system and fuzzy evaluation model for network public opinion

Temporal and spatial evolution of online public sentiment on emergencies

Effect of anger, anxiety, and sadness on the propagation scale of social media posts after natural disasters

Social media rumor refutation effectiveness: Evaluation, modelling and enhancement. Information Processing and Management

A multi-criteria decision making method for urban flood resilience evaluation with hybrid uncertainties

Information diffusion and opinion leader mathematical modeling based on microblog

GAHPSort: A new group multi-criteria decision method for sorting a large number of the cloud-based ERP solutions

Using a hybrid multiple-criteria decision-making technique to identify key factors influencing microblog users' diffusion behaviors in emergencies: Evidence from generations Born after

The relationships between health anxiety, online health information seeking, and cyberchondria: Systematic review and meta-analysis

Internet public opinion risk grading under emergency event based on

Sorting municipalities in umbria according to the risk of wolf attacks with AHPSort II

Transition towards circular supplier selection in petrochemical industry: A hybrid approach to achieve sustainable development goals

Covid-19 communication management in Spain: Exploring the effect of information-seeking behavior and message reception in public's evaluation

Comparison of weighting methods used in multicriteria decision analysis frameworks in healthcare with focus on low-and middle-income countries

An empirical study on novel coronavirus pneumonia report in the people's daily new media matrix

Developing a socio-computational approach to examine toxicity propagation and regulation in COVID-19 discourse on YouTube

A new FlowSort-based method to deal with information imperfections in sorting decision-making problems

Supply chain risk network management: A Bayesian belief network and expected utility based approach for managing supply chain risks

Loneliness, isolation, and social support factors in post-COVID-19 mental health

Ten tips for running an online conference

Perceptions of public on the COVID-19 outbreak in Singapore: A qualitative content analysis

Dealing with uncertainties in MCDA

Voice, power and connectivity in China's microblogosphere: Digital divides on SinaWeibo

A hybrid fuzzy AHP-PROMETHEE decision support system for machine tool selection in flexible manufacturing cell

MCDA swing weighting and discrete choice experiments for elicitation of patient benefit-risk preferences: a critical assessment

Investigating the social, economic and political consequences of the COVID-19 pandemic: A rolling cross-section approach

The spread of true and false news online

Studies on a multidimensional public opinion network model and its topic detection algorithm

Distance measure between intuitionistic fuzzy sets

Characterizing the patterns of China's policies against COVID-19: A bibliometric study

Early-warning analysis of crowd stampede in metro station commercial area based on internet of things

How privacy concerns and cultural differences affect public opinion during the COVID-19 pandemic: a case study

Sustainable supplier selection based on AHPSort II in interval type-2 fuzzy environment

Detecting fake news by exploring the consistency of multimodal data

Influence of online social support on the public's belief in overcoming COVID-19

A new geo-propagation model of event evolution chain based on public opinion and epidemic coupling

A GRA-based intuitionistic fuzzy multi-criteria group decision making method for personnel selection

Content or context: Which matters more in information processing on microblogging sites

Stakeholder interviews with two MAVT preference elicitation philosophies in a swiss water infrastructure decision: Aggregation using SWING-weighting and disaggregation using UTAGMS

Internet public opinion evolution in the COVID-19 event and coping strategies