key: cord-0270645-ka3nk4fj authors: Zhou, Xinyi; Shu, Kai; Phoha, Vir V.; Liu, Huan; Zafarani, Reza title: "This is Fake! Shared it by Mistake": Assessing the Intent of Fake News Spreaders date: 2022-02-09 journal: nan DOI: 10.1145/3485447.3512264 sha: 1ddcfffea73bdd94d98ddebf6191ec65fb907620 doc_id: 270645 cord_uid: ka3nk4fj Individuals can be misled by fake news and spread it unintentionally without knowing it is false. This phenomenon has been frequently observed but has not been investigated. Our aim in this work is to assess the intent of fake news spreaders. To distinguish between intentional versus unintentional spreading, we study the psychological explanations of unintentional spreading. With this foundation, we then propose an influence graph, using which we assess the intent of fake news spreaders. Our extensive experiments show that the assessed intent can help significantly differentiate between intentional and unintentional fake news spreaders. Furthermore, the estimated intent can significantly improve the current techniques that detect fake news. To our best knowledge, this is the first work to model individuals' intent in fake news spreading. A frequently observed and discussed phenomenon is that individuals can be misled by fake news and can unintentionally spread Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. WWW '22, April 25-29, 2022 it [24, 30, 47] . Thankfully, research has pointed out that (1) correction and (2) nudging can effectively prevent such users from spreading fake news. That is, by informing them of the news falsehood, or simply requesting from them to pay attention to news accuracy before spreading the news [24, 36] . Such findings encourage social media platforms to develop more gentle strategies for these unintentional fake news spreaders to reasonably and effectively combat fake news. Clearly, such strategies should vary from the aggressive deactivation and suspension strategies that platforms adopt for inauthentic or toxic accounts (e.g., Twitter 1 and Facebook 2 ). For example, platforms can present such unintentional fake news spreaders with useful facts, motivating the need for new recommendation algorithms. Such algorithms not only recommend topics to these users that they enjoy reading the most (or users they are similar to), but also facts or users active in fact-checking (see Fig. 1 for an example) [18, 36, 47] . To determine (1) if correction or nudging is needed for a fake news spreader, (2) whether the spreader should be suspended or deactivated, or (3) which users should be targeted by fact-presenting recommendation algorithms, one needs to assess the intent of fake news spreaders. Furthermore, knowing that some users had malicious intent in the past provides a strong signal indicating that their future posts are also potentially fake. This information can be immensely useful for fake news detection [47] . While determining the intent is extremely important, it is yet to be investigated. This work: Assessing Spreading Intent. We aim to assess the intent of individuals spreading fake news. Our approach to assessing the intent of fake news spreaders relies on fundamental social science theories and exploits advanced machine learning techniques. In particular, we first look into psychological factors that can contribute to the unintentional spreading of fake news (see Section 2.1). These factors can be categorized as internal influence and external influence [47] . To capture these factors, and in turn, quantify intent, we propose an influence graph; a directed, weighted, and attributed graph. The degree to which fake news spreaders are intentional/unintentional can be assessed with this graph. To evaluate our assessment, we first extend two fake news datasets by introducing annotated intent data of fake news spreaders (intentional or unintentional) due to the unavailability of ground truth. With this data, we validate the assessed intent and show that it can strongly differentiate between intentional and unintentional fake news spreaders. We further show through experiments that the assessed intent can significantly enhance fake news detection. The innovation and contribution of this work are: (1) Modeling Fake News Spreading Intent: To our best knowledge, this is the first work to assess the degree to which fake news spreaders are intentional/unintentional. To this end, we conduct an interdisciplinary study that endows our work with a theoretical foundation and explainability. A new influence graph is proposed that captures factors that contribute to spreading intent as well as multimodal news information. (2) New Datasets on Intent: We leverage manual and automatic annotation mechanisms to introduce the ground truth on the intent of fake news spreaders in two large-scale realworld datasets. These are the first two datasets that provide intent information. We conduct extensive experiments using these datasets to validate the assessed intent of fake news spreaders. (3) Combating Fake News: Our work helps combat fake news from two perspectives. First, we demonstrate that by assessing intent, we can successfully distinguish between malicious fake news spreaders (should be blocked) and benign ones (should be presented with facts or nudged). Second, we present the effectiveness of the assessed spreader intent (and the proposed influence graph) in fake news detection. The rest of the paper is organized as follows. A literature review is first conducted in Section 2. In Section 3, we specify the method to assess the intent of fake news spreaders, followed by the method evaluation in Section 4. We demonstrate the value of assessing intent in combating fake news in Section 5. Finally, we conclude in Section 6 with a discussion on our future work. We first review fundamental social science theories that have been connected to fake news spreading (see Section 2.1). Next, we review the methods developed to combat fake news (see Section 2.2) as we will later utilize the assessed spreader intent to detect fake news. Extensive social science research has been conducted on fake news. We particularly review studies that focus on the psychological factors that contribute to the unintentional spreading of fake news. Lazer et al. [15] attribute this phenomenon to "individuals prefer information that confirms their preexisting attitudes (selective exposure), view information consistent with their preexisting beliefs as more persuasive than dissonant information (confirmation bias), and are inclined to accept information that pleases them (desirability bias). " Scheufele and Krause [30] summarize these factors as confirmation bias, selective exposure, and motivated reasoning (i.e., people tend to use emotionally biased reasoning to make most desired decisions rather than those that accurately reflect the evidence). Grouping aforementioned psychological factors as an internal influence, Zhou and Zafarani [47] further discuss how the external influence on individuals can contribute to their unintentional spreading of fake news. Such social influence can be reflected via, e.g., availability cascade (i.e., individuals tend to adopt insights expressed by others when such insights are gaining more popularity) [14] , social identity theory [3, 13] (i.e., individuals conform to the behavior of others for being liked and accepted by the community and society), and validity effect (e.g., individuals tend to believe information is correct after repeated exposures) [6, 23] . This work shares the social science foundation presented in [15, 30, 47] . Besides understanding why individuals can be misled by fake news and unintentionally spread it, we further conduct quantitative research to assess user intent. The unprecedented growth of fake news and its detrimental impacts on democracies, economies, and public health has increased the demand for automatic methodologies to combat fake news [47] . With extensive recent contributions by the research community, automatic fake news detection has significantly improved in efficiency and explainability. In general, fake news detection methods can be content-based or propagation-based depending on whether the method focuses on investigating news content or how the news spreads on social media. As news articles are mostly text, content-based methods start with manually extracting linguistic features for news representation; LIWC (Linguistic Inquiry and Word Count) [22] has been often employed as a comprehensive feature extractor [7, 25, 27] . Common classifiers, such as SVMs (Support Vector Machines), are then used to predict fake news. With advances in deep learning, recent attention has been paid to employing multimodal (textual and visual) information of news content to detect fake news (see related work such as [1, 26, 39, 43, 46] ). On the other hand, propagation-based methods utilize auxiliary social-media information to predict fake news. Some examples of such information include post stances [33] , post-repost relationships [37] , user comments [32] , and profiles [9] . There have been other strategies proposed to combat fake news. For example, education and nudging have been emphasized to improve individuals' ability to recognize misinformation [15, 17, 30] . Pennycook et al. further provide empirical evidence that unintentional fake news spreading can become less by asking individuals to assess the accuracy of news before attempting to spread it [24] . Lazer et al. suggest incorporating information quality into algorithmic rankings or recommendations of online platforms [15] . Studies have also demonstrated that connecting users active in fact checking with fake news spreaders on social networks is an effective way to combat fake news [18, 36] . As presented in Section 2.1, psychological factors that contribute to unintentional fake news spreading of individuals can be summarized as two: (1) internal influence and (2) external influence [15, 30, 47] . Hence, an individual is more unintentional in spreading a news article if his or her spreading behavior receives more internal and external influence. Specifically, both confirmation bias [20, 21] and selective exposure [12, 19] point out that the more consistent an individual's preexisting attitudes and beliefs are with the fake news, the higher the probability that the individual would believe the fake news and unintentionally spread it (internal influence) [15, 30, 47] . As availability cascade [14] and social identity theory [3, 13] suggest, individuals can be affected by others as well. An individual would be more unintentional in spreading a fake news article if the spreading follows a herd behavior; i.e., the individual's participation matches extensive participation of others and his or her attitude conforms to the attitudes of most participants (external influence) [47] . Problems then arise on social media: where can one find out the preexisting attitudes and beliefs of a user, the participation of users, and their attitudes towards a news article? We note that a user's preexisting attitudes and beliefs can be reflected in his or her historical activities on social media. For most social media sites, such historical activities include past posts, likes, and comments. Similarly, the participation of users often takes the form of posting, liking, and commenting. Hence, mining the content of posts and comments allows understanding users' attitudes. For simplicity, we start with posts in this work to determine users' preexisting beliefs and participation. In sum, a user spreads a fake news article in his or her post more unintentionally if the post is more similar to or influenced by (1) the user's past posts (internal influence), and (2) the posts of other users (external influence). A natural approach to capture the influence among posts is to construct an influence graph of posts. In this graph, a (directed) edge between two posts indicates the (external or internal) influence flow from one post to the other. The edge weight implies the amount of the influence flow. With this graph, the overall influence that a post receives from other posts can be assessed by looking at its corresponding incoming edges and their weights. The more influence a post that contains fake news receives, the more unintentional is the user who is posting it in spreading this fake news. To concretely present our proposed influence graph formed by a group of posts, we start by a pair of posts and , which are represented as tuples ( , , , ) and ( , , , ), respectively. An example is presented in Fig. 2 . In the tuple representing post , denotes the news article shared by post . For simplicity, we first assume that each post can only share one news article (we will consider a more general case later in this section); user and time refer to the user and posting time of ; and content is the post p i a i = a j , c i = c j , t i + ϵ = t j : content, often containing the attitude and opinion of regarding . Next, we discuss how (A) internal and (B) external influence between and can be modeled, respectively. A. Modeling internal influence between and . If internally influences , should be posted earlier than and by the same user of (to capture preexisting beliefs of the user), i.e., < and = . The amount of influence flowing from to can be determined by how similar the news articles and attitudes in are to those of . In other words, how similar and are to and [47] . However, evidence has indicated that the same user spreading the same news, especially fake news, is often a sign of intentional spreading rather than unintentional spreading [31] . Therefore, we exclude internal influence from and if = . B. Modeling external influence between and . If externally influences , should, at least, be posted by a different user from that of (to capture "external") and earlier than (otherwise, is not observable to ); i.e., < and ≠ . We further consider two questions in assessing external influence. First, can a user's post spreading one news article externally influence a post of another user spreading a different news article; in other words, if ≠ , can possibly influences externally with < and ≠ ? Two news articles that differ in text or image may discuss the same event and express the same political stance; hence, this scenario is possible but depends on the similarity between the two news articles [5] . Second, can a user's post possibly be influenced by the other's post if the two users are not socially connected on social media? Due to the platforms' diverse recommendations and services (e.g., the trending in Twitter and Weibo), this scenario is also possible, but the amount of influence depends on how similar news articles and attitudes in are to those of [44, 47] . We summarize the above discussions by answering the following three questions: (1) Edge existence: Can possibly influence ? As discussed, it is barely possible for to (internally or externally) affect , if it is posted later than . Hence, in an influence graph, a directed edge can possibly exist from to , if is posted earlier than (i.e., < ); if ≥ , no edge exists from to . Therefore, there can be either no edge or only one directed edge between two posts. See Fig. 3(a) for an illustration. Note that whether an edge ultimately exists between two posts also depends on the edge weight (we specify below in 3); a zero weight can make an edge "disappear. " (2) Edge attribute: What type of influence (internal vs external) is flowing between and ? We define the influence as external, if and are posted by different users, i.e., ≠ [44] . The influence is internal, if and are posted by the same user and do not share the same news, i.e., = and ≠ [31] . See Fig. 3(b) for an illustration. (3) Edge weight: How much influence flows from to ? We assume that the amount of influence flow is affected by three factors. The first, as discussed, is the news articles shared by and ( versus ); basically, if and spread the same news, the influence flow between them should be greater compared to if they spread completely different news articles [5, 47] . The second, as discussed, is the attitudes held by and on the news ( versus ); basically, if two posts agree with each other, the influence flow between them should be greater compared to if they disagree with each other [47] . Furthermore, we consider the time interval between and ( versus ); instead of "remembering all", users forget past news articles and their corresponding posts over time (with some decay) [40] . Thus, a greater amount of influence flow is assigned to two posts when one is published close in time to the other, compared to those that are published farther apart. See Fig. 3 (c) for an illustration. Next, we formalize the proposed influence graph (see Definition 3.1), and introduce how the intent of (fake) news spreaders can be quantified based on this graph. Clearly, in a real-world scenario, it is possible for a post to contain more than one news article (e.g., multiple URLs). Hence, in this formalization, we no longer assume that each post can only share one news article and generalize to a set of articles, i.e., ( , , , ) becomes ( , , , ). Definition 3.1 (Influence Graph). Given a set of news articles, denoted as = { 1 , 2 , · · · , }, we denote user posts that share these news articles on social media as = { 1 , 2 , · · · , }. Each post ( = 1, 2, · · · , ) is represented as a tuple ( , , , ), where , , , and respectively refer to a set of news articles (can be one article) shared by the post (i.e., ⊆ ), the post content, the posting time, and the user. Influence graph, denoted as = ( , , W), is formed by user posts, i.e., = . Edges exist from to if (i) is posted earlier than , and (ii) and do not share the same news when posted by the same user. In other words, where S( * , * ) assesses the similarity between * and * , T (Δ ) for Δ = − is a self-defined monotonically decreasing decay function to capture users' forgetting, andS( , ) computes the average pairwise similarity among news pairs ( , ) ∈ × . Formally,S Based on the above graph, the overall influence on each post, which we denote as the affected degree, is computed as where the external and internal influence, respectively, refer to For posts sharing fake news articles, greater values of f External , f Internal , and f indicate that user receives more external, internal, and combined (external+internal) influence when spreading the fake news article, i.e., the user engages more unintentionally. Conversely, smaller values of f External , f Internal , and f indicate that the user is affected less and engages more intentionally in fake news spreading. 3 Customized Implementation Details. The implementation of influence graph has several customizable parts; it can be modified by defining different T , developing different techniques to represent news articles and user posts, and designing ways to compute their similarities. Below are our implementations and justifications. To represent news articles and posts, we investigate both textual and visual information within the content. Textual features are extracted using Transformers, which have excellently performed in understanding semantics of text and various NLP (Natural Language Processing) tasks such as machine translation and sentiment analysis [35, 41] . As user posts are often short and within 512 words (e.g., on Twitter, the number of words are not allowed to exceed 280), 4 we use a pre-trained Sentence-RoBERTa model, which modifies RoBERTa by the Siamese network, to obtain the post embedding [28] ; the model performs best in the task of semantic textual similarity. 5 Differently, as news articles are often long and over 512 words, 6 we employ Longformer [4] to derive the semantically meaningful text embedding of news articles. Longformer addresses the limitation of 512 tokens in BERT by reducing the quadratic scaling (with the input sequence) to linear [4] . For visual features, we extract them using a pre-trained DeepRanking model particularly designed for the task of fine-grained image similarity computation [38] . With textual features of news (or post) denoted as t, and its visual features denoted as v, we define the similarity between a news (or post) pair as where * = (for news) or (for posts);ĉ os(., .) = [1 − cos(., .)]/2; and ,ĉ os(., .), S(., .) ∈ [0, 1]. In our experiments, we determine the value of by varying it from 0.1 to 0.9 with a step size 0.1; we set = 0.8 that leads to the best evaluation and prediction results. As for decay function T , we define it as which is inspired by [40] . Δ = − and indicates the chronological ranking of post (i.e., ∈ Z + ); hence, T (.) ∈ (0, 1] due to > . The benefit of such T is two fold. First, it helps normalize the affected degree for any influence graph. Specifically, let f * denote either of f , f Internal , or f External . Letf * denote the normalized version of f * , i.e.,f * ∈ [0, 1] (accurately, heref * ∈ [0, 1)). Then, for f * we have In other words, the upper bound of the affected degree, denoted by max , is ( − 1) −1 . Strictly speaking, posts ( > 1) can be posted at the same time in a real-world scenario, i.e., their ranking, denoted by , is the same. We point out that the upper bound max still holds in this case, if the ranking value after is + rather than + 1. Finally, the normalized affected degreef * for post iŝ Secondly, in the worst case, influence graph can be a tournament, taking up much space. Such T facilitates graph sparsification, while maintaining the performance on tasks (see details in Appendix A). Lastly, we note that we have tested Δ (the time interval) with various units (seconds/minutes/hours/days) in addition to chronological rankings; still, the ranking performs best in all experiments. In this section, we evaluate the proposed method in assessing the intent of fake news spreaders. To this end, evaluation data is required that contains the ground-truth label on • News credibility, i.e., whether a news article is fake news or true news; and • Spreader intent, i.e., whether a user spreads a fake news article intentionally or unintentionally on social media. We point out that this work is the first to model individuals' intent in fake news propagation. Therefore, no data exists that contains the ground-truth label on spreader intent, let alone both news credibility and spreader intent. Next, we first detail how this problem is addressed in Section 4.1, followed by the method evaluation results in Section 4.2. Our experiments to evaluate the proposed method are based on two datasets developed for news credibility research: MM-COVID [16] and ReCOVery [45] . Generally speaking, both datasets collect news information verified by domain experts (labeled as true or fake) and how the news spreads on Twitter. The corresponding data statistics are in Tab. 4(a) in Appendix B; we focus on the news with social context information, and on the English news and tweets to which all pre-trained models can be applied. Although the ground-truth label on news credibility is available, both datasets do not provide annotations on intent of fake news spreaders. We first consider manual annotation to address this problem. Specifically, we invite one expert knowledgeable in misinformation area and one graduate student generally aware of the area. We randomly sample 300 posts (unique in tweet ID and user ID) from MM-COVID and ReCOVery that contain fake news (i.e., users of these posts are all fake news spreaders). Before annotating, we first inform the annotators with the definition and general characteristics of unintentional fake news spreaders. That is, as presented in Section 1: these spreaders are misled by fake news, barely recognize it is fake, tend to believe in the fake news; meanwhile, if informed on news falsehood or presented with facts, such spreading behavior of them can be reduced, or even stopped. In annotating, we present the two annotators with • The tweet's link that spreads fake news, which allows annotators to access the tweet details (as illustrated in Fig. 2 ). • The user's link who posts the tweet, which allows annotators to access the user's profile and historical activities. For each post, we ask the two annotators to (1) Annotate if the user spreads the fake news unintentionally (with an answer of yes or no); (2) Present the confidence level (detailed below); (3) Explain the annotation with evidence; and (4) Provide an estimate on the time spent on annotation. We provide three optional levels of confidence. 0 indicates the annotation result is a random guess; no evidence is found to help annotation, or half the evidence supports but the other half rejects the annotation result. 0.5 indicates a medium-level confidence; among all the evidence that the annotator finds, some of them reject but most of them support the annotation result. 1 indicates a high-level confidence; all the evidence that the annotator finds support the annotation result. With the returned annotations, we compute the agreement of the two annotators by Cohen's coefficient [10] . = 0.61, removing annotations with no confidence; in other words, two annotators substantially agree with each other [10] . To further obtain the ground truth, we only consider the annotations with a confidence score ≥ 0.5 and agreed by the two annotators. Finally, 119 posts sharing fake news have the ground-truth label on their users' intent, among which 59 are unintentional and 60 are intentional. We point out that annotating intent of fake news spreaders is a time-consuming and challenging task. Around five minutes is required to annotate each instance on average. Understanding the user intent behind a post demands evaluating the tweet content and studying the user based on his or her historical behavior on social media. Such manual annotation for large-scale data is hence impractical, which drives us to consider algorithmic annotation that accurately simulates manual annotation in an automatic manner. Interestingly, we observe that annotators are more confident in identifying intentional fake news spreaders than unintentional ones. Specifically, the expert annotator is at 0.93 confidence level in identifying intentional fake news spreaders and at 0.75 confidence level in identifying unintentional fake news spreaders. For the graduate student annotator, the confidence score is 0.84 and 0.57, respectively. Both results have ≪ 0.001 with Mann-Whitney U test. To conduct algorithmic annotation that can accurately simulate manual annotation, we thus start to think "what kind of fake news spreaders can be intentional?" With the explanations given by annotators, we can reasonably assume bots and trolls who have engaged in fake news propagation as intentional fake news spreaders. As inauthentic and toxic accounts, bots and trolls have been often suspended or deactivated by social media platforms (e.g., Twitter and Facebook) regardless of spreading fake news or not. In fact, they have played a significant role in fake news dissemination [11, 31, 34, 47] . As a comparison, unintentional fake news spreaders deserve a "gentle" strategy developed by social media platforms: nudging and fact-presenting recommendation are more reasonable than suspension and deactivation, as we specified in Section 1. Therefore, we separate bots and trolls from unintentional fake news spreaders. We further notice that users active in fact-checking can spread fake news as well, in a correction manner; i.e., they clarify news is false (objectively, and not aggressively) and inform other users of it in their spreading. We call the corresponding posts that spread fake news correction posts and these users correctors later in the paper. These correctors enables recognizing news falsehood. We thus separate them from unintentional fake news spreaders. We identify bots and trolls by collecting data from two wellestablished and widely accepted platforms, Botometer [29] 7 and Bot Sentinel. 8 Ultimately, each Twitter user is assigned a bot score (denoted as ) and a troll score (denoted as ), where , ∈ [0, 1]. To identify correctors, we first annotate each tweet as a correction or non-correction tweet. Then, we assign each fake news spreader a corrector score (denoted as , where ∈ [0, 1]) by computing the proportion of the user's correction tweets to his or her total tweets that share fake news. With a threshold value, ∈ [0, 1], each fake news spreader can be classified as With identified bots, trolls, and correctors (here, we use 0.5 as the threshold, i.e., = 0.5), the algorithmic annotation on intent of fake news spreaders is conducted at two levels: (i) tweet-level and (ii) user-level. At the tweet-level, the algorithm labels all correction tweets and tweets of bots and trolls that share fake news as intentional spreading. The tweet-level annotation captures the user intent for each spreading action of fake news. At the user-level, the algorithm labels all bots, trolls, and correctors as intentional spreaders. The user-level annotation captures the general user intent when spreading fake news. Tab. 4(b) in Appendix B summarizes the corresponding data statistics. Evaluating Algorithmic Annotations. We compare the algorithmic annotation results with the manual annotations. Results are shown in Tab. 1; results are the same at both the tweet-and userlevels. We observe that the algorithmic annotation effectively simulates the manual annotation, whose AUC score is above 0.8 using sampled MM-COVID and/or ReCOVery datasets. Automatic and manual annotations have a substantial agreement with Cohen's coefficient above 0.64 [10] . With annotated intent (intentional or unintentional) of fake news spreaders, we verify if the assessed intent (i.e., affected degree) 7 https://botometer.osome.iu.edu/ 8 https://botsentinel.com/ differs between intentional and unintentional fake news spreaders and if such difference is statistically significant. In particular, our assessed intent can be validated if affected degrees of intentional fake news spreaders are significantly less than that of unintentional fake news spreaders, i.e., if we estimate fake news spreaders who are annotated as unintentional to be more unintentional than those who are annotated as intentional. As specified in last section, annotations are conducted at both tweet and user levels. Correspondingly, affected degrees are computed at two levels; we further obtain the user-level affected degree by averaging the affected degree of the user's posts sharing fake news. Here we present tweet-level verification results; results at the two levels reveal the same pattern, from which we can draw the same conclusions. First, we present the distribution of affected degrees for intentional and unintentional fake news spreaders (see Fig. 4 ). We observe that, in general, the affected degree of intentional fake news spreaders is less than that of unintentional fake news spreaders. Specifically, the average normalized affected degree of intentional fake news spreaders are 0.55 with MM-COVID data and 0.61 with ReCOVery data. For unintentional fake news spreaders, the value is 0.58 and 0.62, respectively. Such difference is statistically significant with a -value of ≪ 0.001 on MM-COVID and < 0.01 on ReCOVery using -test. Therefore, the results validate our assessment. We conduct the same experiment on the subset of data annotated by humans, where we can draw the same conclusion. Second, we compare the affected degree of bots, trolls, and correctors, which all are annotated as intentional fake news spreaders, with that of others, which are annotated as unintentional fake news spreaders. The results are shown in Fig. 5 . The results indicate that bots, trolls, and correctors all have a lower affected degree compared to unintentional fake news spreaders. The results are statistically significant with a -value of ≪ 0.001 on MM-COVID and < 0.01 on ReCOVery using ANOVA test. Meanwhile, Fig. 6 presents the relationship between affected degree and (i) bot score, (ii) troll score, and (iii) corrector score. The results reveal the same pattern: affected degree drops with an increasing bot, troll, or corrector score. In particular, both bot and troll scores are negatively correlated with affected degrees, with a Spearman's correlation coefficient ∈ [−0.32, −0.24] for bots and ∈ [−0.58, −0.36] for trolls. Results, again, validate our proposed method. Note that when investigating the relationship between affected degree and, e.g., bot score, we remove trolls and correctors to reduce noise. Third, we assess the result robustness. As mentioned before, a fake news spreader is labeled as an unintentional spreader with a bot (troll, or corrector) score less than a threshold value (i.e., ∈ [0, ), = { , , }); otherwise, he or she is an intentional spreader Fig. 7 (the left column). We observe that slightly adjusting the threshold value does not change our observations and conclusions made in the first experiment (i.e., the result is robust). We lastly evaluate the proposed method as follows: we label a fake news spreader whose ∈ [0, ) as an unintentional spreader, and whose ∈ [1 − , 1] as an intentional spreader. By decreasing , a fake news spreader is required to have a lower bot (troll, or corrector) score to be unintentional and a higher bot (troll, or corrector) score to be intentional. In other words, a smaller corresponds to a more strict annotation (intentional or unintentional) of fake news spreaders. We vary among 0.5, 0.3, 0.1 -correspondingly, 1 − varies among 0.5, 0.7, 0.9 -and compare the affected degree of intentional and unintentional fake news spreaders. Results are presented in Fig. 7 (the right column) . We observe that the affected degree of intentional fake news spreaders is always less than that of unintentional fake news spreaders with various thresholds. More importantly, such pattern becomes more significant with a smaller (i.e., a more strict annotation), which validates the effectiveness of our assessment. Finally, we point out that we experiment with (i) external affected degree, (ii) internal affected degree, (iii) combined (exter-nal+internal) affected degree, and (iv) combined affected degree where the external one merely exists between post pairs sharing the same news. The combined one (i.e., iii) is the one where significant and consistent patterns are discovered on both datasets. Using MM-COVID and ReCOVery data, we evaluate the effectiveness of user intent in news propagation to detect fake news. We first employ the assessed affected degree of posts in news propagation within a traditional machine learning framework. Then, we utilize the proposed influence graph within a deep learning framework. I. Combating Fake News by Affected Degree. For each news article, we manually extract over 100 (propagation and content) features as its representation. Propagation features include the average (internal, external, and combined) affected degree of posts spreading the news and a set of widely-accepted propagation features. Content features are extracted using LIWC [22] . See Appendix C for feature details. Five-fold cross-validation and XGBoost [8] are then used with these features for training and classifying news articles. Results indicate that this method correctly identifies fake news with an AUC score of around 0.93. As a comparison, dEFEND [32] , a state-of-the-art method that detects fake news by news content and propagation information, performs around 0.90. Furthermore, we observe that, as presented in Tab. 3, the proposed method performs above 0.89 with limited propagation information of news articles, i.e., at an early stage of news dissemination on social media. Notably, internal affected degree of posts greatly contributes to detecting fake news, whose feature importance assessed by XGBoost ranks top five all along. II. Combating Fake News by Influence Graph. We construct the news-post heterogeneous graph (shown in Fig. 8 ); a post is connected with a news article if the post shares the news, and the relation among posts is modeled by the proposed influence graph . Then, we train the HetGNN (Heterogeneous Graph Neural Network) model [42] with this news-post graph to learn news representation, with which XBGoost [8] is further utilized to predict fake news. Varying the percentage of labeled news from 20% to 80%, this method performs with an AUC score ranging from 0.83 (with small-scale training data) to 0.91 (with relatively large-scale training data) on two datasets. To further evaluate the proposed influence graph , we consider two variant groups of the constructed heterogeneous graph as baselines. One replaces by a random version ( Random ): Based on our graph sparsification strategy (see Appendix A), we construct the random graph by randomly selecting a hundred posts for each post ensuring that no self-loops are formed in this graph. The other replaces by its subgraph (i) with internal influence only ( Internal ); (ii) with external influence only ( External ); or (iii) with internal and external influence but the latter only exists between two posts sharing the same news ( Same News ). Tab. 2 presents the full result; Subgraph in the table refers to Same News , which performs best among all subgraphs. We observe that in general, the proposed influence graph outperforms its variants in detecting fake news, especially with limited training data. See Appendix C for other implementation details. We look into the phenomenon that social media users can spread fake news unintentionally. With social science foundations, we propose influence graph, with which we assess the degree to which fake news spreaders are unintentional (denoted as affected degree). Strategies to sparse the influence graph and normalize the affected degree by determining its upper bound are presented as well. We develop manual and automatic annotation mechanisms to obtain the ground-truth intent (intentional or unintentional) of fake news spreaders for MM-COVID and ReCOVery data. We observe that the affected degree of intentional fake news spreaders are significantly less than that of unintentional ones, which validates our assessments. This work helps combat fake news from two perspectives. First, our assessed intent helps determine the necessity of a fake news spreader being nudged or recommended with (users active in sharing) facts. Second, we present that the assessed spreader intent and proposed influence graph effectively help detect fake news with an AUC score of around 0.9. Limitations and Future Work: We effectively assess the degree to which fake news spreaders are unintentional, but remain the task to classify a fake news spreader as an intentional or unintentional spreader. We point out that merely relying on determining a threshold for affected degree is barely enough. To address this problem, we aim to propose a more complicated classification model in the near future, which involves non-posting behavior (e.g., commenting, liking, and following) of news spreaders. This research was supported in part by the National Science Foundation under award CAREER IIS-1942929. We sincerely appreciate the positive and constructive comments of the reviewers. We also thank Chang Liu, Shengmin Jin, and Hao Tian for their useful suggestions in data annotation. VoterFraud2020: A Multi-modal Dataset of Election Fraud Claims on Twitter FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP Social Identity Theory and the Organization Longformer: The Long-Document Transformer Viral Misinformation: The Role of Homophily and Polarization The Validity Effect: A Search for Mediating Variables A Topic-Agnostic Approach for Identifying Fake News Pages XGBoost: A Scalable Tree Boosting System Causal Understanding of Fake News Dissemination on Social Media A Coefficient of Agreement for Nominal Scales What types of COVID-19 conspiracies are populated by Twitter bots? Selective exposure Social Identity Theory Availability Cascades and Risk Regulation The science of fake news MM-COVID: A Multilingual and Multimodal Data Repository for Combating COVID-19 Disinformation Cass R Sunstein, and Ralph Hertwig. 2020. How behavioural sciences can promote truth, autonomy and democratic discourse online Political fact-checking on Twitter: When do corrections have an effect? Cognitive dissonance or credibility? A comparison of two theoretical explanations for selective exposure to partisan news The market for news Confirmation bias: A ubiquitous phenomenon in many guises The development and psychometric properties of LIWC2015 Prior exposure increases perceived accuracy of fake news Shifting attention to accuracy can reduce misinformation online Automatic Detection of Fake News Hierarchical Multi-modal Contextual Attention Network for Fake News Detection Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Detection of Novel Social Bots by Ensembles of Specialized Classifiers Science audiences, misinformation, and fake news The spread of low-credibility content by social bots dE-FEND: Explainable Fake News Detection SciLens: Evaluating the Quality of Scientific News Articles Using Social Media and Scientific Literature Indicators Disinformation's spread: bots, trolls and all of us Attention Is All You Need The Rise of Guardians: Fact-checking URL Recommendation to Combat Fake News The spread of true and false news online Learning fine-grained image similarity with deep ranking EANN: Event Adversarial Neural Networks for Multi-Modal Fake News Detection Two components of long-term memory SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics Heterogeneous graph neural network Multimodal Deception Detection Using Automatically Extracted Acoustic, Visual, and Lexical Features Users' participation and social influence during information spreading on Twitter ReCOVery: A Multimodal Repository for COVID-19 News Credibility Research SAFE: Similarity-Aware Multi-Modal Fake News Detection A survey of fake news: Fundamental theories, detection methods, and opportunities [22] , each of which falls into one of the categories including word count, summary language variables, linguistic dimensions, other grammars, and psychological processes. With HetGNN, we use pre-trained transformers to extract content features of nodes (Longformer [4] for news stories and Sentence-BERT [28] for tweets). The news node is associated with the news embedding and the average embedding of its connected posts. The post node is associated with the post embedding, the average embedding of its connected news, and the average embedding of its connected posts. Hence, the Bi-LSTM length of news content encoder is two, and that of post content encoder is three. For both datasets, the embedding dimension of HetGNN is 1024, the size of sampled neighbors set for each node is 23 (3 news nodes plus 20 post nodes), the learning rate is 0.0001, and the maximum number of training iterations is 50. The other hyperparameters are set the same as mentioned in [42] .