key: cord-0580478-xcwz5ikf
authors: Solovev, Kirill; Prollochs, Nicolas
title: Moral Emotions Shape the Virality of COVID-19 Misinformation on Social Media
date: 2022-02-08
journal: nan
DOI: nan
sha: c6513de6e5e321e1cd5f4e3158e043fd4b6277ae
doc_id: 580478
cord_uid: xcwz5ikf

While false rumors pose a threat to the successful overcoming of the COVID-19 pandemic, an understanding of how rumors diffuse in online social networks is - even for non-crisis situations - still in its infancy. Here we analyze a large sample consisting of COVID-19 rumor cascades from Twitter that have been fact-checked by third-party organizations. The data comprises N=10,610 rumor cascades that have been retweeted more than 24 million times. We investigate whether COVID-19 misinformation spreads more viral than the truth and whether the differences in the diffusion of true vs. false rumors can be explained by the moral emotions they carry. We observe that, on average, COVID-19 misinformation is more likely to go viral than truthful information. However, the veracity effect is moderated by moral emotions: false rumors are more viral than the truth if the source tweets embed a high number of other-condemning emotion words, whereas a higher number of self-conscious emotion words is linked to a less viral spread. The effects are pronounced both for health misinformation and false political rumors. These findings offer insights into how true vs. false rumors spread and highlight the importance of considering emotions from the moral emotion families in social media content.

Social media platforms play an ambivalent role during the COVID-19 pandemic. On the one hand, they represent an important source of health information for large parts of society [39] . On the other hand, however, this crisis has bred a multitude of rumors [24, 26, 33, 36, 45] , and verdicts of reputable fact-checking organizations (e. g., politifact.com, snopes.com) suggest that social media is rife with COVID-19 misinformation. COVID-19 misinformation on social media includes, but is not limited to, misinformation about vaccination, "miracle cures, " and supposed preventives [32] . False rumors can impact the timely and effective adoption of public health recommendations [66] , the effectiveness of the countermeasures deployed by governments [50] , and are sometimes even used as a political weapon [51] . Given that exposure to misinformation frequently manifests in offline consequences [45] , there is an urgency to study the spread of rumors on social media in the context of COVID-19. Tedros Adhanom Ghebreyesus, director-general of WHO, and other experts speak of an "infodemic, " which must be fought [71] .

While previous research -at least for non-crisis situationssuggests that false rumors on social media tend to be more viral than the truth [48, 65] , the mechanism underlying its viral spread, though critical, remains unresolved. In this work, we approach this question through the lenses of morality and emotions and their role in rumor diffusion in polarized social media environments. Social media content delivers not only factual information but also carries moral ideas and sophisticated emotional signals [8] . Moral emotions provide the motivational force for humans to do good and to avoid doing bad [59] and can even serve to "moralize" actions that would otherwise be considered non-moral [68] . Since socially connected users often develop similar ideas and intuitions [8, 13, 23, 39] , moral emotions are a key driver of information diffusion in polarized social media environments [8] . In the context of COVID-19, the overall discussion culture has repeatedly been characterized as highly polarized [1, 17, 21, 31, 32, 34] . For instance, people have been observed to be divided in their perceptions of government responses, confidence in scientists, and support for protective actions [31, 34] . If COVID-19 rumors are highly polarizing to social media users, then the transmission of moral emotions likely plays a key role in the rumors' diffusion through social networks.

The principal moral emotions can be divided into two families [30] . The families are the "other-condemning" family, comprising the emotions contempt, anger, and disgust, and the "self-conscious" family comprising the emotions shame, pride, and guilt [60] . The former, other-condemning emotions, are reactions to the social behavior of others and involve a negative judgment or disapproval of others. In the context of morality, other-condemning emotions are sometimes also referred to as the "hostility triad" [52] . Othercondemning emotions are typically associated with perceived moral violations, for example, in the context of individuals' rights and fairness [62] . While the individual emotions in the other-condemning family (i. e., anger, contempt, and disgust) are often assumed to be not particularly explosive on their own, they can become a dangerous, explosive mix when compressed together [52] . Their counterpart is the family of self-conscious emotions, which are evoked by self-reflection and self-evaluation. These emotions motivate individuals to behave in a socially acceptable fashion and are linked to prosocial behaviors such as empathy and altruism [30, 52] . As such, self-conscious emotions can enable social healing and avoid triggering the contempt, anger, and disgust of others [30] . While previous research [8] broadly distinguished moral vs. non-moral emotions in social media content, we will investigate whether these two clusters of moral emotions (self-conscious vs. other-condemning emotions) have distinct effects on the diffusion of rumors in the context of COVID-19.

Research hypothesis: In this work, we propose that the virality of true vs. false COVID-19 rumors can be explained by the moral emotions they carry. Although previous research suggests that false rumors are statistically more often retweeted [48, 65] , not every false rumor is necessarily more viral than a truthful rumor. Rather, misinformation going viral is oftentimes spread through echo chambers with exacerbated ideological polarization [12] . In these environments, ideological identity is more salient in guiding user behavior [67] and users are moved towards more extreme positions [14] . Polarization not only reduces verification behavior [35, 41] but also makes users more receptive to hostility against others, e. g., for political attacks [61] . Here other-condemning emotions embedded in the source tweets, which start the rumor cascade, may function as accelerators and amplifiers [63] . In polarizing discussions about COVID-19, this would imply that radical ideas and beliefs are strengthened and are more likely to translate into action. Given increased ideological polarization for false rumors [20] , the explosive mix of other-condemning emotions should thus accelerate their spread within social networks. The same reasoning suggests that false rumors embedding self-conscious emotions (that avoid triggering other-condemning emotions [30] ) should be less contagious on social media. In sum, we hypothesize that rumors with a stronger combination of false content and other-condemning emotions in the source tweets reach more people, whereas the combination of false content and self-conscious emotions reaches fewer people.

Data: We collected a unique dataset of COVID-19 rumor cascades propagating on Twitter between January 2020 and the end of April 2021. Each rumor cascade was investigated and fact-checked by at least one of three independent fact-checking organizations (snopes.com, politifact.com, truthorfiction.com). Our data include 10,610 rumor cascades that have been retweeted 24.34 million times.

Methodology: We use textual analysis to extract fine-grained moral emotions (self-conscious and other-condemning) embedded in rumor cascades. Specifically, we employ (and validated) a dictionary-based approach to count the frequency of occurrence of self-conscious and other-condemning emotion words in the source tweets that have initiated the rumor cascades. To measure the diffusion of each rumor cascade, we employ the Twitter Historical API to obtain the number of retweets, that is, the number of users interacting with the rumor cascade. We then fit explanatory regression models to evaluate how variations in moral emotions are associated with differences in the number of retweets for true vs. false rumor cascades. In our regression analysis, we follow previous works [8, 65] by controlling for variables known to affect the retweet rate independent of the main predictors, i. e., the number of followers, the account age, etc.

Findings: We observe that, on average, COVID-19 misinformation is more likely to go viral than truthful information. However, the veracity effect is moderated by moral emotions: false rumors are more viral than the truth if the source tweets embed a high number of other-condemning emotion words, whereas a higher number of self-conscious emotion words is linked to a less viral spread. The effects are pronounced both for health misinformation and false political rumors. These findings offer insights into how true vs. false rumors spread and highlight the importance of considering emotions from the moral emotion families in social media content.

Social media has shifted quality control for the content from trained journalists to regular users [35] . The lack of oversight from experts makes social media vulnerable to the spread of misinformation [53] . Social media has indeed repeatedly been observed to be a medium that disseminates vast amounts of misinformation [e. g., 47, 65] . The presence of misinformation on social media also has detrimental consequences on how opinions are formed in the offline world [2, 4, 19, 42] . As a result, it not only threatens the reputation of individuals and organizations, but also society at large.

Several works have focused on the question of why misinformation is widespread on social media. These studies suggest that it is difficult for users to spot misinformation as it is often intentionally written to mislead others [69] . Moreover, social media users are often in a hedonic mindset and avoid cognitive reasoning such as verification behavior [41] . The vast majority of social media users do not fact-check articles they read [27, 64] . A recent study further suggests that the current platform design may discourage users from reflecting on accuracy [44] . Online social networks are also characterized by (political) polarization [38, 47, 55] and echo chambers [5] . In these information environments with low content diversity and strong social reinforcement, users tend to selectively consume information that shares similar views or ideologies while disregarding contradictory arguments [22] . These effects can even be exaggerated in the presence of repeated exposure: once misinformation has been absorbed, users are less likely to change their beliefs even when the misinformation is debunked [43] .

Several studies have analyzed the spreading dynamics of rumors vs. non-rumors on social media. This includes analyses of summary statistics with regard to, for instance, the number of retweets [e. g., 7, 25] and the rumor lifetime [e. g., 7, 10, 19] . However, these works discern cascades from rumors vs. non-rumors, and do not focus on differences across veracity. Another stream of literature has analyzed rumors concerning specific events (e. g., the 2013 Boston Marathon bombing) with regard to the overall tweet volume or content [e. g., 18, 56, 57] . These works analyze how the user base responds to rumors but again do not analyze the diffusion dynamics of true vs. false rumors.

Only a few works have analyzed differences in the spread of true vs. false rumors. Friggeri et al. (2014) [25] classified the veracity of ≈4,000 rumors from Facebook based on fact-checking assessments from snopes.com. The authors find that a majority of resharing of false rumors occurs after fact-checking. This suggests that social media users likely do not notice the fact-checks; or intentionally ignore their verdict. Closest to our work is the study from Vosoughi et al. (2018) [65] , which provides a comprehensive analysis of summary statistics of true vs. false rumors on Twitter, finding that false rumors spread significantly farther, faster, and more broadly than the truth. However, this work does not analyze the spread of true vs. false rumors in the context of COVID-19. The same dataset [65] has also been used in a recent study [48] that measures emotions embedded in the replies to rumor cascades. The authors find that higher frequencies of certain emotions (e. g., anger) are associated with more viral cascades for false rumors.

In the context of COVID-19, research providing large-scale quantitative analyses of the spread of true and false rumors is scant. Existing works have primarily focused on summary statistics of small sets of hand-labeled rumors or source-based approaches to identify COVID-19 misinformation [e. g., 15, 36, 54] . For example, Cinelli et al. (2020) classify news sources into reliable and nonreliable sources in order to analyze the spread of COVID-19-related content. The authors find no significant differences regarding the spreading dynamics. Notably, however, categorizations of reliable vs. non-reliable sources do not necessarily correspond to true vs. false rumors. In addition, source-based approaches ignore false rumors from influential individuals, emerging websites, and misclassify false rumors from websites that are generally considered as being reliable. Note that there are other recent papers reporting that COVID-19 misinformation is widespread on social media, characterizing COVID-19 misinformation, and expressing concerns about consequences for public health [e. g., 26, 28, 33, 36, 45] . However, these works do not focus on modeling differences in the diffusion of true vs. false COVID-19 rumor cascades.

Our contributions: This work makes two key contributions. (1) We collected a unique dataset of COVID-19 rumor cascades and demonstrate that misinformation is, on average, more viral than the truth. Here, our study connects to previous works [48, 65] , which yielded similar conclusions, yet not in the context of COVID-19. (2) The mechanisms underlying the viral spread of false rumors, though critical, have remained largely unresolved in previous research. Our work is the first to approach the question through the lenses of morality and emotions -finding that moral emotions embedded in source tweets shape the diffusion of false rumors on social media.

Fact-checks: We identified three fact-checking organizations that thoroughly investigate rumors related to COVID-19. The names of the fact-checking organizations are: politifact.com, truthorfiction. com, and snopes.com. These fact-checking organizations list COVID-19 rumors in separate categories or tag them with a topic label (e. g., "COVID-19", "Coronavirus") which allows us to distinguish COVID-19-related rumors from other rumors (see Tbl. 1). We scraped all COVID-19-related fact-checks from these platforms. The fact-checking organizations have different ways of labeling the veracity of a rumor. For example, politifact.com articles are given a "Pants on Fire" rating for false rumors, whereas snopes.com assigns a "false" label. Consistent with Vosoughi et al. [65] , we normalized the veracity labels across the different sites by mapping them to a score of 1 to 5. All rumors with a score of 1 or 2 were categorized as "false, " whereas rumors with a score of 4 or 5 were categorized as "true. " Rumors with a score of 3 were categorized as "mixed. " In some cases, the same rumors have been investigated by multiple fact-checking organizations. Previous research has shown that fact-checking websites show high pairwise agreement [65] , ranging between 95 % and 98 %. Rumors classified as "true" or "false" even showed a perfect pairwise agreement of 100 % [65] . The resulting collection of fact-checks contained the following information: (i) the veracity label ("true", "false", "mixed"), (ii) links to the articles of the fact-checking organizations, and (iii) the headline of the article that is being verified.

Rumor cascades on Twitter: We followed the approach from Vosoughi et al. (2018) to identify rumor cascades on Twitter: A rumor cascade on Twitter starts with a user making an assertion about a topic such as tweeting a text message or a link to an article. Social media users then propagate the rumor by retweeting it. Oftentimes, people also reply to the original tweet. These replies sometimes contain links to fact-checking organizations that either confirm or debunk the rumor in the original tweet. We used such cascades to identify rumor cascades that are propagating on Twitter.

We employed the Twitter Historical API to map the rumors to retweet cascades on Twitter as follows. First, we collected all tweets that contain a link to any of the websites from the fact-checking organizations. Second, for each reply tweet, we extracted the original tweet and the number of retweets of the original tweet. Here, special care is needed to ensure that the replies containing a link to any of the trusted websites address the original tweet. We followed the approach from Vosoughi et al. [65] to address this important issue: (i) we considered only replies to the original tweet and exclude replies to replies. (ii) To ensure that we study how unverified and contested information diffuses on Twitter, we removed all original tweets that are directly linking to one of the fact-checking websites. Note that tweets linking to one of the fact-checking websites do not qualify as they are no longer unverified. (iii) We compared the headline of the linked article to that of the original tweet. For this purpose, we used Universal Sentence Encoder [11] to convert the headline of the fact-check and the original tweet to vector representations that capture their semantic content. We then used cosine similarity to measure the distance between the vectors. If the cosine similarity was lower than 0.4, the tweet was discarded.

The retweet cascades remaining after these filtering steps then represent rumors propagating on Twitter -for which a veracity label is known based on the assessment from the fact-checking organization. In our data, the frequencies of fact-checking labels at cascade level are: 3,748 (=true), 4,979 (=false), and 1,883 (=mixed). These 10,610 rumor cascades have received more than 24.33 million retweets by Twitter users.

Following previous works [8, 65] , we employed the Twitter API to collect a set of additional user variables for each source tweet, i. e., the number of followers, the account age, etc. These variables are known to affect the retweet rate and are later used as control variables in our regression model. Summary statistics of our dataset are reported in Tbl. 2.

The "other-condemning" family of moral emotions comprises the emotions anger, disgust, and contempt, whereas the "self-conscious" family comprises the emotions shame, pride, and guilt [30, 60] . We employed text mining methods to measure the extent to which these emotions are embedded in the source tweets. For this purpose, we first applied standard preprocessing steps from text mining. Specifically, the running text was converted into lower-case and tokenized, and special characters (e. g., hashtags, emoticons) were removed. Subsequently, we applied (and validated) a dictionarybased approach analogous to earlier research [8, 48, 65] .

We measured other-condemning and self-conscious emotions embedded in the source tweets based on the NRC emotion lexicon [40] . This lexicon comprises 181,820 English words that are classified according to the emotions of Plutchik's emotion model [46] . Plutchik's emotion model defines 8 basic emotions and 24 emotional dyads. The emotional dyads represent complex emotions, which are derived as a combination of two basic emotions [49] . We used the NRC dictionary to count the frequency of words in the tweets that belong to each of the emotions. Afterwards, we divided the word counts by the total number of dictionary words in the text, so that the vector is normalized to sum to one across the emotions [48, 65] . In our data, 78.15% of all source tweets contained at least one emotion word from the NRC lexicon. We filtered out tweets that do not contain any emotional words since, otherwise, the denominator is not defined [48, 65] . However, our later analysis yields qualitatively identical results when including these observations (i. e., assigning zero values). Based on the scores for the 8 basic emotions and the 24 derived emotions, and the definitions of the two moral emotion families [60] , we calculated other-condemning emotions by taking the sum of anger, disgust, and contempt. Self-conscious emotions were calculated by taking the sum of shame, pride, and guilt.

User study: In order to test the construct validity of our dictionarybased approach, we employed two trained research assistants to annotate a random subset of 200 tweets that were categorized as being more other-condemning than self-conscious based on the dictionaries; and a random subset of 200 tweets that were categorized as being more self-conscious than other-condemning. For each of the 400 tweets, the annotators were asked to what extent the tweet relates to other-condemning and self-conscious emotions on two 5-point Likert scales, ranging from 1 ("not related to [other-condemning, self-conscious] emotions at all") to 5 ("very related to [other-condemning, self-conscious] emotions"). The annotators viewed the tweets in randomized order and were explained the difference between other-condemning and self-conscious emotions. The annotators exhibited a statistically significant inter-rater agreement according to Kendall's ( < 0.01). Furthermore, the annotators rated the random subset of other-condemning tweets as more "other-condemning" than "self-conscious" [ = 6.53, < 0.001]; and the random subset of self-conscious tweets as more "self-conscious" than "other-condemning" [ = 4.50, < 0.001].

We employed a weakly supervised machine learning framework [70] to infer the topics in the source tweets that have initiated the rumor cascades. The benefit of this state-of-the-art approach is that (i) it is regarded as superior to conventional topic modeling (i. e., Latent Dirichlet Allocation) for short texts [70] , and (ii) its weakly supervised nature allows for an ex-ante selection of topics that we perceive as being particularly relevant in the context of COVID-19. We categorized the rumor cascades into three (not mutually exclusive) topics: Health (e. g., rumors about the safety of vaccines), Politics (e. g., allegations of political opponents), and Other (i. e., rumors that do not fall into one of the other categories). Example tweets for each topic are provided in Tbl. 3.

Our weakly supervised machine learning framework proceeded in three steps (see Yao et al. [70] for methodological details): (1) We started to identify topic-related tweets based on a set of manually selected keywords for each topic. For instance, for the topic Health, we searched for all tweets containing words such as "vaccine, " "flu, " "mask, " etc. (see list of keywords in the Supplementary Materials).

(2) We conducted clustering-assisted manual word sense disambiguation on the keyword-identified tweets [70] . Here we used the -means clustering algorithm with Silhouette criterion to cluster the keyword-identified tweets for each topic. We then manually inspect random tweets sampled from each cluster and assessed whether the tweets in the cluster refer to the topic. We excluded each tweet cluster that does not show the pertinent meaning of the topic keyword. This allowed us to significantly clean and improve the quality of the keyword-identified tweets. (3) We used the created labeled data to train a deep neural network classifier and learn to predict whether or not individual Twitter messages belong to a certain topic. The input data for the training machine learning classifier was a vector representation of the (cleaned) keyword-identified tweets and the topic label. To create vector representations of tweets, we used neural language models in the form of the Universal Sentence Encoder [11] . In our deep neural network classifier, we treated the task of predicting topic labels for (vector representations of) tweets as a multi-label problem considering that one tweet may belong to multiple topics (i. e., Health and Politics). In training, we used an equal number of 1000 keyword-identified Tweets for each topic as positive training instances. In addition, we used the excluded tweets from step (2) and randomly sampled unlabeled tweets equal to the sum of labeled tweets as negative training instances, i. e., with a topic label Other.

User study: To ensure that the topic predictions are accurate, we tested for the presence of errant tweets with the help of two trained research assistants. We randomly sampled 200 tweets for each topic, and instructed the research assistants to annotate the tweets. Each annotator was asked to judge the validity of the topic label on a 5-point Likert scale, ranging from 1 ("not related to [topic] at all") to 5 ("very related to [topic]"). When comparing the human annotations to the predicted topic labels, we found very few misclassified instances. On average, the share of tweets that were not classified as at least "somewhat related to [topic]" was lower than 8.5 % (see Supplementary Materials).

We specified regression models with interaction terms that explain the number of retweets based on rumor veracity and othercondemning emotions and self-conscious emotions. Let RetweetCount i denote the number of retweets for rumor cascade . Furthermore, let OtherCondemning i denote the proportion of other-condemning emotions, SelfConscious i the proportion of self-conscious emotions, and Falsehood i the veracity. Here we define a true rumor as ℎ = 0 and a false rumor as Falsehood i = 1 . We adjusted for variables known to affect retweet rate [8, 47-49, 58, 65] , which included the number of followers (Followers i ) and followees (Followees i ) of the author of the tweet, the account age (AccountAge i ), whether the author was verified by Twitter (Verified i ), and whether media was attached to the tweet (HasMedia i ). Each of these factors was extracted from the Twitter API. We −standardized all continuous predictors in order to facilitate interpretability.

Based on the above variables, we specified the following generalized linear model for our analysis:

RetweetCount is a non-negative count variable, and its variance is larger than the mean. To adjust for overdispersion, we drew upon a negative binomial regression [8, 58] . Note that because we estimate a negative binomial regression model with interaction terms, the coefficients cannot be interpreted as the change in the mean of the dependent variable for a one unit (i. e., standard deviation) increase in the respective predictor variable, with all other predictors remaining constant. The reason is that in nonlinear regression models with interaction terms, marginal effects are nonlinear functions of the coefficients and the levels of the explanatory variables [9] . Instead, the coefficients can be interpreted on a multiplicative scale by calculating the incidence rate ratio (IRR), which is equal to the exponent of the coefficient of the respective variable [9] . Here the coefficients can be interpreted as the natural logarithm of a multiplying factor by which the predicted number of retweets changes, given a one unit increase in the predictor variable, holding all other predictor variables constant [9] .

Our data include 10,610 rumor cascades that have been retweeted 24.34 million times. The total number of COVID-19 rumor cascades peaked in March 2020 when the U. S. government declared a national emergency concerning the coronavirus disease and again in October 2020, the month prior to the U. S. presidential elections (Fig. 1A) . The three fact-checking organizations have categorized 46.9 % of all rumors as false, 35.3 % as true, and 17.7 % as being of mixed veracity. While the absolute number of rumor cascades has decreased over the course of the pandemic, the relative share of false vs. true rumors has increased (Fig. 1A) . Compared to false rumors, a greater fraction of true rumors experienced more than 100 rumor cascades (Fig. 1B) . COVID-19 rumors are not constrained exclusively to health topics (e. g., rumors about the safety of vaccines). Rather, a sizable number of COVID-19 rumors concern political topics (e. g., true or false allegations of political opponents) [17] . We thus applied topic modeling to categorize the rumor cascades in our dataset into three (not mutually exclusive) topics: Politics, Health, and Other. Fig. 1C shows that a large proportion of COVID-19 rumors were thematically related to Politics (76.9 %), Health (38.8 %), while only 12.2 % concerned Other topics (e. g., conspiracy theories). A total share of 34.1 % of rumor cascades were thematically related to both Politics and Health. Regression analysis: We fitted explanatory regression models to evaluate how variations in moral emotions are associated with differences in the number of retweets for true vs. false rumor cascades. In our regression analysis, we followed previous works [8, 65] by controlling for variables known to affect the retweet rate independent of the main predictors, i. e., the number of followers, the account age, etc.

As a baseline, we started our regression analysis with a negative binomial regression explaining the number of retweets solely based on the veracity label and control variables (see Supplementary Materials). Here false rumors (Falsehood = 1) were estimated to receive 15.66 % more retweets than true rumors (IRR 1.16; < 0.01). Subsequently, we extended the negative binomial regression by including interaction terms between rumor veracity and other-condemning emotions, and between rumor veracity and self-conscious emotions ( Fig. 2A) . The coefficient estimates for these two interaction terms were statistically significant, which implies that false rumors' virality depended on the moral emotions embedded in the source tweet. Specifically, a one standard deviation increase in other-condemning emotions for false rumors was linked to a 26.99 % increase in the number of retweets (IRR 1.27; < 0.01). In contrast, a one standard deviation increase in self-conscious emotions for false rumors was linked to a 23.43 % decrease in the number of retweets (IRR 1.23; < 0.01). We found no statistically significant effect of other-condemning and self-conscious emotion words for true rumors. In sum, we observed that false rumors were more viral than the truth if the source tweet embedded a high proportion of other-condemning emotion words, whereas a high proportion of self-conscious emotion words was linked to a less viral spread (see Fig. 2B, 2C) .

Analysis across topics: We also examined the effect of moral emotions across different topics. For each topic from Fig. 1C , we generated observation subsets and re-estimated our regression model (Fig. 3) . We observed differences in the effects of moral emotions on the number of retweets. The effect of other-condemning emotions on the number of retweets was pronounced both for false rumors from the Health category (IRR 1.34; < 0.01) and for false rumors from the Politics category (IRR 1.61; < 0.01). For the Other category (with comparatively smaller sample size), the coefficients pointed in the same directions but were not statistically significant at common significance thresholds.

Analysis of deleted tweets: Major social media platforms such as Twitter have intensified their efforts to combat the spread of misinformation on their platforms by deleting misinformation [6] . While the Twitter API does not provide access to the content of source tweets that have been deleted, we were still able to analyze some of their characteristics. As part of an exploratory analysis, we found that 3663 potential rumor starter tweets have been deleted (either by Twitter or by the users themselves). An overwhelming majority of those (68.35 %) are potentially false rumors. Hence, even though these numbers suggest that a relevant proportion of false rumors on Twitter has been deleted, the vast majority of false rumors continue to circulate. For those rumors, our results demonstrate that falsehood can be more viral than the truth.

Additional checks 1 : Numerous exploratory analyses and checks validated our results and confirmed their robustness: (i) Since selfconscious emotions can be regarded as the counterpart of othercondemning emotions, we tested an alternative model specification in which we included the difference between the emotion scores for other-condemning and self-conscious emotions instead of two individual variables. Consistent with our main analysis, we found that false rumors are more viral than the truth if the source tweet embeds a high proportion of other-condemning emotion words, whereas a high proportion of self-conscious emotion words is linked 1 Detailed results are reported in the Supplementary Materials. to a less viral spread. (ii) In our main analysis, we focused on rumors that are clearly true or false. However, 17.7 %, of all rumors have been categorized as being of mixed veracity by the fact-checking organizations. We tested whether counting rumors of mixed veracity as either true or false affects the validity of our results. We find that our results are robust and that the combination of othercondemning emotion and mixed veracity is similarly viral as the combination of other-condemning emotions and false veracity. (iii) In our data, 9.48 % of rumor starters have started more than one retweet cascade. To ensure that our models are not biased due to this source of non-independence, we dropped all users with clustering and reestimated the models. The results are robust and support our findings. We also repeated our analysis with monthly fixed effects to control for differences in the virality of rumor cascades due to different start dates. Also here, the results confirmed the findings from our main analysis. (iv) We repeated our analysis for subsets of rumor cascades that have been started by users that are either verified or not verified by Twitter. We find that our main findings hold for both user groups.

Here we provide evidence that moral emotions play a crucial role in the spread of COVID-19 misinformation on social media. Using a comprehensive dataset of COVID-19 rumors that have been fact-checked by three independent fact-checking organizations (snopes.com, politifact.com, truthorfiction.com), we establish that other-condemning emotions -also known as the hostility triadare linked to a more viral spread of false rumors.

While false rumors pose a threat to the successful overcoming of this pandemic, an understanding of how rumors diffuse in online social networks is -even for non-crisis situations -still in its infancy. Analyzing the spreading dynamics of fact-checked rumors is to a great extent generalizable to the spread of other (non-factchecked) rumors on social media [65] . Our finding that COVID-19 misinformation is, on average, more viral than the truth directly connects to the study from Vosoughi et al. [65] , which yielded similar findings, yet outside the context of COVID-19. Previous research has also shown that misinformation on social media can have negative offline consequences. Among other instances, this has previously been confirmed to be the case during humanitarian crises [57] and elections [2] [3] [4] 29] . Our observation that COVID-19 misinformation is both widespread and viral on social media is at least equally concerning. COVID-19 misinformation not only poses severe health risks to individuals but also undermines the integrity of the political discourse [50] .

The results of this study highlight the role of moral emotions in rumor diffusion. Previous research [8] broadly distinguished moral vs. non-moral emotional expressions in social media content, while this work demonstrates that the two clusters of moral emotions (self-conscious vs. other-condemning emotions) have distinct effects on social transmission in the context of true and false rumors. We observe that false rumors receive more retweets than true rumors if the source tweets embed a high share of other-condemning emotions, whereas we find the opposite pattern, yet of smaller magnitude, for self-conscious emotions. Another relevant finding is that the expression of other-condemning emotion on virality is pronounced both for health misinformation and political misinformation. These findings may be partially explained by the high level of polarization of social media users in the context of COVID-19. In polarizing debates, radical ideas and beliefs are strengthened and more likely to translate into action. It thus seems plausible that the explosive mix of other-condemning emotions accelerates the spread of false rumors about those topics within social networks.

From a practical perspective, policy initiatives around the world urge social media platforms to limit the spread of false rumors [37] . While previous research has studied emotions in replies to rumor cascades [48, 49] , our work highlights the importance of considering (moral) emotions in the source tweets that have initiated the rumor cascades. These findings could eventually be leveraged in machine learning models in order to detect false rumors more accurately. Emotion scores for source tweets are available immediately upon the beginning of the diffusion process -a time point at which features from propagation dynamics are scarce [16] . Our findings may also be relevant with regard to other downstream tasks such as educational applications. Altogether, considering moral emotions in social media posts might help future works to develop more effective strategies against false rumors.

This work is subject to the typical limitations of observational studies. We report associations and refrain from making causal claims. Future work should seek to corroborate our conclusions in controlled laboratory experiments and, in particular, test the causal influence of exposure to moral-emotional language on attitudes and behavior. Our inferences are also limited by the accuracy and availability of our data, specifically those from the three different fact-checking websites. For those, however, our data comprises all COVID-19 rumor cascades on Twitter until the end of April 2021. Despite these limitations, we believe that observing and understanding how misinformation spreads is the first step toward containing it. We hope that our work inspires more research into the causes, consequences, and potential countermeasures for the spread of misinformation -both in crisis and non-crisis situations.

While false rumors pose a threat to the successful overcoming of this pandemic, an understanding of "what makes false rumors viral" is -even for non-crisis situations -still in its infancy. In this work, we approach this question through the lenses of morality and emotions and their role in rumor diffusion in polarized social media environments. For this purpose, we collected a unique dataset of COVID-19-related rumor cascades from Twitter and empirically analyze their spreading dynamics. We find that COVID-19 misinformation is, on average, more viral than the truth. However, the veracity effect is moderated by moral emotions: false rumors are more viral than the truth if the source tweets embed a high number of other-condemning emotion words, whereas a higher number of self-conscious emotion words is linked to a less viral spread. These findings offer insights into how true vs. false rumors spread and highlight the importance of considering moral emotions in social media content.

A TOPIC MODELING Table 4 shows the manually selected seed words that were used to identify topic-related tweets in weakly supervised learning. Table 4 reports the results for our user study testing for the presence of errant tweets. On average, the share of tweets that were not classified as at least "somewhat related to [topic]" was lower than 8.5 %. Table 4 : Seed words used to identify topic-related tweets in weakly supervised learning. Various word forms of the keywords are also considered, e. g., "masks" and "masking" are also considered for the keyword "mask". 

We tested a model specification in which we only incorporated control variables from previous works. Table 6 shows that rumors receive a particularly high number of retweets if they are false and if they have been started by users with a larger number of followers. Sign. levels: * < 0.05, * * < 0.01, * * * < 0.001; standard errors in parentheses

We repeated our analysis for subsets of rumor cascades that have been started by users that are verified or not-verified by Twitter. Table 7 shows that our main findings hold for both user groups. Sign. levels: * < 0.05, * * < 0.01, * * * < 0.001; standard errors in parentheses D RUMORS WITH MIXED VERACITY In our main analysis, we focused on rumors that are clearly true or false. However, 17.7 %, of all rumors have been categorized as being of mixed veracity by the fact-checking organizations. We tested whether counting rumors of mixed veracity as either true or false affects the validity of our results. Table 8 shows that our results are robust and that the combination of other-condemning emotion and mixed veracity is similarly viral as the combination of other-condemning emotions and false veracity.

In our data, 9.48 % of rumor starters have started more than one retweet cascade. To ensure that our models are not biased due to this source of non-independence, we dropped all users with clustering and reestimated the models. Table 9 show that the results are robust and support our findings. We also repeated our analysis with monthly fixed effects to control for differences in the virality of rumor cascades due to different start dates (Table 9 ). All results confirm the findings from our main analysis. Sign. levels: * < 0.05, * * < 0.01, * * * < 0.001; standard errors in parentheses

Since self-conscious emotions can be regarded as the counterpart of other-condemning emotions, we tested an alternative model specification in which we included the difference between the emotion scores for other-condemning and self-conscious emotions instead of two individual variables (Table 10) . Consistent with our main analysis, we found that false rumors are more viral than the truth if the source tweet embeds a high proportion of other-condemning emotion words, whereas a high proportion of self-conscious emotion words is linked to a less viral spread. Sign. levels: * < 0.05, * * < 0.01, * * * < 0.001; standard errors in parentheses

Polarization and public health: partisan differences in social distancing during the coronavirus pandemic

Social media and fake news in the 2016 election

Protecting elections from social media manipulation

Exposure to ideologically diverse news and opinion on Facebook

Tweeting from left to right: Is online political communication more than an echo chamber?

Coronavirus: World leaders' posts deleted over fake news

Science vs conspiracy: Collective narratives in the age of misinformation

Emotion shapes the diffusion of moralized content in social networks

Stata tip 87: Interpretation of interactions in nonlinear models

Information credibility on Twitter

Cris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil

Jinyoung Han, and Ted Taekyoung Kwon. 2020. Rumor propagation is amplified by echo chambers in social media

Connected: the surprising power of our social networks and how they shape our lives. Little

The echo chamber effect on social media

2020. The COVID-19 social media infodemic

It's always April fools' day! On the difficulty of social network misinformation classification via propagation features

Falling into the echo chamber: the Italian vaccination debate on Twitter

The anatomy of a scientific rumor

The spreading of misinformation online

Polarization and fake news

How affective polarization shapes Americans' political beliefs: a study of response to the COVID-19 pandemic

Explicit warnings reduce but do not eliminate the continued influence of misinformation

Dynamic spread of happiness in a large social network: longitudinal analysis over 20 years in the Framingham Heart Study

Surge of virus misinformation stumps Facebook and Twitter

Rumor cascades

Assessing the risks of 'infodemics' in response to COVID-19 epidemics

Fake news on Facebook and Twitter: investigating how people (don't) investigate

COVID-19 vaccine hesitancy in Canada: Content analysis of tweets using the theoretical domains framework

The moral emotions

2020. Covid19? Politicization and polarization in COVID-19 news coverage

Partisan public health: how does political ideology influence support for COVID-19 related misinformation

COVID-19-related infodemic and its impact on public health: A global social media analysis

Characterizing partisan political narrative frameworks about COVID-19 on Twitter

Says who? The effects of presentation format and source rating on fake news in social media

Coronavirus goes viral: quantifying the COVID-19 misinformation epidemic on

The science of fake news

Social media, news consumption, and polarization: Evidence from a field experiment

Building trust while influencing online COVID-19 content in the social media world

Crowdsourcing a word-emotion association lexicon

Fake news on social media: People believe what they want to believe when it makes no sense at all

Community intelligence and social media services: A rumor theoretic analysis of tweets during social crises

Prior exposure increases perceived accuracy of fake news

Shifting attention to accuracy can reduce misinformation online

Fighting COVID-19 misinformation on social media: Experimental evidence for a scalable accuracy-nudge intervention

Emotion: Theory, research, and experience

Community-based fact-checking on Twitter's Birdwatch platform

Emotions explain differences in the diffusion of true vs. false social media rumors

Emotions in online rumor diffusion

Can't we just disregard fake news? The consequences of exposure to inaccurate information

Using misinformation as a political weapon: COVID-19 and Bolsonaro in Brazil

The CAD triad hypothesis: a mapping between three moral emotions (contempt, anger, disgust) and three moral codes (community, autonomy, divinity)

Hoaxy: A platform for tracking online misinformation

A first look at COVID-19 information and misinformation sharing on Twitter

Hate speech in the political discourse on social media: disparities across parties, gender, and ethnicity

Examining the alternative media ecosystem through the production of alternative narratives of mass shooting events on Twitter

Rumors, false flags, and digital vigilantes: Misinformation on Twitter after the 2013 Boston marathon bombing

Emotions and information diffusion in social media: sentiment of microblogs and sharing behavior

Moral emotions and moral behavior

Putting the self into self-conscious emotions: a theoretical model

Social media, political polarization, and political disinformation: A review of the scientific literature

Radicalization and violent emotions

Individuals in movements: A social psychology of contention

The rise of guardians: Fact-checking url recommendation to combat fake news

The spread of true and false news online

The spread of medical fake news in social media-the pilot quantitative study

Virality prediction and community structure in social networks

Hypnotic disgust makes moral judgments more severe

Misinformation in social media: definition, manipulation, and detection

Weakly-supervised fine-grained event recognition on social media texts for disaster management

How to fight an infodemic

This study was supported by a grant from the German Research Foundation (DFG grant 455368471).