key: cord-0531751-5k13alop
authors: Ruffo, Giancarlo; Semeraro, Alfonso; Giachanou, Anastasia; Torino, Paolo Rosso Universita degli Studi di; University, Utrecht; Valencia, Universitat Politecnica de
title: Surveying the Research on Fake News in Social Media: a Tale of Networks and Language
date: 2021-09-13
journal: nan
DOI: nan
sha: 8f7239d7c2db5b84fe0c76bc2adbe5661db4e8e4
doc_id: 531751
cord_uid: 5k13alop

The history of journalism and news diffusion is tightly coupled with the effort to dispel hoaxes, misinformation, propaganda, unverified rumours, poor reporting, and messages containing hate and divisions. With the explosive growth of online social media and billions of individuals engaged with consuming, creating, and sharing news, this ancient problem has surfaced with a renewed intensity threatening our democracies, public health, and news outlets credibility. This has triggered many researchers to develop new methods for studying, understanding, detecting, and preventing fake-news diffusion; as a consequence, thousands of scientific papers have been published in a relatively short period, making researchers of different disciplines to struggle in search of open problems and most relevant trends. The aim of this survey is threefold: first, we want to provide the researchers interested in this multidisciplinary and challenging area with a network-based analysis of the existing literature to assist them with a visual exploration of papers that can be of interest; second, we present a selection of the main results achieved so far adopting the network as an unifying framework to represent and make sense of data, to model diffusion processes, and to evaluate different debunking strategies. Finally, we present an outline of the most relevant research trends focusing on the moving target of fake-news, bots, and trolls identification by means of data mining and text technologies; despite scholars working on computational linguistics and networks traditionally belong to different scientific communities, we expect that forthcoming computational approaches to prevent fake news from polluting the social media must be developed using hybrid and up-to-date methodologies.

Some of these aspects have been deeply analysed since then, even if some questions are still unanswered, and some others correspond to "moving targets" that are unlikely to reach stable solutions in the short term.

The first scientific question that is mentioned by Lazer et al. falls within the so called prevalence and impact problem: "How common is fake news, and what is its impact on individuals?". This is a call for more empirical evidence than just observing generically that we are overloaded by fake news, and that many of us share it, intentionally or not. Therefore, empirical analyses that provide some quantification of the problem of prevalence should be mentioned here: Alcott et al. [6] found that on average, an American encountered from 1 to 3 stories from fake news publishers before the 2016 elections. Vosoughi et al. [204] collected and analysed a dataset of ≈ 126, 000 stories tweeted by ≈ 3 million people more than 4.5 million times. They classified news as true or false using six independent fact-checking organisations that exhibited 95 to 98% agreement on the classifications. They found that false stories spread significantly farther, faster, deeper, and more broadly than the truth in all categories of information, and that among fake news, the effect was more pronounced for political stories than for news about terrorism, natural disasters, science, urban legends, or financial information. However, engagement with fake content can be very heterogeneous among users: Grinberg et al. [97] analysed 2016 US presidential elections noticing that only 1% of individuals accounted for 80% of fake news source exposures, and 0.1% accounted for nearly 80% of fake news sources shared. Due to these empirical findings, we assessed that the problem is real and that its impact on society is mainly negative and not negligible. The rest of the survey is organised to introduce, for each section, a relevant research problem, and how it has been addressed.

This survey is partially guided by data. Although we have a general understanding of the state of the art, that we have also contributed to on our own, we are also aware that in the last years the size of related literature has grown considerably, and that it is difficult to overview the state of the art objectively and exhaustively. Moreover, results have been published and discussed in journals and conferences that belong to different scientific communities, often distant from our areas of expertise. In this section, we describe the methodology we followed to select the contributions mentioned in this survey, and how a data driven analysis supported us to fill some gaps and spotting underlying narratives. The reader should notice that the analysis presented here is based on methodologies from network science and, partially, computational linguistics. Methods, definitions and basic concepts are reviewed in Appendix A.

The principal objective of our analysis was to create a corpus of all the recent contributions on fake-news, and related topics, plus the main references. Although the nature of such a dataset is subject to change over time, its purpose is to represent a comprehensive library of the existing literature on this topic. On top of this library on fake news research, we implemented some tools to explore the papers, follow citations, and execute some complex network analysis to identify clusters of the related graph, that could suggest the spontaneous emergence of underlying sub-narratives. Moreover, we wish to easily and objectively identify published contributions that should not be missed in a survey; in fact, such dataset can be represented by a directed graph, whose nodes are papers, and links are citations.

To create the "fake news research" library, we started from a sets of seed papers we were aware of. Then, we queried the Microsoft Academic Graph (MAG) APIs [145] to download all the papers, along with their metadata, whose titles contain the following keywords: fake news, disinformation, misinformation, hoax, false news, social bot, fact checking, infodemic, information disorder. Some stemming and lemmatization was applied to include most common variants: e.g., hoax and hoaxes, as well as fact checkers, fact checking, fact-checkers and fact-checking. This search yield to 17,158 papers, for any of which we kept this information: paper MAG id, title, authors, authors' institution, publication date, abstract, list of so called Field of Study (i.e., tags that the MAG engine extracts from the paper), list of references (only to papers in MAG databases), and abstract. We filtered out all the contributions in other languages than English, and we kept only papers published in journals or conference proceedings. We call this set of papers the library's core, since it contains contributions that focus explicitly on above mentioned key terms. Fig. 1a shows the number of these papers by year. This is a newborn field, yet it is a very trendy one: most of the thousands of papers published -most of the knowledge we have -is just 4 years old, or even less. Fig. 1b shows the cumulative number of papers published with one of the query keywords in the title. To properly comment the plot, there is an important caveat to address: the keyword "misinformation" is often related to the "misinformation effect" in psychology, i.e. how patients (mis)remember facts and how memories can be manipulated years later. Given that, while "misinformation" and "hoax" have a long history of scientific publications, but maybe not directly related to disinformation as we study it today, the other keywords have become popular in the last years, especially "fake news", which is born around 2016. Also, this research field is an extremely multidisciplinary one: MAG automatically extracts from the papers a list of tags, namely Field of Study (Fos); FoS are organized in a 5-level hierarchical acyclic directed graph, where each FoS may have parents FoS and children FoS. Level-5 FoS are specific (a) Yearly percentages of papers w.r.t. the size of the collection itself. Although the trend is growing since 2000s, it is after 2016 that the scientific production really spiked.

(b) Cumulative number of papers by year and by keyword. "Misinformation" and "hoax", related not only to fake news, have a longer history of research. The other keywords have came to huge scientific attention strictly after 2016.

(c) Percentages of papers per level-0 Field of Study to the total library's size. Political and computer sciences are the disciplines most interested with the fake news problem. Psychology is the third most represented discipline in our library, but this result is largely due to the polysemy of the word "misinformation", that is also related to a cognitive effect that leads to inaccurate memories. topic, techniques or keywords related to some specific problem; level-0 FoS are the most general FoS, and they roughly define disciplines more than topics. In Fig. 1c , we reported the distribution of level-0 FoS. This field of research is dominated by contributions that suggest how the spread of misinformation in social media can have a big impact on democracies, elections, and government agendas, so its not surprising that political and computer science are the predominant disciplines. Also psycho and social scholars are heavily engaged in this field of research, even if we have to consider that this is likely due to the double sense of "misinformation". The other most involved disciplines are history, medicine, art and business, and so on, as a demonstration that disinformation is a complex problem, studied from several perspectives.

Assuming the presence of topicality in the underlying citation graph, as well as it happens in other information networks [210] , we followed all the references from papers in the core library, establishing directed links between papers, and we also downloaded all the new papers that were not retrieved after the first round of queries to MAG APIs. This is equivalent to run one step of a snowball sampling from the library core, and it has two immediate consequences: we have a greater chance to find other relevant papers that we skipped with searching only by means of key words, and then to include in our library also some fundamental papers of interest for whom is planning to do research on fake news, even if the presented contribution is not directly related to this topic. For example, this can happen for methodological papers, or for other "classics" that can be only loosely connected to disinformation, or that pioneered this field when it was not a trendy topic at all. We call this new set of 54,572 papers the "library's periphery". The entire library counts 71,730 papers (nodes of the graph), with a total of 1,790,608 references (links) between them.

Of course, this collection may include some miscast paper, but we decided to keep any contribution: the "fake news research" library is intended as an automatically searchable scientific repository, and designed to be self-updating: a Table 1 : Most frequent level-0 and level-1 FoS in each of the eight biggest clusters found in the core library network. Colour code as in Fig. 2 .

stricter topic-based filter would have possibly cancelled some interesting contribution. Otherwise, it is relatively easy for a scholar to understand if a contribution is off-topic just from the title and the abstract. We then use Complex Network Analysis (see Appendix A.1 for a quick review) to better understand the structure of the network representing the "Fake News Research" library, to find clusters, and to identify central nodes. The goal is to make sense of this analysis, to partially support the selection of contributions we include in the present survey. First of all, we observe that the core library network is made of hundreds of very small disconnected components, plus the giant component whose size is the ≈ 89% of the graph. In Fig. 2 we visualise the giant component that emerges from our core library only. We run the Louvain community detection algorithm [26] to identify tightly-knit groups of papers, to let emerge a coarse grained characterisation of the underlying sub narratives. In Table 1 , we describe high level differences between these clusters, in terms of their papers' most frequent "Field of Study" tags.

The structure of the core library networks (Fig. 2 ) and the papers' fields of study (Table 1) suggest that we have a central sub-graph of highly multidisciplinary contributions, that is composed of clusters 2, 3, 4, 5 and 6, whose predominant disciplines are political science, psychology and computer science, with a keen interest toward internet privacy; also, on the "bottom-right" of the giant component, we find cluster 1, that includes mainly more technical contributions: papers in this group are likely to present novel algorithms and approaches to detect fake news automatically, using machine learning models as well as computational linguistics approaches, as revealed also from the most frequent words in titles and abstracts after 'fake' and 'news', e.g., 'detection', 'model', 'information', 'feature', and 'learning', among the others. Finally, at the "leftmost" part of the graph drawn in Fig. 2 , we observe that cluster 7 is only loosely connected to the giant component: apparently, the majority of its papers present contributions that are of interest especially for psychologists; in fact, a finer grained analysis of this cluster shows that 'misinformation effect' and 'memory' are the most frequent key terms in papers' titles and abstracts, supporting with evidence the hypothesis mentioned above in this section, i.e., there is a preexisting trend of research on 'misinformation' that is only loosely related to the current interest for 'fake news' spreading in social media. Quite interestingly, cluster 7 is more connected to cluster 2 than to other groups of papers. As a matter of fact, cluster 2 is the second largest community in our core library network, and its papers are more related to psychological effects that can explain propensity of people to believe to fake news. These cognitive biases are widely mentioned in many recent contributions, even if their main focus is not on psychological effects. Given the centrality of these researches also in more technical papers, we review briefly some of the most cited cognitive biases in Sec. 4. Also, we have a plethora (hundreds) of smaller clusters, e.g., the eighth largest cluster accounts for only 1% of the papers in the library core, but it is strikingly loosely connected to the rest of the library, at the bottom and rightmost side of Fig. 2 , with only few citations from and to papers in cluster 1. This is just an example that many contributions are neglected or hidden from the rest of the scientific communities, and their impact is unfortunately limited.

Another interesting thing that can be observed in Fig. 2 is that we can easily spot some hubs (i.e., nodes with high in-degree) in clusters 1, 2, 3, and 5 by eye. Let us recall here that a reference is usually an endorsement to someone else's work, hence, highly cited papers are publications to which many other researchers recognise value. Therefore, we want to exploit centrality measures to find contributions that are highly cited (in-degree centrality), that are likely to bridge different clusters and disciplines (betweenness), or that are proven to be influential in the long term (page rank). The reader can find the detailed lists of central papers in Appendix B, with comments and remarks on the reasons that led us to exclude from this survey some contributions that are objectively relevant for this field. As already discussed in the introduction, we decided to focus on a dual network science and computational linguistics perspective, partially because there are already some excellent overviews of existing proposals to automatically detect fake-news (e.g., [59] ). However, this is not the only reason for such a choice: it is worth noticing that an indisputable fraction of papers that attracted a high attention so far, are characterised by results of multidisciplinary interest (from political to computer science, from cognitive psychology to social science), and by the adoption of both empirical and theoretical methodologies that represent data and systems as networks, and that exploit computational linguistics techniques to make sense of textual content.

Finally, we must acknowledge the limitations of a data driven approach to find "hidden gems" in scientific literature, and to spot predominant narratives; in fact, even if MAG has a good coverage on scientific disciplines, it does not include the 100% of published papers [110, 162] . Many relevant papers may have been excluded and many references may be missing, and both impact not only on the papers repository coverage, but also on the network structure and therefore on centrality measures. Also, we have our own biases, and although a data driven approach supported us to give relevance also to contributions that can be far from our scientific interest and expertise, the selection of papers to overview is still the result of a personal, subjective choice of topics and publications. Therefore, in order to allow everyone to perform their own queries to the "Fake News Research" library, we decided to make our search tool accessible through the Web at http://fakenewsresearch.net. We meant not only to release the dataset we employed, as in this ever-changing field this dataset will age-old soon, but a long-lasting service, an always up-todate publication repository about disinformation, misinformation and other related problems. The user can query the database searching papers by citation count, date, page rank, hub or authority scores, and betweenness, and also by author, institution or field of study (as provided by MAG APIs). Plus, we provide a basic recommendation system that suggests, for each paper, the top 10 most similar publications, where similarity is simply computed on the abstracts with Doc2Vec [128] .

The term "fake news" has become extremely popular in the last years, trespassing the domain of journalism and social media users and entering the everyday language. Intuitively, it reminds of some kind of false information broadcasted by one or more news outlets, and generally speaking to the manipulation of the information systems in order to deceive someone's opinion. This pseudo-definition is however broad and unclear, as it contains several shades of disinformation practices, while at the same time it excludes bad practices that indeed contribute to the diffusion of misleading news. In the previous sections, we mentioned terms such as "fake news", "misinformation", "disinformation": the purpose of this section is to report the state of the art on the debate on the rigorous definitions of these terms.

Especially after the nomination as 2017 "Word of the year" by the Collin's Dictionary, "fake news" has been widely used for several, often very different, bad practices. This kind of confusion is so common that some scholars tried to put order. In 2015, Rubin et al. [178] discussed of three kind of fakes: a) serious fabrications (uncovered in mainstream or participant media, yellow press or tabloids); b) large-scale hoaxes; c) humorous fakes (news satire, parody, game shows). Later, Tandoc et al. [116] , collected 34 previous works about "fake news", only to find out that they were targeting six distinct practices: satire, news parody, fabricated news, manipulated pictures, advertising, and political propaganda. When different kinds of manipulation conflate in the term "fake news" without a global agreement on what it actually means, and when researchers target and study as "fake news" quite different phenomena, it is hard to agree on findings, and to verify the results with scientific rigour. Such a buzzword needs to be replaced by a proper terminology, a terminology that solves two problems. First, false information online may come in different fashions, and one term could not include them all. Second, definitions must be exact. If we want to compare the answers, we must ask the same questions. For instance, the Merriam-Webster Dictionary, while explaining the decision not to include "fake news" in the Dictionary, gives it a short definition [57] :

Fake news is, quite simply, news ("material reported in a newspaper or news periodical or on a newscast") that is fake ("false, counterfeit").

However, this simple definition raises many questions, as argued by Mould [151] : And to what extent a news must be "fake" to be considered a "fake news"? Fabricated, non-factual news are certainly fake news, but what about slight manipulations of partially true news? Does the omission of a crucial detail make the news fake? And what about verified news framed in biased narratives? Tandoc et al. [116] is also an important entry point for getting to the core of the "fake news" problem, as it introduces a possible taxonomy of problems distinct by two dimensions: facticity, i.e. adherence on factuality of a news, and intentionality, i.e. if the news is written with the intent of deceive the reader. The accent on the intention of the writer seems to be an agreed factor, when defining "fake news". Allcott et al. [6] , also insists on the publisher's intentions:

We define "fake news" to be news articles that are intentionally and verifiably false, and could mislead readers.

Another similar definition is given by Gelfert [80] , with the specification that "fake news" have the goal to mislead "by design":

This paper argues that it should be reserved for cases of deliberate presentation of (typically) false or misleading claims as news, where these are misleading by design. The phrase 'by design' here refers to systemic features of the design of the sources and channels by which fake news propagates and, thereby, manipulates the audience's cognitive processes.

The previous considerations narrow the domain of "fake news" to false information shared with harmful purpose, which to the best of our knowledge is accepted as a good definition, because it includes both the veracity of the news, and the malicious intention of the author. However, there is no doubt that this is only part of a bigger problem. Some of the above questions remain unanswered, as partially false or re-framed pieces of information may have the same impact in the end that entirely fabricated contents: how to consider them? In addition, in the last years we observed the rise of many issues related to the dissemination of fake news online, like coordinated disinformation campaigns held by automated accounts, constellation of false information supporting conspiracies, propagation of unverified rumours.

For the reasons stated before, the popular term "fake news" is now considered insufficient to represent a broader range of malicious activities. A 2017 report by the Council of Europe [159] first, and a 2018 UNESCO handbook [197] later, found a more comprehensive definition in information disorder, a term which in turn encompasses misinformation, disinformation, malinformation as three distinct problems.

The rationale behind this taxonomy is a two-dimensional evaluation of a piece of information, similar to the one in Tandoc et al. [116] : the veracity of the news and the malicious intentions of the author. Misinformation is information false or misleading, but shared without harmful purposes. All the false news published with bona fide mistakes are examples of misinformation, as well as news in which the title or a captivating picture are not adequately supported by the body of the article, like click-baiting. Malinformation is (partially) true, verifiable information, but shared with malicious intent. Articles characterised by hate speech belong to this category, like publishers cherry-picking real news in order to foster hate against some minorities. Disinformation is, finally, an intersection of the two above: a false information shared with harmful intent. Impostor content, like information falsely credited to authoritative newspapers, falls in this category, as well as: manipulated content, such true facts distorted to serve a false narration; fabricated content, i.e., proper fake news in the definition of Allcott et al. [6] ; and true news framed in a narrative they do not belong (in this case it is the context to be false, rather than the content). Table 2 shows an attempt to define some key terms as disinformation/misinformation/malinformation, to give a general overview of the information disorder categories and main practices, including the key terms proposed in [213] . Of course, the list of terms in Table 2 is by no means exhaustive. We also included a peculiar form of deceptive news (satire) which is part of the spectrum of the possible categories of news, also according to [116] . misinformation unintentionally spread false information deceiving its recipients malinformation information that is partially or totally true, but spread with malicious intent disinformation intentionally spread and/or fabricated misinformation fake news disinformation in the format of news hoax disinformation that can have also humorous purposes rumour information that can be true and accurate, but still unverified; if it is falsified, it becomes misinformation conspiracy theory explanation of an event that assume a conspiracy by powerful group; a theory can make use of fake news, rumours as well as true information. urban legends kind of folklore made of rumours characterised by supernatural, horrifying or humorous elements infodemic mixture of misinformation and true information about the origins and alternative cures of a disease; expecially observed during COVID-19 pandemic propaganda malinformation that aims to influence an audience and a political agenda click-bait misinformation based strategy to deceive Web users and enticing them to follow a link cherry-picking malinformation practice that selects only the most beneficial information to the author's argument from what is available. hate speech abusive malinformation that targets certain groups of people, expressing prejudice and threatening cyberbullying form of bullying that uses electronic communication, usually social media, that can contain misinformation, rumour and hate speech troll social media user that uses disinformation to increase the tension between different ideas astroturf disinformation practice of orchestrating a campaign masking its supporters and sponsors as grass-roots participants crowdturf crowdsourced astroturf spam unsolicited information that overloads its recipients social bot a social media user controlled by a software that mimic human behaviour; often used a tool for spamming, spreading misinformation, and astroturfing satire false information but intentionally harmless in the majority of cases, even if often it has strong political references [126] and can be misused as a propaganda practice Table 2 : A list of terms referring to different key aspects characterising information disorder.

Other important contributions to the debate on terminology can be found in Lewandowsky et al. [133] , Edson et al. [117] , Fallis [64] , Floridi [71] , Stahl [189] , Fetzer [68] , Berghel [21] . The interested reader can check out also fakenewsresearch.net to explore the paper's dataset presented in Sec. 2.

4 Psychology of Fake-News: the problem of cognitive biases

In Sec. 2 we presented the results of the network analysis of the citation graph underlying our "Fake News Research" library, a collection of scientific papers on fake-news, disinformation, and related topics. We found that psychology is apparently the discipline that studied misinformation the most, even if this is partially due to the polysemy of the word 'misinformation' itself, that also refers to a well studied cognitive effect that occurs to persons that alter the memory of an event with information acquired successively. The so called post-event misinformation effect has been studied for decades, also due to its implications on the accuracy of eyewitness testimony(e.g., see Loftus et al. in 1978 [136] , McCloskey and Zaragoza in 1985 [142] ), and should not be confused with the recent interest to the spread of fake-news on social media. However, we also observed that there are clusters of papers that focus on the psychological reasons behind our tendency to share misinformation, even if the fake content has already been debunked; in fact, there are many cognitive biases that usually drive our judgement for the best, but that can mislead us when exposed to deceptive news. Therefore, in this section we mention the most important psychological effects that are used to give tentative explanations to reported empirical observations on how and why fake news spread in social networks.

Why do users fall into fake news in the first place? Researchers have enquired the relationship between users and false news online, which keep to be attractive even when blatantly false. Disinformation works through a series of cognitive hooks that make the information appealing, triggering a psychological reaction in the reader. First of all, fake news are attractive. They usually contain more novelty than regular news [204] , something that is captivating for a distract reader. Plus, they deliver an information simpler to process [109] compared to regular news, and a simple message that can be easily digested even when a person is paying low attention. Also, fake news often trigger an emotional response [208, 13, 190, 204] , proven to be one driver for online virality. Readers prefer high arousal news [13] , which catches more attention than regular news and sticks easily on mind.

Another factor that determines the success of disinformation is the repeated exposure to the same fake news, as argued by Pennycook et al. [165] , because repetition increases the content perceived accuracy.

Repetition is a well known propaganda mechanism, and it is exploited by social media filtering algorithms that decide what content deliver to whom, by indulging the users' tastes, eventually building a reinforcement feedback loop on users walls. This makes appealing content even more visible, as long as the user interacts with them, and conversely it may hinder content that the user dislikes. When applied to political issues, for instance, this algorithmic bias easily produces a filter bubble [161] , a content selection that the social media platforms may apply on one user's online feed. Additionally, when algorithms highlight the content shared by a user's friend according to their opinions, this effect may grow stronger, triggering the emergence of echo chambers [192] ; for a deeper discussion on echo chambers and polarisation dynamics, see Section 5. Disinformation may grow on the user's mind, as repetition has been proven as one of the most effective techniques of persuasion for online news [198] , a phenomenon also known as validity effect, or illusory truth effect [104] .

Repeated exposure to others' behaviours is also a key element of the so called 'complex contagion' effect (see Appendix A.2). In other words, if a user is exposed repeatedly to an opinion (or a behaviour, an innovation, and so on), the probability of adopting it is higher. The hypothesis that information and diseases spread as 'simple contagions', and that behaviours require multiple interaction with the source was proposed by Centola and Macy in [37] . Also, a behaviour spreads faster in clustered network because of social reinforcement, as proven by Centola with his experiment on a controlled social system [36] .

Other reasons to engage with low-quality, partisan content often root in social behaviour: the bandwagon effect [154] , for instance, is the tendency to align to the opinion of what is perceived as a big mass of other individuals; our perception of what is right and what is wrong may depend on the social group we think we belong: in this mindset, the group's opinion is assumed to be the public opinion, whereas discordant news are perceived as hostile attacks towards the public. The normative social influence theory [55] hypothesises that people conforms to the behaviours and ideas of the social group they belong to, while the influence of social identity has been showed to play an important role in news perception [183] . These misperceptions are then backed up by other cognitive biases inherited by our social behaviour in case of us-against-them dynamics. It is the case of the third-person effect [49] , the tendency to believe that disinformation affects more the other group than the one the individual belongs. These effects are often associated with what is called naive realism [177] , the tendency of an individual to assume that the reality they perceive is objective and factual: if applied on news and the perception of the world, it creates a false consensus effect [176] , a tendency for individuals to think their beliefs to be widely accepted; under this perspective, adverse news by hostile news outlets can be classified as a malicious attempt to undermine a widespread acknowledged fact.

All of these biases are reflected and also amplified in the structure of the social network the users belong to; in fact, ego-networks, i.e. the persons connected to single individual (the "ego"), are not unbiased, and they do not cover the full spectrum of available beliefs, ideas, opinions. Homophily in networks (see McPherson et al. for a review on this topic [143] ) is a well documented tendency of users to be friends of similar persons: socio-demographic indicators are usually very homogeneous in the list of acquaintances of an individual, and this may favour an homogeneity of opinions toward economics, politics and society. This is a key factor for the formation of clusters and echo-chambers in social networks, as we will discuss in Sec. 5.

Finally, there are many biases working as reinforcement feedbacks for one individual's beliefs, in a loop that may strengthen their misperception. In fact, reinforcement and repetition of a news are not only the result of algorithmic biases, echo chambers, and clustered social networks, but also a self-inflicted deception. People tend to notice one content if they are already thinking of it, a mechanism known as attentional bias [212] : if we convince ourselves of a shocking, disturbing fact, it is possible we will start noticing that fact often. The highly mentioned confirmation bias [156] also plays an important role in news selection, as users tend to focus on information that confirm prior beliefs, instead of news that challenges them. Confirmation bias is often found in literature after the name of selective exposure [72] ; in fact, people firmly rely on news that agree with them, but rarely seek for disproofs of their beliefs, a bias known as congruence bias [207] . Some headlines can be perceived stronger despite the weak arguments that support the conclusion in an article, if the conclusion resonates with one's expectations, as effect of the belief bias [131] . Also, as mentioned in Sec. 4.1, disinformation often is conveyed through articles with a high emotional valence. The emotional bias [20] may distort the perception of an argumentation, for instance bringing one person to refute to acknowledge facts that upset them. Ferrara and Yang proposed an analysis about "emotional contagion" with Twitter data to find that individuals overexposed to negative content are likely to share negative messages, and vice-versa. In fact, emotional content can be exploited by malicious agents to provoke inflammatory responses and exacerbating polarisation in on line debates, as observed by Stella et al. on Twitter during the Catalan independence referendum campaign [190] .

Misinformation correction and debunking are difficult for many different reasons. Generally speaking, individuals tend to overstate their knowledge about some topics, and to perceive their partial information as highly accurate, two biases named illusion of asymmetric insight [171] and overconfidence effect [160] , which may lead one person to think they are undoubtedly right. Correction of unsubstantiated beliefs about politics often fail when individuals with strong ideological opinions are corrected, as experimentally analysed by Nyhan and Reifler in [158] . They observed also the so called backfire effect, i.e., correction may actually increase misperception and ideological biases. Quite interestingly, this publication is one of the most cited in our "Fake News Research" library periphery (see Sec. 2 and Appendix B), suggesting its importance as a pioneering contribution in this field.

Apparently debunking could be supported by the hyper-correction effect, i.e., when an individual commits a mistake with high confidence and receives a corrective feedback, the same individual is more likely to remember the right information. However, Butler et al. [32] observed that the hyper-correction effect seems to vanish after one week, and that misperceptions with higher confidence are more likely to return after a given period of time.

Lewandosky et al. [134] review many cognitive biases and effects that make misinformation correction so difficult, suggesting a list of guidelines for debunkers, educators, journalists, science communicators, and health professionals. However, the continued influence of misinformation, that makes people rely on information that is already retracted or falsified, is still a crucial and controversial point for designing proper correction narratives: a common strategy is to avoid repeating misinformation during correction, because it may strengthen the effectiveness of false information. However, Ecker et al. in 2017 [62] found that retractions that explicitly mentioned the misinformation were more effective than corrections that avoided to repeat the false information. The analogies and differences between the continued influence effect and the post-event misinformation effect (mentioned at the beginning of this section) have been studied by Ecker et al. in 2015 [63] , establishing a link between two important traditions of causal explanations.

5 The problem of the interplay between network structure, opinion dynamics, and the spread of misinformation

The relationship between fake news spreading and echo-chambers formation is one of the most studied aspect of this new born scientific field; in fact, misinformation's prevalence in social media is often attributed to the so called "echo chambers" on social networks, tightly knit clusters of users that keep interacting with one-sided content until they get radicalised, as a result of a reinforcing feedback loop. Studying such relationship is particularly interesting because of a non linear characterisation of the corresponding causality. Trivially speaking, homophily triggers social network clustering, that amplifies opinion polarisation, leading to the emergence of echo-chambers. However, we can also observe a process following a reversed direction: echo-chambers amplifies polarisation, that increases network clustering, that leads to higher homophily. Co-evolution of network structure and dynamics is also exacerbated by filter bubbles and social media business models. Misinformation has been observed to circulate consistently within echo-chambers [24, 53] , so studying network dynamics (see Appendix A.2) is fundamental to understand the key factors that determine information spreading. In this section, we focus on opinion dynamics to better contextualise the contribution on misinformation and echo chambers.

Since echo-chambers are well connected to the concept of polarisation, we need a definition. As stated in Dimaggio et al. [58] :

Polarization is both a state and a process. Polarization as a state refers to the extent to which opinions on an issue are opposed in relation to some theoretical maximum. Polarization as a process refers to the increase in such opposition over time.

Henceforth, we will refer to opinions as states of the users (or the nodes of a social network), and polarisation as mainly the process that leads to an irredeemable divide, i.e., an equilibrium point or a stationary state of the system, between two groups, each representing a one sided view over a topic. An illustrative example of such a process is given in Fig. 3 , where we show four possible states of a social network, the first representing a well mixed society where different opinions do not correspond to a structural division, to an extreme polarisation whose outcome is a graph made of two disconnected components. Models explaining the emergence of consensus or polarisation by means of opinion dynamics are reviewed in Sec. 5.1.

As stated above, there is a widely documented empirical observation that the spreading of misinformation is amplified by echo-chambers; however, fake-news circulate widely even in networks without a clear structural polarisation. In the sub sections below, we will give references to the debate that is still going on, pointing out some research questions that remain unanswered. A review of the many empirical findings on well known information networks are accounted in Sec. 5.2.

(a) well mixed network, with no clustering (b) high homophily may coexist with a cohesive society (c) higher separation leads to polarisation and echo-chambers (d) extreme polarisation can be irremediably divisive Figure 3 : Illustrative example of the structural evolution of an artificially generated network of 1000 nodes, each belonging to a different class (e.g., exposing two different opinions, or adopting non compatible innovations). The network, artificially generated with a stochastic block model, keeps the same internal groups cohesion, but with a progressively higher separation between the two classes of nodes, until no cross links are left, and the graph is finally divided in two components.

Social dynamics models have been extensively used to explain how individuals engage with opinions of others (see Lorenz [137] , Castellano et al., [34] , Dong et al. [59] for more detailed surveys on this topic). In some of these models, individuals are entitled of an opinion, but they also are connected with each other, and they regularly update their state depending (to some extent) on the opinions of users they are linked with, interpreting the concept of peer pressure, and the way social influence may amplify homophily in a social network. This simple yet powerful idea may return good explanations of real social networks dynamics, which may be hard to unveil otherwise. In fact, social networks work for an user as proxies of opinions of their neighbours, and contagion of ideas is a powerful effect on individuals' behaviour (see Appendix A.2 for a quick overview on social contagion in networks). Plus, these models offer a general framework that can be modified to embed many of the biases that social media platforms induce in the process of opinion diffusion: as we noted in Sec. 4, online users appraise information filtered by algorithmic biases, and by other widely studied social and cognitive biases that create additional pressures to the individual. Hence, the purpose of this models' review presented here is threefold: (1) to give to scholars studying misinformation a general overview of models well known in some scientific fields (e.g., social science, political science, statistical physics) that may be less popular to researchers active in other disciplines; (2) to propose a general and unique formalising framework to take into account and compare the many existing contributions; (3) to identify under which constraints opinions polarisation may eventually emerge in social networks.

In the following, we will not define opinions as discrete values, because we endorse a view of opinions as a continuum.

After an initial configuration where each agent is assigned with a value, the system will eventually reach a steady state, where no nodes will change their opinion anymore. Such final state can represent a consensus, where every nodes will share the same opinion, a polarisation, where a group of nodes will share one opinion, and the rest will share the opposite opinion. Another typical stationary state is fragmentation, when opinions are concentrated around more than two values.

We start with a very simple model from French-Harary-DeGroot [75, 103, 52] . In this model, each agent in a network of n agents is assigned with an initial opinion, a scalar x ∈ [0, 1], which represents their degree of support at time step t for the position represented by 1. At each time step t (time is assumed to be discrete), each agent i updates its opinion as a linear combination of its neighbours opinions, as follows:

where α ij is a weight on the link between agents i and j. The stronger is the relationship between them, the higher is the weight. Lets observe that if i and j are not connected, then α ij = 0. By averaging each users' opinion with their neighbour's, i.e., posing α = 1 n , this model eventually reaches consensus. In such a model no polarisation can emerge yet, even with a starting setting with arbitrary homophily, as demonstrated in Dandekar et al. [48] . However, users do not process information coming from other users equally. It should be noticed that these dynamics can be studied also with the so called "majority model", introduced in 1963 in statistical physics by Glauber [94] in the context of spin models.

Weights α ij may be a function of the strength of a social tie, or even a function of similarity of two opinions. In such case, the model would embed a confirmation bias or selective exposure (see Sec. 4.4): by weighting more neighbours' opinion when they are close to their own opinion, agents would consider only ideas they agree with. The tendency to discredit opposite views may be also bounded with a confidence value, as suggested in the theory of social comparison by Festinger [67] : in the Bounded Confidence Model (BCM) introduced by Deffuant et al. [51] , each individual interacts exclusively with "close enough" opinions, i.e., i evaluates j's opinion only if |x i (t) − x j (t)| < . In such scenario, is a representation of mind broadness of the agent, or also the quantification of the agent's confirmation bias. Discordant opinions may be neglected, or used as a negative reinforcement. For instance, in Hegselmann et al. [106] authors evaluate BCMs where an agent i averages its opinion with concordant agents j. We can simply check concordance by means of a Kronecker delta:

so that opinions are updated again using eq. 1, where α ij = 0 if δ ij = 0, and α ij = ( n z=1 δ iz ) −1 if δ ij = 1 (i.e., the weight is set to a value inversely proportional to the total number of agents whose opinion is concordant with both i and j). In such model, reaching a consensus is difficult, but convergence towards fragmentation or polarisation depend on : small confidence intervals mean narrow-minded agents, which rarely accept other ideas, thus more likely to stick to their initial position; larger confidence intervals means that agents adopt ideas from others more often, resulting in a reduced numbers of surviving opinions: polarisation may arise. Authors note, for instance, that with an = 0.01 up to 38 opinions survived in the end, while with = 0.15 the system converged on two poles.

The overconfidence effect, i.e., the tendency of individuals to undervalue other people's ideas, and to update their opinion with hesitancy even when confronted with new evidences, is properly considered by Friedkin and Johnsen [77, 76] as a generalisation of the basic DeGroot model: it incorporates a resistance in each agent's update rule. At each time step t, an user i modifies their opinion as follows:

where g i is a susceptibility factor: the higher g is, the more an agent sticks to their initial opinion. Let's observe that the original French-Harary-DeGroot model in eq.1 is a case of Friedkin and Johnsen's model where g i = 0. This model also leads to disagreement among agents. In a similar fashion, Dandekar et al. [48] extend the classic DeGroot model as follows:

individual i weights each neighbour j's opinion x j (t) by a factor (x i (t)) bi and weights the opposing view (1 − x j (t)) by a factor (1 − x i (t)) bi , where b i ≥ 0 is a bias parameter. Informally, b i represents the bias with which i assimilates his neighbours opinions. When b i = 0, our model reduces to DeGroot's, and corresponds to unbiased assimilation.

Authors confirmed with their simulations that basic French-Harary-DeGroot model is not polarising, but this model is, due to the biased assimilation of other's ideas.

Confirmation and overconfidence biases are not the only cognitive effects that we can consider to explain the emergence of polarisation. In particular, limited attention [95] can have an impact on the users' perception of online news. All the previous models assume that each user would evaluate every neighbour's ideas at each time t. This assumption may not hold in real social networks: each user may evaluate only a fraction of the available ideas among their friends, as the attention that online users offer to news content is actually limited [209] , with repercussions on what information is eventually absorbed [132] (see Sec. 4.1). Hence, this theoretical framework can be adapted to get even closer to a real social network, in order to account not only for users' biases, but also for automatically selected information or other platform induced biases. In fact, there are externalities that can impact on opinions diffusion in real social networks, as broken or rewired links, or content sorted and filtered according the strategies of underlying personalisation algorithms. Friending and unfriending users, i.e., establishing or revoking a connection with another user, is a factor that can contribute to polarisation, too. Sasahara et al. [179] define a network of agents, each of them with a limited screen that shows the most recent messages posted (or reposted) by their neighbours. These messages convey an opinion with value x ∈ [−1, 1]. Opinions are defined as concordant or discordant depending on a bounded confidence interval, as in the BCM. At each time step t, an agent i sees only the first l messages on the screen. The neighbours that posted or re-posted these messages are listed in set N l i The opinion of user i then changes based on the concordant messages on the screen:

where µ is an influence strength parameter (a measure similar to 1−g i ). Also, with probability q, each agent unfollows a discordant neighbour j and establish a new connection with a new user. Authors try different strategies, like friending random users, concordant users not currently among friends, or a random user among the originator of a re-post on i's screen (opinions are modelled as explicit messages to be posted or re-posted). Eventually, this system converges to a polarised setting, independently on the rewiring strategy employed. Also, unfollowing accelerates the convergence of the polarisation process, and the formation of two distinct clusters.

Polarisation is not necessarily a consequence of structural clustering: in Fig. 3 (a) we showed an example of a not clustered network where nodes have two opposite views. However, we know that the presence of clusters make the diffusion of new ideas very difficult [150] . Hence, if a fake-news is very common within a very cohesive cluster, then its debunking may be very difficult, and sometimes it could radicalise beliefs due to the back-fire effect. This is why echo chambers, as a peculiar instance of polarisation, are considered particularly dangerous by many. Studying the co-evolution of networks and opinions is one of the most important points in the scholars' agendas. The first models considering co-evolution are due to Holme and Newman [107] and Gil and Zanette [91] , Gross et al. [98] and Iñiguez et al. [111] . Effects of rewiring in a network may be heavy, even when the tendency to break links with discordant opinions is mild. Schelling's model of segregation [182] shows that even when an agent allows one third of its neighbours to be discordant, still the system converges on a segregated population, i.e. homogeneous neighbourhoods.

Del Vicario et al. [200] proposed a mix of a peculiar bounded confidence model and a rewire strategy. They define an Unbounded Confidence Model (UCM), i.e., a BCM where discordant opinions are also taken into account but with opposite values. Agent i updates its status following j's opinion as:

with µ being the influence factor as defined above, and ρ being a correction factor for extreme opinions. Then, both strategies are paired with a rewiring mechanism, yielding other two variants (RBCM and RUCM), where discordant links are broken and random new links are established. Given non-zero and µ parameters, all the four variants of this model lead eventually to a bi-modal distribution of opinions, with two distinct chambers in the network.

Finally, users' choices about which opinion to acquire and neighbours to unfriend are ultimately determined by the algorithms that populate their news feeds. Simple models that mimic a feed algorithm that maximises user interaction can also successfully explain the rise of echo chambers. In Perra and Rocha [167] , authors propose a model of opinion dynamics that takes into account, among other factors, an algorithmic personalisation of each user's feed. This model shows some differences w.r.t models discussed above, like a binary A-B opinion, and a network of users that are not active at each step t, meaning that they share ideas and change their opinions only occasionally. Authors present several scenarios where opinions from neighbours to user i are arranged in a queue, sorted with different strategies (LIFO, FIFO, random, according to the similarity to i's ideas, or with an opinion preferentially nudged by the system) and limited to the first N entries, simulating an attention limit from user i. Simulations clearly show that algorithmic personalisation of content can create a fragmentation where each group of neighbours have a dominant idea, isolating the subordinate ones, when the underlying network is clustered enough and the initial distribution of ideas uneven.

In previous section, we reviewed several attempts to explain the emergence of polarisation in social network, also in terms of some of the socio-psychological biases mentioned in Sec. 4, as well as newly effects introduced by social media platforms that contribute to change the structure of the underlying social network, and to inject in the system ideas that influence agents in different ways. However, modern social media gives data to validate many hypotheses and conjectures.

The tendency of online users to only engage with other users and select news that are in agreement with prior beliefs, while disregarding information that challenges them, is well documented. Conservatives versus liberals [42, 41] , pro-Euro versus anti-Euro [96] , or pro-vax versus anti-vax [44] , are a few examples of the many divisive topics that fuel heated debates on online social media. It happens on Facebook [14] , Twitter [41] , Youtube [25] , Reddit and Gab [40] . It happens also on blogs [2] , and even email recipients appear divided in two stances [157] on political topics. Some topics are divisive by nature and ultimately require each observer to take one of two possible sides. Politics is the main apple of discord [96] : : it is the case, for instance, of the polarisation in the American politics in which both voters [42, 69] and candidates [9, 196] are getting more and more polarised. In Bakshy et al. [14] authors determined the political alignment of 10 million Facebook users, and found that neighbours of liberal users are way more likely to be liberals than neutral or conservative, and the opposite holds for conservatives. Similar findings are displayed in Colleoni et al. [41] , where authors report that Democrats more than Republicans are more likely to belong to an ideologically enclosed network, on average, but the highest homophily is due to highly partisan Republicans. Same findings are also supported by Boutyline et al. [28] . Adamic et al. [2] shows that even conservative blogs are more likely to reference other conservative blogs, compared to liberal blogs. However, opposite communities have been found between science and conspiracies supporters [24, 25] , or between users debating about climate policies [211] .

Polarisation is not only measured in terms of homophily of users' neighbours, but also in terms of interaction with other users online. Barbera et al. [18] estimated the political leaning of 3.8 million Twitter users that produced over 150 million tweets, confirming that information about politics circulated mostly in closed loops between ideologically lenient users. Conover et al. [42] looked at 250,000 tweets before 2010 U.S. Presidential campaign, confirming that the network of retweets shows high partisanship, as most of the retweets (intended as a form of endorsement) are ingroup for both conservatives and liberals. In-group and out-group dynamics are predominant in Del Vicario et al. [53] , Bessi et al. [24] , Zollo et al. [219] and Bessi et al. [25] , where pro-science users are opposed to pro-conspiracies users, and the two communities rarely overlap. Homophily and ideological segregation, however, only trigger partisanship on flammable contents, like politics. Barberá et al. [18] measured polarisation over 12 diverse topics, and found that mainly political issues caused a high partisanship of users, while other events did not (such us Super Bowl, Oscar 2014, Winter Olympics). Garimella et al. [79] collected 10 datasets of tweets, divided in political (i.e., online debates over political topics) and non-political. They also found that political topics are more divisive, while non-political did not trigger the same partisan response.

Users' individual and social biases play a relevant role in polarisation dynamics. For instance, Bessi [22] investigates personality traits of conspiracy followers and pro-science users. Boutyline et al. [28] instead establishes a link between a preference for certainty, stability and familiarity of conservative voters and political extremists, and higher levels of homophily and in-group sense of belonging. Such conformity to in-group ideas brings social advantages to the individual. In Garimella et al. [79] authors noted that bi-partisan users, i.e. users that bridge opposite echo-chambers, show lower Pagerank centrality in the network, while also receive less explicit appreciation (e.g., with likes), retweets and mentions than partisan users. Politicians on Twitter with an extremist, polarising stance have more followers than politicians with same activity and same presence on other media [108] . On the contrary, out-group dynamics can be openly adversarial. Conover et al. [42] noted high homophily between in-group users in retweets, but also a big, mixed cluster with low homophily between in-and out-group mentions, that authors verified to be made of provocative interactions. Williams et al. [211] show similar findings on the two communities of skeptics and activists about climate change on Twitter. Lai et al. [125, 124] confirmed that also during Brexit and the 2016 Italian constitutional referendums, retweets were mainly used as endorsement, as opposed to mentions and "reply to" messages, that were adopted by users to engage a debate with someone with an opposite views. They also noticed that very active users tend to mild their own stances on Twitter after referendums results. It should be observed that all of these analyses make use of tools from network science and also computational linguistics (see Sec. 7), such as sentiment analysis and automatic stance detection, for a better understanding of the overall dynamics. In Bail et al. [12] authors surveyed Democrats and Republicans users before, during and after a month of observation, during which they exposed users to a bot spamming content from the opposite party. Republicans that followed the liberal bot resulted to be substantially more radicalised than before. Munson et al. [152] suggested that educating users about how much they are exposed to a single-minded environment may lead to better results in correcting polarisation.

Polarisation may be the long term outcome of a process driven by inner and outer biases on each user online experience. The interplay between personal biases, social biases and algorithmic filters is addressed in Geschke et al. [81] , where agents with limited attention are exposed to information coming from neighbours, personal searches, and algorithms. The effectiveness of algorithms in shielding users from discordant ideas is in fact disputed: along with empirical findings that confirm the existence of a filter bubble [161] accountable to algorithms [188] , there are also evidences of the contrary, of a minor or non existent role of algorithms in driving the polarisation process [153] . Nikolov et al. [157] measured diversity of information consumed by approximately 100.000 users in mail, search logs and social media. They found that social media algorithms lead to a narrower variance of opinions the user could pick, compared to web searches. This view is shared also by Cinelli et al. [40] , where authors compared homophily on several social networks, claiming that substantial differences between Reddit's and Facebook's content consumption patterns are due to the news feed algorithm of the latter. Also Flaxman et al. [70] found similar results: authors show that "articles found via social media or web search engines are indeed associated with higher ideological segregation than those an individual reads by directly visiting news sites". However, they also highlight how users are more exposed to diverse ideologies through social media than through web engines, addressing to users biases the lack of variance in news read. The same consideration holds in Bakshy et al. [14] , where conservative Facebook users news feed was estimated not far from a random sample of political ideas. In the authors words, "Within the population under study here [10.1 million users], individual choices more than algorithms limit exposure to attitude-challenging content in the context of Facebook". Haim et al [102] reject the hypothesis that Google news feed may overindulge with one-sided news.

Although the radicalisation of political views is not entirely to blame on the web [112, 153] , it is still reasonable to believe that online social media may play a role in this process, because they represent a platform for media outlets to reach visibility, and also for user generated content to potentially spread at an unprecedented rate.

It is worth noting that also the existence or the impact of echo chambers to the spreading of misinformation are not universally accepted as facts. The aforementioned contributions from Barberá et al. [18] and Flaxman et al. [70] claim that the echo-chambers phenomenon is overstated. Bruns [30] investigates on the Australian twittersphere, finding limited tendencies to political homogeneous clusters of users. It also cites a research from the Pew Research Center, that shows how "only 23% of U.S. users on Facebook and 17% on Twitter now say with confidence that most of their contacts' views are similar to their own. 20% have changed their minds about a political or social issue because of interactions on social media". Dubois et al. [60] shows the results of a survey conducted in the United Kingdom, and concludes that only a small fraction of people find themselves into an echo chamber. Guess et al. [99] is also critical of the echo chamber phenomenon, and authors present an exhaustive list of arguments to support their claim.

Last, we focused on polarisation mainly, but we have to point out that this binary lineup does not necessarily mean that there are only two surviving opinions on one issue. An individual is entitled with doubts and soft opinions, they may acknowledge partial grounds to the opposite party, or even change their opinion when confronted with new evidences. This is also accounted in many opinion dynamics models we outlined in Sec. 5.1. For example, the Twitter's debate on immigration in Italy shows that there is an "almost" silent majority of users with mild or less polarised opinions between others expressing radicalised views or following political leaders with a clear (sometimes extreme) stance on this topic (see Vilella et al. [201] ). Fragmentation is likely more common in debates than polarisation, even if an empirical observation of such phenomena is in general more challenging due to the difficulties of assigning scores to multi-faceted opinions that have values in a continuum. However, studying fragmentation in real scenarios instead of polarisation could be one of the forthcoming problems to be addressed to better understand how misinformation spreads even in less radicalised debates. Computational linguistics (Sec. A.3) provides a set of tools to detect stances representing different clusters in a network.

Understanding how misinformation spreads, and which are the characteristic temporal patterns of such dynamics, is an instance of a broader problem of understanding information diffusion on communication networks and social media. Such simple observation is a promising starting point, because we have tools for tracking cascades of "memes" propagating in social media, as well as empirical studies on temporal patterns that can be found in news cycle, and models that characterise information spreading. For example, Leskovec et al. [132] proposed a methodology to track memes, and to study their temporal dynamics, and Matsubara et al. [141] developed a model to observe and understand the rise and fall of information diffusion patterns; also, the role of the exposure to information through social media platforms and its implications in re-sharing has been empirically studied extensively since the work of Bakshy et al. [15] .

Some of the earliest studies on the peculiarities of rumour and misinformation spreading are due by Moreno et al. [149] , Acemoglu et al. [1] , Chierichetti et al. [38] , Kwon et al. [123] . Acemoglu et al. [1] proposed a game-theoretic approach to model (mis)information spreading, with some selected agents that can influence the opinions of their neighbours, without changing their own (i.e., a behaviour that is elsewhere referred as zealotry, as in Mobilia et al. [148] ). They also assume that all the agents may receive information from others. Under these assumptions, they observe that all the beliefs reach a stochastic consensus, coherently with the first class of opinion dynamics models we have introduced in Sec. 5. Rumour spreading is also represented by means of diffusion of beliefs and opinions over networks by Chierichetti et al. [38] , Moreno et al. [149] . In search for empirical characterisation of rumour propagation, Kwon et al. [123] studied how some stories spread in online social networks, finding that news cycles follow daily fluctuations and are likely to resurface to the attention of social media users from time to time. These temporal patterns can also be found in the analyses that we will discuss in more details in Sec. 7.3 by Vosoughi et al. [204] , Shao et al. [184] , and others: even if the fake-news has been corrected by some, it can still be shared by many. Another class of models take into account branching processes, as in Del Vicario et al. [200] , aiming at understanding opinion polarisation and its interplay with misinformation spreading, as we already discussed in Sec. 5.

In previously discussed models, fact-checking is considered as a remedy after the misinformation infection, and the underlying mechanisms embedded in these frameworks are related to how individuals can be influenced by others they are in contact with, or how the structure of the network can change diffusion dynamics or vice-versa. However, the competition between a fake-news and its correction, may actually change misinformation spreading patterns in ways that do not affect "ordinary" news cascades. As already discussed in Sec. 2, debunking does not imply necessarily that the information will be properly corrected or removed from social media: hyper-correction, back-fire effects, and other cognitive biases, can lead people who have been exposed to misinformation, on the long term, to forget the fact-checking, or to remember only the false information, or also to deny facts to continue supporting previous beliefs and opinions. Therefore, even worse than ineffective, fact-checking can be counterproductive: not only fact-checkers contribute, during the publication of the correction, to the spread of the original (false) news, but they may also play a fundamental role in the emergence of opinion polarisation and echo-chambers (see Sec. 5). As a consequence, although indirectly, debunking could be one of the elements that fertilises the soil for further propagation of fake news. Nevertheless, we need stronger arguments and evidences to better understand the role of debunking, its efficacy, and under which conditions, if any, it may help to eradicate misinformation from a network. Moreover, we may wonder if, analogously with compartmental modelling and the definition of the basic reproductive number R 0 (see Appendix A.2), is there a measure that can be compared with the "epidemic threshold", that tells us if and when we can stop worrying because the misinformation will be vanishing without any intervention.

The intuition that there exists a natural analogy between the diffusion of diseases and rumours, can be traced back to 1964 and the seminal work of Daley and Kendall [47] . Their approach was based on compartmental models (see App. A.2): the population is divided into compartments that indicate the stage of the "infection", similarly with SIR and SIS, whose states can be Susceptible, Infected, Recovered. The evolution of the spreading process is ruled by transition rates between states in differential equations. In the adaptations of the models to rumours and news, a similarity between the latter and infectious pathogens is considered.

Instances of compartmental models that have been used to study misinformation diffusion include the addition of an exposed state by Jin et al. [114] , a forgetting transition and an hibernator state by Zhao et al. [215] , periodical bursts in time evolution by the already mentioned Matsubara et al. [141] , super-spreaders and apathy with heterogeneous activation patterns by Borge-Holthoefer et al. [27] . A general model that has the aim to unify all the previous one was proposed in 2016 by de Arruda et al. [50] .

Fact-checking, as one of the underlying mechanisms that shape the dynamics of the diffusion process, has been introduced with the hoax epidemic model 2 by Tambuscio et al. [195, 193, 194] . Also, other mechanisms such as a varying forgetting behaviour and different underlying network structures are ingredients of the model, to embed other cognitive and relational driven characteristics to the analytical framework, and to test different what-if scenarios.

An agent in the hoax epidemic model can be in any of the following three states: 'Susceptible' (S), if they have not been exposed neither to the fake-news nor the fact checking, or if they have previously forgotten about it; 'Believer' (B), if they believe in the news and choose to spread it; and 'Fact-checker' (F) if they know the news is actually false -for example after having consulted a trusted debunking site -and choose to spread the fact-checking. The ith [195, 193, 194] .

agent at time step t is a node in a social network and it has a number of neighbours in state X ∈ {S, B, F } denoted by n X i (t). Also, the fake-news has an intrinsic credibility α ∈ [0, 1], and that the overall spreading rate is β ∈ [0, 1]. Transition probabilities between states are defined accordingly and displayed in Fig. 4 :

• spreading [S → B, S → B]: the susceptible agent, once exposed to the news, believes it with probability f i , or fact-check it with probability g i ;

• verifying [B → F ]: the believing agent verifies the news with a p v probability;

• forgetting [B → S, F → S]: both the believer and the fact-checker can forget with probability p f .

The model assumes that a believer can verify the news and then to turn into a fact-checker, but not the opposite. However, this can happen for an agent that forgets the correction or previous belief, and go back to the susceptible state. Unlikely compartmental models used to study the transmission of biological pathogens, some social contagion aspects are embedded here: the spreading probabilities admit bandwagon effects and group pressures. Hence, they change in function of the number of believers or fact-checkers in the neighbourhood of agent i at time t, as it follows:

In [195] , the authors analyse the behaviour of the model at equilibrium. Starting from few agents seeded as believers, they derived the expressions for the density of susceptible agents, believers, and fact-checkers, at the infinite-time limit denoted by S ∞ , B ∞ , F ∞ , respectively. The results were confirmed with both simulations on different network topologies, and with analytical analysis by mean field equations: the value of S ∞ stabilises, independent of the network topology (randomly generated), the value of p v , and α. This means that we reach a consensus within the population (a large majority of believers or fact-checkers), a fragmentation, or a polarisation, dependent on given parameters. In particular, and analogously with the SIR and SIS models, it is possible to calculate an epidemic threshold, to predict under which conditions the fake news would stop from spreading in homogeneous networks, i.e., when the number of believers at equilibrium converges to zero:

This means that there exists a critical value of p v above which the fake-news is eradicated from the network, and such value depends on α and p f , but not the spreading rate β.

As investigated in [193] , the model allows to put lights also to the interpretation of the given epidemic threshold and to polarisation phenomena in segregated networks. First of all, to reduce the numbers of parameters, verification probability can be set as a function of the (personalised) credibility of the fake-news. therefore, if p v = (1 − α), the epidemic threshold of Eq. 8 can be rewritten in function of the forgetting probability:

The above inequality shows a not linear relationship between p f and α, but however for some combination of these two parameters, the fake-news can be removed from the network. The forgetting probability can be considered as a measure of the attention toward a specific topic, meaning that if there is a large discussion around the subject, then exposed people tend to have a stable opinion about it, leading to polarisation between believers and fact-checkers, otherwise the probability to forget the belief and the news will be higher. The presence of this threshold in Eq. 9 suggests that the level of attention plays an important role in fake news global spread and persistence. This is probably what differentiates conspiracy theories from mere rumours: in the first case, forgetting probability is very low, and we have the persistence of false information spreading within a segregated network; in the latter case, the news vanishes out following the ordinary pattern of news cycle, and even if they resurface, they are likely to disappear again. Also, the model can help to understand the impact of a clustered structure of the underlying network on the competition between fact-checkers and believers, expecially if the population's relationships are homophily driven: individuals with very similar beliefs are more likely to get along than individuals with different opinions. It is possible to assign different values of α to sub-populations: the individuals can be sceptic (with low alpha) or gullible (with high alpha). Then, groups of sceptic and gullible agents can be well-mixed together or highly separated in different segregation scenarios, like red and blue nodes in Fig. 3 . Using agent-based simulations with different segregation configurations, it was observed that if the forgetting probability p f is large (e.g., the case of mere rumours), segregation has a different effect on the spread of the fake-news, and the presence of links among clusters can promote the diffusion of the misinformation also within the sceptic groups. However, if such forgetting rate is small (e.g., the case for conspiracy theories) then the fraction of the population that believes the fake-news will be large or small depending on whether the network is, respectively, segregated or not.

The results presented so far have implications on understanding the effect of fact-checking over individuals debating on conspiracy theories: as already discussed in Sec. 5, when individuals have strong opinions toward a topic, the emergence of polarisation and echo-chambers is very likely, as observed by Bessi et al. [24, 25] between followers of science or conspiracies pages and channels in social media. That means that we can expect that these cases are characterised by large segregation between sceptic and gullible individuals, and low forgetting probability. According the hoax epidemic model, under these conditions the fraction of believers in the population is maximised, and we can expect to find believers even among the sceptic individuals, making the eradication of the fake-news quite impossible.

Questioning the efficacy of fact-checking when conspiracy theories are into play is well grounded and substantiated by empirical evidences, mathematical modelling, and agent-based simulations. However, it is still possible to identify the most promising "immunisation" strategies to contain the spread of misinformation as much as possible. Several instances of the hoax epidemic model have been compared in the what-if analysis presented in [194] . First of all, to represent as much as possible a world where fake-news related to some conspiracy theories are spreading, the forgetting probability should be set to a very low value (e.g., p f = 0.1), and no believer can turn to a fact-checker directly (i.e., p v = 0). Under these more "pessimistic" assumptions, very few stochastic realisations of the model return a scenario where the believers are a minority w.r.t. the fact-checkers at equilibrium. However, there is still some important lessons to learn: apparently, if every node, even fact-checkers, is allowed to change its status, the fakenews will always win the competition against debunking, and the system will reach a consensus toward a population of believers. Vice versa, verification (p v > 0), and the presence of zealot fact-checkers that do not ever forget (see again Mobilia et al., [148] , and Acemoglu et al., [1] ), can give a chance to fake-news to vanish out or being confined only to gullible clusters: fact-checking can be ineffective, but a world without debunking would be much worse. In fact, if the system is forced to keep a minority of some strongly committed debunkers, some positive equilibria are possible: for example, if the top 10% of the hubs of the sceptic population (i.e., highest degree sceptic individuals) are zealot fact-checkers, the fake-news will be spreading mainly within the gullible clusters, without "infecting" the sceptic groups.

Of course, there are many shortcomings to be considered when we try to apply these models' outcomes: first, validation of results is difficult because many nodes can be in a hibernator state, as observed by Zhao et al. [215] , e.g., individuals believed the fake news, but they do not share their belief on social media, but they can still convince someone else off-line. In such cases, it is difficult to estimate the numbers of "infected" individuals, and all the subjects that have been exposed should be tested directly, like with epidemics where also asymptomatic patient could transmit the disease. Second, convincing influencers to openly debunk fake-news could be hard. Third, calculating values for parameters such as α, β, p v and p f on real stories can be unrealistic, especially at the earliest moment of the cascades formation. However, this could help a social media company to embed, in their platforms, automatic methods to identify "influencers", and label as "likely false" some of the stories they have been exposed to (for example, using the classification system proposed by Ciampaglia et al. [39] ), to suggest them to use some trusted debunking site before spreading the news.

For a more focused review of existing propagation models for misinformation and fake news mitigation techniques, see also the recent chapter by Saxena et al. [180] .

The development of systems that can automatically detect fake news and prevent their propagation is very challenging since they can be manipulated in a number of different ways in order to confuse the users. One of the most important source of information for the systems can be extracted from the content of the news article. Theories developed by forensic psychology have implied that deceptive statements are different to the true ones in terms of writing style and sentiments expressed [220] . In addition, the study by Vosoughi et al. [204] showed that false stories inspired fear, disgust, and surprise in replies whereas true stories inspired anticipation, sadness, joy, and trust. Also, they showed that the content of false news was more novel than true news.

Computational linguistics, that is the scientific study of language from a computational perspective (see Appendix A.3 for a quick introduction), and the tools that provides can be very useful in the development of automatic fake news detection systems. In this section, we will present related work that focuses on the detection of fake news from the perspective of computational linguistics either applied only on the textual content of the article or in combination with additional information extracted from images or user profiles. Moreover, we will briefly discuss how bots and trolls, both playing an important role in spreading disinformation in social media, can be detected on the basis of the language they use.

Recently, a number of fact-checking websites have been developed with the aim to raise awareness to public regarding news that contain false information. These websites rely on domain-experts that check and verify the news content. Some of those websites assess the credibility of claims made by politicians (e.g., politifact 3 ), whereas other websites provide labels for the type of the disinformation (e.g., HoaxSlayer 4 ). Some of those websites including Politifact, Snopes 5 and GossipCop 6 have been extensively used by the researchers as a means to create annotated collections on which the proposed methodologies can be evaluated.

Manual fact-checking is very slow and only few news articles can be checked. To this end, the development of systems that can automatically detect fake news is critical. One of the information that has been explored is the credibility of the websites that publish the article. That is because a website that has already published false information in the past, it is likely to do that again in the future. Journalists as well leverage the source of the news as a basic information in order to verify the credibility of an article. One of the works that focused on the prediction of the factuality of a news site was presented by Baly et al. [16] . For the prediction, Baly et al. used a wide range of features extracted from a sample of articles published by the news site, its Wikipedia page, its Twitter account, its URL structure and web traffic information. The experiments that were performed with a Support Vector Machine (SVM) classifier showed that the textual features extracted from the articles led to the highest performance on factuality prediction.

Other researchers used external knowledge to verify a claim as an additional source of information. Karadzhov et al. [120] proposed a deep neural network with Long Short Term Memory (LSTM) encoding and task-specific embeddings to verify a claim. To address the task, they retrieved documents that were relevant to the claim using a search engine. The snippets and the most relevant sentences were then compared to the claim. The final classification was performed with an SVM classifier. The architecture of the proposed system is depicted in Figure 5 .

Other researchers proposed to model the external evidence using neural networks. Popat et al. [170] proposed DeClarE, a neural network model that aggregated signals from external evidence articles, the language of these articles and the trustworthiness of their sources. The system first used the claim as input to retrieve relevant documents from the Web. A bidirectional LSTM (biLSTM) was then used to capture the language of the web articles and an attention layer to focus on the parts of the articles that were relevant to the claim. Information about the claim source web article contexts, attention weights, and trustworthiness of the underlying sources was aggregated for the final assessment of the claim. The architecture of the proposed system is depicted in Figure 6 . 

A large number of studies focused on information extracted from the textual content of the news articles that is one of the most important information for the detection of fake news. In particular, several works have studied the impact of linguistic and semantic information on fake news detection [35, 88] . The textual information proposed in the different studies ranges from simple features such as word frequencies and text length [35, 216] to more complicated such as latent representations extracted using deep learning methodologies [118] .

Early works were based on manual features and traditional representations of text such as Bag-of-Words and term frequency-inverse document frequency (TF-IDF) [35, 166] . These representations lead to high dimensional feature vectors and do not capture the co-occurrence statistics between words. To overcome the limitations of the traditional representations, researchers explored the effectiveness of low text representations. These representations, known as embeddings, can be at word [146] , sentence or document level [128] , and can be used as input to traditional classifiers [216] or neural network architectures [88, 85, 174, 205] . Some studies have used pre-trained embeddings [170, 87] such as GloVe [164] and ELMo [168] whereas others built word embeddings from the input text [138] .

Recently, some researchers have proposed methods that are based on the Bidirectional Encoder Representations from Transformers (BERT) model [54] with the aim to capture the context representation of a sentence [118, 119] . BERT is based on Transformers, a deep learning model in which each output element is connected to every input element, and the weightings between them are dynamically calculated based on their connection. Jwa et al. [118] used BERT to analyse the relationship between the headline and the body text and its impact on fake news detection, whereas Figure 7 : Emotionally-infused neural network architecture for false information detection proposed by [85] Kaliyar et al. [119] proposed FakeBERT by combining different parallel blocks of the single-layer Convolutional Neural Networks (CNN) with BERT.

Together with the general textual representation, some researchers were also interested in exploring the impact of semantic information. To this end, the role of emotions on fake news detection has been studied by several researchers due to the fact that fake news tend to produce different emotions and of different intensity to the users compared to the real news [204] . Giachanou et al. [88] proposed EmoCred, a system for credibility assessment based on emotional signals extracted from the text of the claims. The architecture of EmoCred was based on an LSTM model and concatenated textual and emotional features for credibility detection. Three different approaches were explored for estimating the emotional signals based on emotional lexicons and a CNN network. The results showed that incorporating emotional signals can increase the effectiveness of fake news detection. Ghanem et al. [85] explored the role of emotions in the detection of the different types of fake news (i.e., satire, hoax, propaganda, clickbaits and real news). The proposed network consisted of two branches, the first one was based on word embeddings and the second on emotion frequencies. Figure 7 shows the architecture of the proposed system. In a follow-up work, the FakeFlow system was proposed to model the flow of affective information in fake news articles using a neural architecture [84] . Recently, Guo et al. [101] proposed a dual emotion-based fake news detection model to learn content and comment emotion features and their relationship for publishers and users respectively.

It is very common that fake news also contain images that have been manipulated in a number of different ways. Visual information can be useful for the systems to detect whether an article is fake or not. Several researchers proposed methodologies that incorporated visual information and analysed its impact on the performance of fake news detection. Visual information can be captured by hand-crafted features [115] or latent representations extracted by neural networks [206, 121] . Wang et al. [206] proposed the Event Adversarial Neural Networks (EANN) model that consisted of a textual component represented by word embeddings and a visual one that was extracted using the VGG-19 model pre-trained on ImageNet. Figure 8 shows the architecture of EANN and its different components. Khattar et al. [121] proposed the Multimodal Variational Autoencoder model based on bi-directional LSTMs for the textual representation and VGG-19 for the image representation. Singhal et al. [187] proposed SpotFake, a multimodal system for fake news detection based on text and image modalities. Singhal et al. applied BERT to learn text features and to incorporate contextual information, and pre-trained VGG19 to extract the image features. Their system managed to outperform the baselines.

Some researchers incorporated the similarity between the image and the text in their models since this information can be valuable during the learning process. Giachanou et al. [89] proposed a multimodal system that combined textual, visual and semantic information. For the textual information they used word embeddings and sentiment. For the visual information Giachanou et al. combined image tags extracted with 5 different pre-trained models and local binary patterns, whereas for the semantic information they used the cosine similarity between the embeddings of the image tags and text. Different to the majority of the systems which were based on visual features extracted from one image, Giachanou et al. [90] proposed to use visual features extracted from more than one image. In particular, they combined Figure 8 : The architecture of Event Adversarial Neural Networks proposed by [206] visual features extracted from three images, textual features and image-text similarity. The textual representation was based on pre-trained BERT, whereas for the image representation they leveraged the VGG-16 model to extract image tags that were then passed to an LSTM model to capture the sequence of the images. Finally, the image-text similarity was calculated using the cosine similarity between title and image tags embeddings.

Zlatkova et al. [218] focused on the relevant problem of detecting fake claims about images. To address this task, Zlatkova et al. explored a variety of features extracted from the claim (e.g., claim text), the image (e.g., google tags, URL domains) and the relationship between the two (e.g., cosine similarity, embedding similarity). In their study, they found that URL domains play a very important role in predicting the factuality of a claim with respect to an image.

Although the textual and visual information extracted from the content of the articles is very critical for the effective detection of fake news, the role of users is also important. Users are the ones that decide whether to share a piece of fake news or not when this arrives on their screen. Contrary to what one may expect, Vosoughi et al. [204] showed that bots accelerated the spread of true and false news at the same rate, implying that humans have a significant contribution to the spread of false news.

Given the role of users in the propagation of fake news, profiling users that are involved either in the propagation or prevention of fake news based on their profile information or the language they use is very important. Shu et al. [186] analysed different characteristics of users that were sharing fake news and those who were sharing real news. They explored the impact of explicit profile features such as number of statuses and implicit profile features such as age, location and personality traits on fake news detection. An analysis on the importance of the features showed the registration time and whether the user is verified or not are the two most important features for the detection of fake news.

With the emergence of the fact-checking sites, a new group of social media users appeared that aims on raising awareness regarding the factuality of news. Those users are known as fact-checkers. Giachanou et al. [87] analysed and explored the impact of different linguistic features and personality traits on differentiating between fact checkers and fake news spreaders. Vo and Lee [203] analysed linguistic characteristics of fact-checking tweets and found that the fact-checkers tend to use formal language and use few swear words and slangs. In addition, they proposed a deep learning framework to generate responses with fact checking intention. The Factchecking Response Generator framework is based on Seq2Seq with attention mechanisms.

Giachanou et al. [86] focused on users that tend to propagate conspiracy theories, known as conspiracy propagators. In their study, Giachanou et al. performed a comparative analysis over various profiles, psychological and linguistic characteristics of conspiracy and anti-conspiracy propagators. In addition, they proposed ConspiDetector, a model based on a CNN and which combined word embeddings with psycho-linguistic characteristics extracted from the tweets of users with the aim to differentiate between conspiracy and anti-conspiracy propagators. Their experimental results showed that ConspiDetector improved the performance in detecting conspiracy propagators by 8.82% compared to the baseline.

Ghanem et al. [83] proposed an approach to detect non-factual Twitter accounts such as propaganda, hoax and clickbait accounts. They treated post streams as a sequence of tweets' chunks. They explored a range of different semantic and dictionary based features including word embeddings, emotion, sentiment, morality and style. Their methodology was based on a LSTM neural network.

In order to facilitate research on profiling fake news spreaders, Rangel et al. [172] organised an evaluation task on profiling users who have shared some fake news in the past. In order to create the collection Rangel et al. collected tweets that were relevant to news articles that were annotated as fake or real on different fact-checking websites. After a manual inspection of the tweets, they collected the timelines of the users that have shared a high number of either fake or real news. Users that have shared at least one fake news was labelled a potential fake news spreader. Otherwise, the user was labelled as real news spreader. A wide range of features (e.g., word embeddings, n-gram, emotions, personality traits, punctuation marks) and learning approaches (e.g., SVM, Logistic Regression, CNN, LSTM) were used by the participants. The highest performances were achieved by the approaches that used n-grams [169, 31] .

Bots aim at influencing users with commercial, political or ideological purposes. They often spread disinformation trying, for instance, to enhance some political opinions or supporting some political candidates during elections [23] . For example, Stella et al. [190] showed that 23.5% of 3.6 million tweets about the 1st October 2017 referendum for the Catalan independence were generated by bots. About 19% of the interactions were from bots to humans, in form of retweets and mentions, as a way to support them (echo chamber). Regarding the former US presidential election, Bessi and Ferrara [23] showed that, in the week before the election day, around 19 million bots tweeted to support Trump or Clinton 7 . Paul and Matthews [163] stated that in Russia fake accounts and bots have been created to spread disinformation 8 , and around 1,000 Russian trolls would have been paid to spread fake news about Hillary Clinton 9 .

Pioneer researchers such as Lee et al. [129, 130] proposed the use of honeypots to identify the main characteristics of online spammers. To this end, they deployed social honeypots in MySpace and Twitter as fake websites that act as traps to spammers. They found that the collected spam data contained signals strongly correlated with observable profile features such as contents, friend information or posting patterns. They used these observable features to feed a machine learning classifier with, in order to identify spammers with high precision and low rate of false positives. More recently, Alarifi et al. [5] have collected and manually labelled a dataset of Twitter accounts including bots, human users, and hybrids (i.e. tweets posted by both human and bots). In this work, random forest and Bayesian algorithms reached the best performance at both two-class (bot/human) and three-class classification (bot/human/hybrid). Gilani et al. [93] proposed a framework for collecting, pre-processing, annotating and analysing bots in Twitter. Then, in [92] they extracted several features such as the number of likes, retweets, user replies and mentions, URLs, and follower friend ratio, among others. They found that humans create more novel contents than bots, which rely more on retweets or URLs sharing. Dickerson et al. [56] approached instead the bot detection from an emotional perspective. They wondered whether humans were more opinionated than bots or not, showing that sentiment related factors help in identifying bots. The Botometer tool 10 (formerly known as BotOrNot) by Onur et al. [199] was employed in several works, for instance by Shao et al. in a work where they analysed the spread of low-credibility contents [184] . Botometer was also used by Broniatowski et al. [29] to analyse the effect of Twitter bots and Russian trolls in the amplification around the vaccine debate. Recent methods use representation learning based on social and interaction networks [45] or employ deep learning techniques. For instance, Kudugunta and Ferrara [122] fed an LSTM network with both content and metadata. Similarly, Cai et al. [33] used an LSTM to analyse temporal text data collected from Twitter.

Echoing the importance of automatically detecting bots, DARPA held a 4-week competition [191] with the aim at identifying influence bots supporting a pro-vaccination discussion on Twitter. Influence bots can be defined as those bots whose purpose is to shape opinion on a topic, posing in danger the freedom of expression. Although there are several approaches to bot detection, almost all of them rely on metadata beyond text. Rangel et al. [173] organised an evaluation task, both in English and Spanish, with the aim to determine whether the author of a Twitter feed is a bot or a human on the basis of her textual information. Participants used different features to address the task (mainly n-grams, stylistics features and word embeddings) that were fed to machine learning algorithms such as SVMs or deep learning techniques such as CNNs, RNNs, and FeedForward neural networks.

Different to bots that are automated, a troll is a real user. Current research on that area focuses on identifying trolls [3] and analysing their political comments [73] and a few health topics such as vaccines [29] , diet [147] , the health of Hillary Clinton, abortion, and food poisoning [135] . Recently, Atanasov et al. [11] proposed an approach with the aim to analyse the behaviour patterns of political trolls according to their political leaning (left vs. news feed vs. right) using social media features. They conducted experiments in both supervised and distant supervision scenarios. In the proposed methodology, they leveraged the community structure and the text of the messages from which they extracted different types of representations such as embeddings. Ghanem et al. [82] proposed a text-based approach to detect the online trolls of the US 2016 presidential elections. Their approach is mainly based on textual features which utilise thematic information, and profiling features to identify the accounts from their way of writing tweets. Alsmadia and O'Brien [8] addressed the problem both at account level, considering features related to how often Twitter accounts are tweeting, and at the tweet level. They noticed that bot accounts tend to sound more formal or structured, whereas real user accounts tend to be more informal in that they contain more slang, slurs, cursing, and the like. Finally, Jachim et al. [113] introduced TrollHunter, an automated reasoning mechanism to hunt for trolls on Twitter during the COVID-19 pandemic in 2020. To counter the COVID-19 infodemic, the TrollHunter leverages a linguistic analysis of a multi-dimensional set of Twitter content features to detect whether or not a tweet was meant to troll.

Using machine learning and computational linguistics techniques to identify and categorise messages shared on social media platforms and that are supposedly related to some topic under study (e.g., debates on immigration, vaccination, mask wearing), has become very common even in not computer science related disciplines, such as political and social sciences. The same is true if the researcher's goal is to identify the entities who controls social media accounts, that can be humans, bots, and trolls. This is usually important if the academic scholar's goal is to better understand how disinformation campaigns and astroturfing can be conducted and coordinated to maximise their impact. The interested reader can read, for instance, the special issue on "Beyond Fake News: The Politics of Disinformation" of the Political Communication Journal (see the issue's introduction by Freelon and Wells in [74] ), and also He and al. [105] that proposed, in the Journal of the American Medical Informatics Association, a comprehensive analysis of tweets shared during the covid-19 pandemic, to find that there was a constant 10% level proportion of anti-mask messages throughout their study period.

In particular, automatically detecting a bot from its behaviour and its writing styles is a moving target: if a software learns mimicking humans' behaviour, we can expect that, even in the future, malicious companies and programmers will take advantages of novel tools also to improve the performances of their AI controlled accounts. The problem is twofold: (1) should we be worried of the real impact of bots in social media? (2) given that studying the impact of social bots is worth studying, how much should we trust classification tools?

A negative answer to the first question would mean that only humans, and not bots, are to blame for spreading misinformation, for manipulating online debates, and for poisoning social relationships. This would stop the necessity to continue investigating for aiming at the design of better bot detection systems. However, it is more complicated than that.

Aiello et al. [4] reported the result of an experiment where one bot that reached an unexpected popularity in a popular social media platform for book lovers, was found very effective for leading users to accept link recommendations and also for determining polarisation in the underlying social and communication networks. Although many bots can have benign or also harmless purposes, they can be designed with the goals of persuading, manipulating, or deceiving humans, as brought to attention by Ferrara et al. [65] . Bots' contribution to the spread of low-quality information (including misinformation) was documented in the already mentioned studies by Shao et al. [184] , Vosoughi et al., [204] , and Stella et al. [190] . However, the question remains: who is to blame the most, bots or humans?

The answer to this question remained controversial for some time, also because of two very influential papers published in high impact journals (respectively Science and Nature Communications), reporting apparently contradictory results: Vosoughi et al., [204] , after the analysis of ≈ 126, 000 stories tweeted by ≈ 3 million people more than 4.5 million times, concluded that humans, and not bots, contributed more to the acceleration of the spread of fake-news; Shao et al. [184] , after the analysis of 14 million messages spreading 400 thousand articles on Twitter during ten months in 2016 and 2017, observed instead a "disproportionate" role of bots in spreading low-credibility articles. However, this is an insubstantial contradiction, because Shao et al. found that bots adopt the following strategy: they amplify the diffusion of articles before they go viral, also exploiting the structure of the network and the role of single accounts. For example, bots target users with followers through replies and mentions. We know that humans are vulnerable to manipulations, and they tend to re-share content posted by others if their messages are aligned with their personal beliefs (see Sec. 4). When an account controlled by an (influential) human shares an article, the probability that that message will became viral is much higher. For this reason, humans should be blamed the most: if bots are responsible for injecting the fake news in social media, humans should be in charge of checking the veracity of the message before re-sharing it. When they are fooled, and they are also followed by many, the strategy reached bots' ultimate goal: misleading a much larger portion of humans, as summarised by Aral in [10] .

In his book "The Hype Machine" [10] , S. Aral points out that social media platforms must take into account that humans are vulnerable to deception, and change their business plans to embed new countermeasures to protect their users against bots' manipulative actions. For example, shutting down accounts that are likely controlled by software 11 has been presented by social media companies as one signal in that direction. However, this means that the research problem of automatically detecting bots (and trolls) will be in many scholars' agendas for years to come. It should also be noticed that targeting users with many followers, is not the only successfully implemented strategy by malicious agents; in fact, it is possible to exploit natural fragmentation of the network. As observed by Stella et al. [190] during the 2017 Catalan independence referendum campaign, bots started mentioning incessantly accounts controlled by humans in peripheral areas of independentists and costitutionalists groups. A similar bots' strategy was observed more recently by Vilella et al. [202] during the Italian debate on immigration on Twitter: in this fragmented but not necessarily polarised context, social bots tend to inject in the network more content from unreliable sources w.r.t. humans. However, such content often remains confined in a limited number of clusters. Quite interestingly, bots share also a huge amount of content from reliable (or mainstream) sources in order to diversify their reach, and to provoke wider and deeper cascades through the underlying, highly fragmented, information network.

The results presented so far suggest that the impact of social bots in social science research is worth studying, and that tools for the automatic classification of the entities controlling social media accounts are of paramount importance to scale down the complexity of analysing huge volumes of data, looking for significant signals of behavioural patterns. As already mentioned, researchers are well aware that bot identification is a moving target, e.g., Cresci et al. [46] noticed that neither Twitter, nor humans, nor up-to-date tools available in 2017 were capable of accurately detecting a novel family of social spam-bots. As a consequence, state-of-the-art bot detection models still need to deal with generalisation and computational efficiency challenges, that could greatly limit their applications. It is not surprising that Botometer by Varol et al. [199] , likely the most popular bot detection tool in computational social science, is continuously updated: Yang et al. [214] propose to properly reduce the account meta-data size, to scale up the efficiency of the system when evaluating a stream of public tweets. Also, learnt model accuracy on unseen cases can be improved by strategically selecting datasets from a rich collection used for training and validation. Sayyadiharikandeh et al. [181] also observed that an ensemble of classifiers specialised for each class of bots that combine the decisions through the maximum rule, can better generalise for unseen accounts across datasets. They observed that such a methodology leads to an average improvement of 56% in F1 score.

The tools' obsolescence problem is still an important issue and it is likely to remain so in the years to come. Gallwitz and Kreil [78] find that this intrinsic limit makes the whole social bot research destined to fall 12 . Rauchfleisch and Kaiser also point out that tools' unreliability is an important issue in [175] , focusing their analysis on Botometer as the most popular tool for social bot identification. In particular, they substantiate their criticism with the observation that many scholars use a threshold of the "bot score" to discriminate bots from humans after the application of Botometers, although this practice is not recommended by the authors themselves in the FAQ of their easy to access API. Rauchfleisch and Kaiser further discuss that thresholds, even when used cautiously, can lead to a high number of humans classified as bots (false positives), and vice-versa (false negatives). According their observations, scholars that want to draw general conclusions in computational social science research from the application of identification tools, must keep away from just relying on classifiers results: researchers are encouraged to (1) manually check a significant sample of accounts, calculating the inter-coder reliability between the tool's outcomes and the human's classification; (2) validate results over time, since the scores can naturally change because of modifications to the classification engine or to the status of the entity controlling the account; (3) ponder that some systems are optimised for some languages and not for others (e.g., Botometer is specialised for English); (4) share account IDs (as allowed by the Twitter developer terms) to let other researchers to replicate and further validate the analyses. 11 Twitter 'shuts down millions of fake accounts', BBC news, 9 July 2018, https://www.bbc.com/news/ technology-44682354 12 It is worth noticing that this (rather controversial) contribution is discussed in a pre-print at the time of this survey's writing

Information disorders are one of the most studied problems of the recent years, and yet one very hard to see a solution anywhere near. The problem itself is complex, being the outcome of the interplay of different factors: algorithms that promote to users an unfair selection of news, social and personal biases that induce users into reading and sharing manipulated news, deception operated through linguistic shenanigans, social network dynamics of diffusion and peer influence, impersonation and spamming from social bots. Scholars have put a great effort so far on discovering how junk news spread online and why, and how to counter it, but more is needed in order to develop viable solutions. In the next years, research will be crucial to answer many of the unanswered questions we have today. For instance, fragmentation of opinions needs to be better addressed instead of simple polarisation, as it provides a landscape to study how misinformation spreads in non-radicalised debates, and how people get radicalised over time; efficacy of debunking must also be tested under different conditions, with special regard to the early identification of news that need to be debunked immediately from news that have less potential to harm; the role of social bots must be clarified, in order to design proper defence strategies. Research can also contribute to the development of useful tools against the spread of misinformation online. From one side, research can assist the defence of user online against deceptive news, for instance designing social network interfaces that empower individuals' judgement and promote spontaneous fact checking. From the other side, it can assist the development and the continuous maintenance of tools for a quick detection of fake news online and automated accounts, as both are moving targets, which constantly evolve, and overcome the problems of generalisation and computational efficiency that affect such tools today.

Complex Network Analysis (CNA) is a set of quantitative methods that allow the analysts to study the structure and the dynamics of a a (physical, social, biological, technological) complex system that could be represented and modelled by means of networks or graphs. In this section we introduce some of the basic measures and concepts of complex networks and graph theory; nevertheless, we recommend the reader to consider some introductory textbooks such as [144, 17, 61, 155] for a more detailed introduction to the wide field referred as Network Science.

Even if we should remember that not every complex systems can be modelled or represented by means of graphs, networks are pervasive in many domains: a social system is a made of actors (or also individuals, or agents) and the relationships they form are ties that link pairs of actors; information systems (like the Web) are sets of (hyper)links, or citations, from documents (such as files, books, web pages) to other documents, and so on. Such representations help us to easily describe the interconnections between small 'units', so that the goal of a CNA is to characterise the overall systems in terms of structural and dynamical features, that can help us to understand, predict and also manipulate higher level phenomena such as synchronisation, traffic and congestion, spreading processes (such as epidemics and social contagion).

We represent such systems as generic graphs G = (N, L), where N = {1, 2, . . . , n} is the set of nodes' unique identifiers, and L = {(i, j) : i, j ∈ N } the set of links (or edges) between them. Graphs can be directed or undirected, or also weighted or unweighted. For each node i we have a set of neighbours N i , and a degree k i , that is the number of links that connect i to its neighbours: k i = |N i |. In directed networks, neighbours of i can be predecessors P i or successors S i , according the direction of the connection that can be, respectively, incoming from or outgoing to the given neighbours. As a consequence, for each node i we have also an in-degree k in i = |P i | and an out-degree k out i = |S i |, so that k i = k in i + k out i . In weighted graphs, we define strength, in-strength, and out-strength as the weighted version of degree, in-degree, and out-degree:

Traditionally, a graph G is mathematically represented by means of a n×n matrix A = {a ij } called adjacency matrix, s.t. a ij = 1 iff (i, j) ∈ L, and a ij = 0 otherwise, when G is unweighted; if the graph is weighted, then a ij = w ij . Note that when G is undirected, then A is also symmetrical. This notation is very convenient for mathematical reasoning and stronger formalisation: measures can be defined easily through this notation (e.g., k i = j a ij ), and it allows to make explicit the relationship between the graph and the eigenvectors and the eigenvalues of the corresponding adjacency matrix, that is exploited in the so called spectral graph analysis. However, it is unpractical for large graphs, likely very sparse, that are translated to memory consuming adjacency matrices full of zeros. For computational purposes, adjacency lists or edge lists are commonly adopted, as well as other more specific data structures tailored for sparse matrices.

The main difference between a simple and a complex network is that a simple graph can be represented by very few characteristics: stars, rings, grids, paths are example of simple networks. Without a proper language and framework, it is usually hard to distinguish the topology and the structure of a complex network created by a real dataset (e.g., from relationships declared by social media users) from an artificially generated random graph. First of all, we define a path of length l between two nodes s and t as a sequence p = {n 1 , n 2 , . . . n l }, such that n 1 = s is the source, n l = t is the target, and ∀x ∈ [1, l − 1] : (n x , n x+1 ) ∈ L. When s = t the path is a cycle. If we have multiple paths from source to target, we define the distance l st between s and t as the shortest path length. The longest shortest path in a network is the diameter. When the average distance l and the diameter are proportional to log n, the graph is said to hold the small world property. The graph is connected if for every pair of nodes we can always find at least a path connecting them; otherwise, the graph G has multiple connected components, or simply components, that are connected subgraphs of G. The largest component is usually called giant component, because for large scale graphs it happens often that its size is > 50% of the whole network. even if the graphs are sparse.

Beyond connectedness, we usually look for classes of nodes with given properties, whose centrality and importance in the system can contribute to assign them different roles. Degree is a measure of "popularity" of a node, and high degree nodes are commonly called hubs. Degree centrality d i of node i is usually defined as a normalised version of node's degree: d i = k i /n. A node that is closer to all the other nodes on average has its own centrality in a network: the closeness of node i is calculated as c i = ( j =i l ij ) −1 . Instead of popularity or topological centrality of a node, we may be interested to bridges, that are nodes that are crossed by paths more often than other nodes in the networks. If ρ st is the number of shortest paths from s to t, and ρ st (i) is the number of shortest paths from s to t passing through node i, the betweenness of i is b i = s =t =i ρst(i) ρst . Nodes with higher betweenness are the bridges of our system. Links connected to nodes with high betweenness are said to have high link betweenness, and they are also referred as bridges of the network. Observe that calculating degree centrality takes constant computational time; on the contrary, betweenness and closeness are calculated after measuring shortest path lengths of involved pairs of nodes. This can be computationally expensive for large graphs, and for many experimental settings algorithms adopting some heuristics are used to reduce such cost. Another set of measures is made of variants of the so called eigenvector centrality, exploiting the spectral properties of a given graph. Page Rank and Hubs and Authorities scores (calculated by the HITS algorithm) are examples of such measures. Their purpose is to define iteratively the importance of a node according the importance of its neighbours, or only its predecessors in a directed graph in particular. Page Rank, for instance, is widely adopted as a measure of node's influence.

All the centralities described above are just few of the many quantitative measures that can be defined for nodes in a graph, and they can be used to rank nodes accordingly. In small networks we can ask which nodes are the most important, conversely on large scale networks this approach is often futile. In these cases we use ranks' intervals to identify classes of nodes (or links) with comparable properties, to assume that, under given circumstances, they have similar roles in the network and in its dynamics. Such statistical approach is commonly applied by plotting centrality distributions, that very often show heavy-tailed shapes, corresponding to highly heterogeneous classes of nodes. Focusing on degree centrality, the heterogeneity of the distribution leads to the emergence of hubs phenomenon, that means that some nodes' degrees are much higher than the rest of the graph, making statistically meaningless the adoption of the average degree k as a scale of this measure. In other words, the average degree should not be used as a good estimate of a randomly selected node's degree in many real large scale networks.

The emergence of hubs and centralities heterogeneity has important implications in many different scenarios. For example, when we want to assess the robustness of a networked system, we can simulate what happens when we remove progressively larger fractions of nodes (or links) from the graph. One consequence of this process is that the network, after some iterations, starts to fall apart; in other words, the initial connected graph is split into smaller components and the giant component's size is reduced considerably. When removing nodes (or links) randomly at each step, we are actually simulating nodes' failures, and if the systems is robust, then the giant component size reduces linearly with the number of steps. This happens for many real network, such as the Internet, where random local failures do not usually affect global transmission functions. However, when we target hubs (or also nodes with high betweenness, high page rank, and so on), real networks are proven to be extremely fragile, and the corresponding giant components are usually disconnected in many smaller components after very few removal steps. This means that some classes of nodes are definitely much more important than others and a targeted attack on a system may lead to the rapid disconnection of the network. However, studying network robustness is a great tool for understanding which is the most promising vaccination or immunisation policy in case of epidemics, for mitigating the diffusion of computer viruses or also controlling information propagation; in fact, finding hubs and bridges in networks is sometimes equivalent to understand how to quickly break down contact and transmission networks.

Targeted attacks are very effective in many real networks because their topologies differ significantly from randomly generated graphs in at least one important aspect: the presence of clusters (or communities), i.e., subgraphs whose internal nodes are tightly connected to each other (high intra-community cohesion), and that are loosely connected to external nodes (high inter-communities separation). The existence of such communities in real work networks put new lights on the interpretation of the above mentioned weaknesses against targeted attacks; in fact, targeting hubs, bridges and other central nodes in our networks is like removing nodes or links that are responsible for cross-communities connections, as well as internal clusters' cohesion: communities rapidly degenerate into disconnected components. Although exploring the whole space of partitions of a graph is a computationally intractable problem, there are many community detection algorithms that use different heuristics to find clusters. Communities in social networks are often due to homophily, i.e., the tendency of individuals to link with similar ones. Also information networks can be shaped by homophily or topicality: two documents on similar topics are more likely to have hyperlinks connecting them or to show smaller distances than two other documents on different topics. This has an impact on the design of modern web crawlers, and the biases of datasets collected by way of snow ball sampling processes.

Graph drawing is also an important step in many empirical analyses: properly visualising a graph can help us to easily compare (sub)structures, or spotting interesting network's areas or nodes. Many network analysis packages and tools have layout algorithms that place nodes on a plane to better emphasise some structural features of the graph. Forcebased layouts are the most widely adopted: they exploit repulsion and attraction laws from physics to place connected nodes near each other, and to move reciprocally away nodes that are topologically distant. This is a way to let clusters be revealed graphically when the network is not too large nor too dense.

Networks' structure has a role in the way nodes' properties change over time; when we focus on aspects such as diffusion of information, transmission of diseases, adoption of innovations or behaviours, then we are studying on networks' dynamics. Studying networks dynamics may require an additional burden for the beginner, because of the many existing approaches and models proposed over the years in different disciplines such as psycho-sociology, statistical mechanics, graph theory, epidemiology, and so on. The spread of misinformation, in particular, is one instance of information diffusion, and the emergence of polarisation and echo chambers is considered one outcome of opinion dynamics: both processes are just two examples of the most studied phenomena that are traditionally observed in networks. If the reader is interested to a comprehensive overview of this field, they can be referred to the complex networks introductory textbooks we already mentioned, and also to [19] that is more focused on dynamical processes. For a survey on information diffusion, check also [100] out. Because of the relevance to the subject of this survey, opinion dynamics are also reviewed in Sec. 5.

There is an important interplay between structure and dynamics, because the first can change, develop or emphasise the second, and vice-versa. This makes some phenomena functions of time, and the observation of their key factors becomes quite subtle in many empirical settings. For example, the emergence of homophily is responsible for high clustering in social networks, and there are two possible mechanisms that can explain this: one is selection, the principle according which similar nodes are more likely to be connected, the other is social influence, that makes connected nodes more and more similar to each other. However, when we study networks with high clustering, it could be difficult to understand if selection is prevalent w.r.t. social influence, or the opposite, or if both factors are at interplay. Furthermore, we can observe that communities change over time: they can rise and dissolve, merge, split, or even grow, shrink, stabilise. Changes in structure, may have effects on nodes exposed to different information and behaviours, that can lead them to yield to group pressure, or to decide to leave their cluster and join another community. Some of these processes may increase the fragmentation of the network, that is responsible of the formation of echochambers and group thinking, also without considering other factors such as filtering algorithms and recommendation systems, that can accelerate this spontaneous tendency to group conformity. One classical example is given by the Schelling's model showing that urban segregation is likely to be a natural phenomenon that can emerge spontaneously even in very tolerant societies, with the consequence that the causes of ghettoization cannot be solely led back to racism or intolerance. However, segregation may lead to inequalities, that could accelerate ghettoization processes, and then racism and intolerance. Similar dynamics can be found even in online social networks.

Network scientists have many different models to use to study networks dynamics. Although some of these (mainly mathematical) models have been proposed using common design principles, there are important differences to be considered if processes under observation are biological, social, economic, technological, and so on.

Compartmental models in epidemiology are often applied to the mathematical modelling of infectious diseases, and are based on compartments of a population assigned to different states, such as S (Susceptible), I (Infected), and R (Recovered). They have been introduced at the beginning of the 20th century for well mixed populations, and they are often solved with differential equations providing deterministic results. They can also be applied on randomly generated networks, running several simulations on different (stochastic) realisations of the models, in order to fit more realistic scenarios. The basic SIR model is based on the definition of a transmission probability β between S and I, and a recovery probability µ. These parameters allow the definition of a basic reproduction number R 0 = k β/µ, where k is the average degree in a random network. In a well mixed population, or also if the underlying contact network can be assumed to be equivalent to a random graph, it is possible to prove that when R 0 < 1, the epidemics is expected to die out naturally very soon. As a consequence, this measure is widely adopted for assessing the danger of an infectious pathogen and to predict the likelihood for an epidemic to scale up as a pandemic, just comparing the empirically calculated value of R 0 with the so called epidemic threshold. However, it has also been proven that the epidemic threshold tends to 0 the more heterogeneous a network is, implying that any epidemics in a real global scale contact network is likely to escalate to a pandemic. Compartmental models are also used to run simulations to compare different immunisation and vaccination strategies: for the sake of simplicity, it is like testing the robustness of the network 'forcing' some given classes of nodes to the R (recovered) state, that is equivalent to remove such nodes from the transmission network.

Many social contagion models have been inspired by compartmental models, although epidemiological phenomena are inherently different to the way opinions, ideas, innovations, or ideas spread in a network. The simplest family of frameworks that try to capture how individuals' decisions can be influenced by means of peer pressure is the so called threshold model. In this scenario, nodes can be active or inactive; the state of an inactive node i flips when the number of its active neighbours is above a given threshold t. We can also consider a fractional activation condition instead of a linear one, just testing if a ratio of i's active neighbours is above a threshold with values in [0, 1]. The model can be even more complicated to better fit different and heterogeneous scenarios: for example, we can have a personalised threshold t i for each node, making some individuals less or more vulnerable to peer pressure. If a spreading phenomenon can be well modelled with such a framework, then clusters in a network are important barriers against social contagion: an innovation, for instance, will not penetrate into a densely connected community whose members continue adopting an obsolete technology.

Alternatively to models based on peer pressure, independent cascade models assume that social influence can also work on a one-to-one basis; in fact, a node can be, in principle, convinced by a single, very influential, individual. Another difference with threshold based approaches is that independent cascade models are probabilistic: active node i can convince an inactive node j with a given probability p ij . This makes more difficult to control a cascade, and clusters do not necessarily block propagation, but many realisations can be simulated to look for particularly interesting initial configurations.

Social contagion is also studied by means of many other models, that are variants of the threshold or the independent cascade models. Information diffusion, rumour spreading, opinion dynamics, and other phenomena that need more realistic assumptions, are usually better understood with more complicated models, and also validated with data retrieved from social media platforms. Nevertheless, it must be recalled here that social phenomena are often examples of the so called complex contagion, whose dynamics are not fully captured by the basic models described above. In many realistic settings, there is a fundamental difference between the spreading of rumours or jokes, and the diffusion of a valuable information, the adoption of a new behaviour or innovation, or social mobilisation: individuals usually need social reinforcement, i.e., they must be exposed to the same idea or product more than once, before changing their habits, or opinions. Therefore, under complex contagion, the structure of the network or the roles of nodes with high centrality may change substantially w.r.t. simple contagion processes: first, an innovation spreads faster in highly clustered network, even when the diameter is large, in contradiction to what observed with threshold models. Second, bridges on big structural holes can be ineffective to push innovation across different communities, contrary to what is expected for high betweenness nodes with infectious diseases. Third, hubs internal to clusters can be super spreaders, equivalently to what happens with epidemics.

Computational linguistics is the automatic processing of natural language. When the aim is to analyse large amounts of raw data and find relevant insights, text mining is applied. Combined with machine learning, it can help in developing text analysis models that learn to classify or extract specific information based on previous observations. A thorough and comprehensive explanation of the different aspects of the topic are presented among other, by Allen [7] and Manning and Schutze [140] .

The majority of the studies that have explored fake news detection from the perspective of computational linguistics are addressing the task as a text classification task. The aim of a classification task is to assign specific labels to observations based on a model that has been previously trained on known data. The first step of a text classification task is the collection of the data that need to be labelled. After this initial step, the preprocessing follows in order to transform the data in a form that systems can understand. Text mining systems use several Natural Language Processing (NLP) techniques, like tokenization, parsing, lemmatization, stemming and stop removal, to build the input for the classification model.

In particular, the data have to be transformed into vectors since this is what the algorithm can take as input. Vectors represent different features of the existing data. One of the most common approaches for vectorization the text is called bag of words, and consists on counting how many times a word appears in a text. Another way is to use the term frequency-inverse document frequency (TF-IDF) that also reflects how important a word is to a document in a specific collection. Word embeddings is a more sophisticated way to transform text into vectors. Word embeddings represent a word in a lower-dimensional space allowing words with similar meaning to have a similar representation. There are different approaches that can be applied to get word embeddings such as word2vec and Continuous Bowl of Words (CBOW). Another way is to use pre-trained word embeddings such as fastText and SpaCy.

Once there is a vectorized representation of a document, the vector can be fed into a machine learning (e.g., SVM, NB, Decision Tree) or deep learning algorithm (e.g., LSTM, CNN) algorithm which learns to make predictions. Machine learning algorithms build a model based on sample data, known as training data, in order to make predictions or decisions on unseen data. Deep learning is a subset of machine learning and structures algorithms in layers to create an artificial neural network. Recently, researchers also consider the Bidirectional Encoder Representations from Transformers (BERT) to address text classification tasks. BERT that is developed by Google, makes use of Transformer, an attention mechanism that learns contextual relations between words in a text. during Hurricane Sandy 2013 - Table 8 : Top-10 higher page rank papers in the library's periphery. Paper with no references are not covered in this survey.

In Table 7 , we show the top-10 papers in our library core with highest page rank. Page rank has already been used for finding "hidden gems" in scientific network: although some papers do not receive a huge number of citations such as other hubs, they can be referred as very influential by other very important contributions. Nevertheless, here we find that all the papers in this list have been already mentioned because of their high in-degree or betweenness scores, so that apparently page rank does not lead to surprising results. However, we must consider that this network measure works well in the medium-long term, because we need to have different layers of "important" papers that cite other papers. This scientific field is probably to young to give page rank more insights abilities than other simpler measures.

We observe a similar pattern even in Table 8 , that displays the 10 papers in the library's periphery with highest page rank. All the contributions have been already mentioned because of their high in-degree or betweenness, although at least a "hidden gem" is found: paper with id 36, that we cover in Sec. 5, is a well known contribution for scholars interested to the echo chambers problem, because it reports the results of a fundamental experiment run at Facebook that shed new lights in the interpretation of opinion polarisation and the role of filter bubbles in social media. The other "new" paper (id 37) is a relevant contribution for automatic deception detection, that is out of the scope of this survey.

Finally, it is worth mentioning that we also calculated closeness for our papers in the core library, to find contributions that are "topologically" more central than others in the network (see Table 9 ). All the found contributions are very recent (with still little citations), and with a quite technical content that is mainly related to automatic fake-news detection proposals. We did not cover these papers, even if we are happy to suggest to survey from Bagri et al. [59] (id 40) to the reader interested to an up-to-date review on fake news detection techniques.

Spread of (mis) information in social networks

The political blogosphere and the 2004 u.s. election: Divided they blog

Linguistic cues to deception: Identifying political trolls on social media

People are strange when you're a stranger: Impact and influence of bots on social networks

Twitter turing test: Identifying social machines

Social media and fake news in the 2016 election

Natural language understanding

How many bots in russian troll tweets? Information Processing & Management

The rise of partisanship and super-cooperators in the u.s. house of representatives

The Hype Machine: How Social Media Disrupts Our Elections, Our Economy, and Our Health-and How We Must Adapt. Currency

Predicting the role of political trolls in social media

Exposure to opposing views on social media can increase political polarization

Fake news and the economy of emotions

Exposure to ideologically diverse news and opinion on facebook

The role of social networks in information diffusion

Predicting factuality of reporting and bias of news media sources

Network Science

Tweeting from left to right: Is online political communication more than an echo chamber?

Dynamical Processes on Complex Networks

See it with feeling: Affective predictions during object perception

Lies, damn lies, and fake news

Personality traits and echo chambers on facebook

Social bots distort the 2016 us presidential election online discussion

Viral misinformation: The role of homophily and polarization

Users polarization on facebook and youtube

Fast unfolding of communities in large networks

Emergence of influential spreaders in modified rumor models

The social structure of political echo chambers: Variation in ideological homophily in online networks

Weaponized health communication: Twitter bots and russian trolls amplify the vaccine debate. American journal of public health

Echo chamber? what echo chamber? reviewing the evidence

An ensemble model using n-grams and statistical features to identify fake news spreaders on twitter

The hypercorrection effect persists over a week, but high-confidence errors return

Behavior enhanced deep bot detection in social media

Statistical physics of social dynamics

Information credibility on twitter

The spread of behavior in an online social network experiment

Complex contagions and the weakness of long ties

Rumor spreading in social networks

Computational fact checking from knowledge networks

The echo chamber effect on social media

Echo chamber or public sphere? predicting political orientation and measuring political homophily in twitter using big data

Political polarization on twitter

Automatic deception detection: Methods for finding fake news

Falling into the echo chamber: the italian vaccination debate on twitter

A decade of social bot detection

The paradigm-shift of social spambots: Evidence, theories, and tools for the arms race

Epidemics and rumours

Biased assimilation, homophily, and the dynamics of polarization. Proceedings of the National Academy of Sciences

The Third-Person Effect in Communication

Unifying markov chain approach for disease and rumor spreading in complex networks

Mixing beliefs among interacting agents

Reaching a consensus

The spreading of misinformation online

Pre-training of deep bidirectional transformers for language understanding

Social acceptance and rejection: The sweet and the bitter

Using sentiment to detect bots on twitter: Are humans more opinionated than bots?

The real story of 'fake news

Have american's social attitudes become more polarized?

A survey on the fusion process in opinion dynamics

The echo chamber is overstated: the moderating effect of political interest and diverse media. Information

Networks, Crowds, and Markets: Reasoning About a Highly Connected World

Reminders and repetition of misinformation:: Helping or hindering its retraction

He did it! she did it! no, she did not! multiple causal explanations and the continued influence of misinformation

What is disinformation? Library Trends

The rise of social bots

Measuring emotional contagion in social media

A theory of social comparison processes

Disinformation: The use of false information. Minds and Machines

Political polarization in the american public

Filter Bubbles, Echo Chambers, and Online News Consumption

Brave.net.world: The internet as a disinformation superhighway?

Selective exposure11the preparation of this paper was supported in part by nsf grants to the authors

Russian twitter disinformation campaigns reach across the american political spectrum. Misinformation Review

Disinformation as political communication

A formal theory of social power

Social influence networks and opinion change

Social influence and opinions

The rise and fall of 'social bot' research. SSRN

Political Discourse on Social Media: Echo Chambers, Gatekeepers, and the Price of Bipartisanship

Fake news: A definition

The triple-filter bubble: Using agent-based modelling to test a metatheoretical framework for the emergence of filter bubbles and echo chambers

Textrolls: identifying russian trolls on twitter from a textual perspective

Factweet: profiling fake news twitter accounts

Fakeflow: Fake news detection by modeling the flow of affective information

An Emotional Analysis of False Information in Social Media and News Articles

Detection of Conspiracy Propagators using Psycho-linguistic Characteristics

The Role of Personality and Linguistic Patterns in Discriminating Between Fake News Spreaders and Fact Checkers

Leveraging Emotional Signals for Credibility Detection

Multimodal Fake News Detection with Textual, Visual and Semantic Information

Multimodal multi-image fake news detection

Coevolution of agents and networks: Opinion spreading and community disconnection

Of bots and humans (on twitter)

Stweeler: A framework for twitter bot analysis

Time-dependent statistics of the ising model

Competition-induced criticality in a model of meme popularity

Social media, sentiment and public opinions: Evidence from #brexit and #uselection. Political Behavior: Voting & Public Opinion eJournal

Epidemic dynamics on an adaptive network

Avoiding the echo chamber about echo chambers: Why selective exposure to like-minded political news is less prevalent than you think

Information diffusion in online social networks: A survey

Dean: Learning dual emotion for fake news detection on social media

Burst of the filter bubble?: Effects of personalization on the diversity of google news

A criterion for unanimity in french's theory of social power

Frequency and the conference of referential validity

Why do people oppose mask wearing? A comprehensive analysis of U.S. tweets during the COVID-19 pandemic

Opinion dynamics and bounded confidence, models, analysis and simulation

Nonequilibrium phase transition in the coevolution of networks and opinions

Political polarization on twitter: Implications for the use of social media in digital governments

This just in: Fake news packs a lot in title, uses simpler, repetitive content in text body, more similar to satire than real news

The coverage of microsoft academic: Analyzing the publication output of a university

Opinion and community formation in coevolving networks

Red media, blue media: Evidence of ideological selectivity in media use

Epidemiological modeling of news and rumors on twitter

Novel visual and statistical image features for microblogs news verification

Defining "fake news

Defining "fake news

Automatic fake news detection model based on bidirectional encoder representations from transformers (bert). Applied Sciences

Fakebert: Fake news detection in social media with a bert-based deep learning approach

Fully automated fact checking using external sources

MVAE: Multimodal Variational Autoencoder for Fake News Detection

Deep neural networks for bot detection

Prominent features of rumor propagation in online social media

#brexit: Leave or remain? the role of user's community and diachronic evolution on stance detection

Stance polarity in political debates: A diachronic perspective of network homophily and conversations on twitter

The irony of satire: Political ideology and the motivation to see what you want to see in the colbert report

The science of fake news

Distributed representations of sentences and documents

Uncovering social spammers: social honeypots+ machine learning

Seven months with the devils: A long-term study of content polluters on twitter

The nature of reasoning. Number Sirsi) i9780521810906

Meme-tracking and the dynamics of the news cycle

Beyond misinformation: Understanding and coping with the "posttruth" era

Misinformation and its correction: Continued influence and successful debiasing

Troll factories: Manufacturing specialized disinformation on twitter

Semantic integration of verbal information into a visual memory

Continuous opinion dynamics under bounded confidence: a survey

Detecting rumors from microblogs with recurrent neural networks

Fake news and the public

Foundations of statistical natural language processing

Rise and fall patterns of information diffusion: Model and implications

Misleading postevent information and memory for events. arguments and evidence against memory impairment hypotheses

Birds of a feather: Homophily in social networks

A First Course in Network Science

Project academic knowledge

Efficient estimation of word representations in vector space

Many americans say made-up news is a critical problem that needs to be fixed

On the role of zealotry in the voter model

Dynamics of rumor spreading in complex networks

Contagion. The Review of Economic Studies

Introduction to the special issue on fake news: Definitions and approaches

Encouraging reading of diverse political viewpoints with a browser widget

Do not blame it on the algorithm: an empirical assessment of multiple recommender systems and their impact on content diversity. Information

New evidence about the existence of a bandwagon effect in the opinion formation process

Networks: An Introduction

Confirmation bias: A ubiquitous phenomenon in many guises

Measuring online social bubbles

When corrections fail: The persistence of political misperceptions

Information disorder: Toward an interdisciplinary framework for research and policy making

The role of individual differences in the accuracy of confidence judgments

The Filter Bubble: What the Internet Is Hiding from You. Penguin Group , The

Comparison of microsoft academic (graph) with web of science, scopus and google scholar

The russian "firehose of falsehood" propaganda model

GloVe: Global vectors for word representation

Prior exposure increases perceived accuracy of fake news

Automatic detection of fake news

Modelling opinion dynamics in the age of algorithmic personalisation

Dissecting contextual word embeddings: Architecture and representation

Using n-grams to detect fake news spreaders on twitter

Declare: Debunking fake news and false claims using evidence-aware deep learning

You don't know me, but i know you: The illusion of asymmetric insight

Overview of the 8th Author Profiling Task at PAN 2020: Profiling Fake News Spreaders on Twitter

Overview of the 7th author profiling task at pan 2019: Bots and gender profiling in twitter

Truth of varying shades: Analyzing language in fake news and political fact-checking

The false positive problem of automatic bot detection in social science research

The "false consensus effect": An egocentric bias in social perception and attribution processes

Naive Realism: Implications for Social Conflict and Misunderstanding

Deception detection for news: Three types of fakes

Social influence and unfollowing accelerate the emergence of echo chambers

Fake News Propagation and Mitigation Techniques: A Survey

Detection of Novel Social Bots by Ensembles of Specialized Classifiers

Dynamic models of segregation

We are the people and you are fake news: A social identity approach to populist citizens' false consensus and hostile media perceptions

The spread of low-credibility content by social bots

Fake news detection on social media: A data mining perspective

Understanding User Profiles on Social Media for Fake News Detection

SpotFake: A Multi-modal Framework for Fake News Detection

Fake news and ideological polarization: Filter bubbles and selective exposure on social media

On the difference or equality of information, misinformation, and disinformation: A critical research perspective

Bots increase exposure to negative and inflammatory content in online social systems

The darpa twitter bot challenge

The law of group polarization

Network segregation in a model of misinformation and fact-checking

Fact-checking strategies to limit urban legends spreading in a segregated society

Fact-checking effect on viral hoaxes: A model of misinformation spread in social networks

Party polarization in the us congress: Member replacement and member adaptation

fake news' and disinformation: A handbook for journalism education and training

Truth by repetition: Explanations and implications

Online human-bot interactions: Detection, estimation, and characterization

Modeling confirmation bias and polarization

Immigration as a divisive topic: Clusters and content diffusion in the italian twitter debate

The impact of disinformation on a controversial debate on social media

Learning from Fact-checkers: Analysis and Generation of Fact-checking Language

The spread of true and false news online

liar, liar pants on fire": A new benchmark dataset for fake news detection

EANN: Event Adversarial Neural Networks for Multi-modal Fake News Detection

On the failure to eliminate hypotheses in a conceptual task

Emotions, partisanship, and misperceptions: How anger and anxiety moderate the effect of partisan bias on susceptibility to political misinformation

Competition among memes in a world with limited attention

Topicality and impact in social media: Diverse messages, focused messengers

Network analysis reveals open forums and echo chambers in social media discussions of climate change

Cognitive psychology and emotional disorders

Misinformation in social media: Definition, manipulation, and detection

Scalable and generalizable social bot detection through data selection

Rumor spreading model considering forgetting and remembering mechanisms in inhomogeneous networks

Fake news early detection: A theory-driven model

A survey of fake news: Fundamental theories, detection methods, and opportunities

Fact-checking meets fauxtography: Verifying claims about images

Debunking in a world of tribes

Verbal and nonverbal communication of deception

When Corrections Fail: The Persistence of Political Misperceptions

Semantic Integration of Verbal Information Into a Visual Memory

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Detecting rumors from microblogs with recurrent neural networks

Misleading postevent information and memory for events: arguments and evidence against memory impairment hypotheses

Glove: Global Vectors for Word Representation

Deception detection for news: three types of fakes

Faking Sandy: characterizing and identifying fake images on Twitter during Hurricane

Prominent Features of Rumor Propagation in Online Social Media

Top-10 higher in-degree papers in the library's periphery

* Misinformation and Its Correction: Continued Influence and Successful Debiasing

* Fake News Detection on Social Media: A Data Mining Perspective

Reducing Misinformation Effects in Children With Cognitive Interviews: Dissociating Recollection and Familiarity

Reminders and Repetition of Misinformation: Helping or Hindering Its Retraction?

Disinformation on the Web: Impact, Characteristics, and Detection of Wikipedia Hoaxes

He did it! She did it! No, she did not! Multiple causal explanations and the continued influence of misinformation

* The science of fake news

Community identifiers, as defined in Sec.2, are shown as well

Die wahre Fälschung auf den Spuren von Arthur Schütz als Pionier der journalistischen Qualitätsforschung

Memory Distortion in Eyewitness Pairs Who Observed Nonconforming Events and Discussed Them 1996 -29 Human paleobiology: Contents 2000 -30 Who compares and despairs? The effect of social comparison orientation on social media use and its outcomes 2015 -31 5-valued Non-deterministic Semantics for The Basic Paraconsistent Logic mCi 2008 -32 Identity in Narrative: A study of immigrant discourse

Algorithmic Public Relations: Materiality, Technology and Power in a Post-Hegemonic World

Collaborative Recall Reduces the Effect of a Misleading Post Event Narrative

Similarity Measures for Text Document Clustering

* The science of fake news

Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection

Automatic deception detection: methods for finding fake news

Fake News": A typology of scholarly

* Misinformation and Its Correction: Continued Influence and Successful Debiasing

* The rise of social bots

Community identifiers, as defined in Sec.2, are shown as well. All the papers in this list have already been mentioned in the top-10 ranks based on in-degree or betweenness

When Corrections Fail: The Persistence of Political Misperceptions

* Deception detection for news: three types of fakes

Exposure to ideologically diverse news and opinion on Facebook

Pre-training of Deep Bidirectional Transformers for Language Understanding

Global Vectors for Word Representation

Syntactic Stylometry for Deception Detection

* Detecting rumors from microblogs with recurrent neural networks

Weaponized Health Communication: Twitter Bots and Russian Trolls Amplify the Vaccine Debate

* Faking Sandy: characterizing and identifying fake images on Twitter

 Table 3 : Top-10 higher in-degree papers in the core library. Community identifiers, as defined in Sec.2, are shown as well. Paper with no references are not covered in this survey.The algorithms are trained on a set that is called training set. A validation set is also usually used to optimise the parameters of the model. The performance of the classification model is evaluated on test documents and is measured with accuracy, recall, precision and F1-metric.

In this appendix, we follow up the analysis described in Sec. 2 to list all the papers with high centrality values (indegree, betweenness, page rank, and closeness). For each measure, we ranked the papers corresponding to nodes in the "Fake News Reserch" library network, and in the following tables we show the top-10 for every rank. This procedure is executed for both the core and the periphery of the network. Of course, this information can be outdated after some time because of new relevant papers added to the library and to varying citation dynamics, so we recommend the reader to use the search engine at http://fakenewsresearch.net for updated information and ranks.In-degree is a network measure of "popularity": if a paper receives many citations, it is likely that the community is endorsing it as a relevant contribution to the field. In Tables 3 and 4 we show the top 10 papers ranked according in-degree respectively in the core and in the periphery of our library network. Many papers that are in these lists have been properly covered in our survey; in fact, in the core we find some successful contributions to the definition of a proper terminology and of the related research problems (papers with ids 4 and 7), or to the exploitation of network and/or computational linguistics based approaches (ids 2, 9, and 10) to find empirical evidence of the prevalence of fake news in social media, on the role of echo chambers, and the impact of social bots in spreading manipulated content. The difficulties of debunking and a set of recommendations on how correct misinformation are described in paper with id 6, grounded in cognitive psychology. In this list, we find also a reference to a benchmark dataset used to train fake news detection systems (id 3), and some recent surveys to automatic detection techniques (ids 5, and 8).Looking at the periphery (see Table 4 ) allows us, as expected, to go back to pioneering studies that preceded the boost of interest aroused after 2016 (ids 11 and 20), or to groundbreaking algorithms, methods and technologies that have been widely used in empirical studies, as BERT (id 14), and Glove (id 17). Again, we find other contributions related to the misinformation effect (ids 13, and 16), and to other cognitive biases that make correction of misinformation difficult (id 12). Finally, there are papers on the problem of detecting and classifying fake content: one proposing a neural network approach (id 15), a survey (id 18) and another on fake images (id 19), that is out of the scope of the present review.A node with high betweenness is likely to be a "bridge" in a graph. Given the clustered structure of our library network, that could also mean that corresponding papers can be or heavily cited (with high in-degree), or considered relevant for studies belonging to different disciplines, or both; in fact, here we find some studies on cognitive biases, such as papers with ids 6, 21, 23, and 25, confirming that explanations to misinformation are usually grounded on psychology and social psychology. Among some papers that already appeared in in-degree based ranks, in Table 5 we notice, quite interestingly, a technical contribution to automatic fact-checking (id 22) that adopts a mixed network/computational linguistics based approach, plus other papers that we are not covering in this survey for the already mentioned reasons.Papers with high betweenness in the periphery (see Table 6 ) were mainly out of the scope of this survey, being only loosely connected to our research problems. Table 9 : Top-10 higher closeness papers in the core library. Community identifiers, as defined in Sec. 2, are shown as well. Paper with no references are not covered in this survey.

We used Microsoft Academic Graphs API [145] to download papers and follow their references to build the "Fake News Research" library described and analysed in Sec. 2. Although it was announced the Microsoft Academic website and underlying APIs will be retired on Dec. 31, 2021, we will keep updating our repository, that is accessible and searchable via the platform available at http://fakenewsresearch.net.Network analysis was performed with most common tailored Python pachages (e.g., pandas, networkx, igraph), and graph plots (as in Fig. 2) were created with Gephi https://gephi.org].