key: cord-0121738-7giy6iof authors: Heuer, Hendrik title: Helping People Deal With Disinformation -- A Socio-Technical Perspective date: 2021-04-09 journal: nan DOI: nan sha: 6449b946adcd3d15c99122ee6ca1fce5b687c05a doc_id: 121738 cord_uid: 7giy6iof At the latest since the advent of the Internet, disinformation and conspiracy theories have become ubiquitous. Recent examples like QAnon and Pizzagate prove that false information can lead to real violence. In this motivation statement for the Workshop on Human Aspects of Misinformation at CHI 2021, I explain my research agenda focused on 1. why people believe in disinformation, 2. how people can be best supported in recognizing disinformation, and 3. what the potentials and risks of different tools designed to fight disinformation are. or deliberately manipulated content like conspiracy theories or rumors. As such, disinformation is a special case of misinformation. While disinformation is connected to an intent to harm, misinformation also includes unintentional mistakes. I believe that disinformation is a challenging, multifaceted phenomenon that requires an appropriate sociotechnical response. In my work, I research 1. why people believe in disinformation, 2. how people can be best supported in recognizing disinformation, and 3. what the potentials and risks of different tools designed to fight disinformation are. My work on disinformation is informed by my background in human-computer interaction and machine learning. The workshop would be a great opportunity for me to discuss the ethical implications of the disinformation detection solutions that I am developing. I would love to discuss ways of enabling fast and effective disinformation detection while ensuring that freedom of speech is protected. To understand disinformation in the contemporary media climate, one has to understand social media and the machine learning-based curation systems used on social media. My doctoral thesis provides a socio-technical perspective on users and machine learning-based curation systems [3] . The thesis presents actionable insights on how ML-based curation systems can and should be explained and audited. Motivated by the role that ML-based curation systems play in the dissemination of disinformation, I examined the user beliefs around such systems in detail. In a recent CSCW paper, I, together with my collaborators, examined how users without a technical background, who regularly Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). interact with YouTube's ML-based curation systems, think the system works [1] . Our semi-structured interviews with participants from Belgium, Costa Rica, and Germany show that users are aware of the existence of the recommendation system on YouTube, but that users' understanding of the system is limited. This has important consequences for the dissemination of disinformation. With my upcoming work on disinformation, I extend on previous work on how users quantify trust in news on social media [4] . In this paper, I identify factors that influence this trust and show that while the majority of users can provide nuanced ratings that correspond to ratings of media experts, a small number of extreme users tend to over-and under-trust in news. A follow-up work focused on the output of ML-based curation systems showed that users can provide trust ratings that distinguish trustworthy recommendations of quality news stories from untrustworthy recommendations [5] . However, a single untrustworthy news story combined with four trustworthy news stories is rated similarly as five trustworthy news stories. This could be the first indication that untrustworthy news stories benefit from appearing in a trustworthy context. To understand why people are vulnerable to disinformation and what people really need to recognize disinformation, I interviewed domain experts from different fields like media science, law, psychology, sociology, political science, and others. I asked the experts why they think people are susceptible to disinformation. My preliminary results show that a variety of influence factors is recognized by the different domain experts. Some experts say that people believe in disinformation because it aligns with their beliefs and because it speaks to their cognitive biases. Other experts think users may lack the education to recognize disinformation. Some users may be tempted to follow recommendations from their peers. Others may simply be overwhelmed by an increasingly complex world and a large amount of information. Certain people may even use disinformation intentionally for personal gain. Together with the domain experts, I developed a variety of solutions to the problems they described, both technical and non-technical. These solutions range from formal education and laws to the flagging of content or sources. The solutions include tools that automatically fact-check articles as well as chatbots that train people to recognize disinformation. In an upcoming paper, I map out the design space for solutions that help people recognize disinformation. Based on the solutions proposed by the experts, I am currently developing a number of other tools, e.g. to support users in assessing the source of a news article, to identify the author of a news article, and to support users in factchecking the content of an article. I will compare these tools to written checklists and to a setting where the user has no support. One proposed solution that I investigated in depth was a machine learning-based, automated flagging system that recognized articles based on style. Considering the nature of news as information about recent events, systems based on lexical features are not able to account for new concepts and the changing meaning of words. To mitigate these limitations of an ML-based approach focused on content, I developed a system that can detect disinformation based on the style of a news article, extending on previous lexical approaches [10, 12] . Developing a machine learning system that detects disinformation based on style, rather than content was motivated by previous research on the linguistic and stylistic signals related to misinformation and fact-checking [7, 8, 11] . The style-based system is able to detect disinformation based on stylistic features with F1-scores of 80 or higher. However, a pilot study in October 2020 provided evidence that explanations like "The average number of words per sentence is low", the "Usage of perceptual words related to hear is low" and "The amount of words related to people is high" are not perceived as helpful by users. Even educated participants had trouble understanding explanations of ML-based systems, a problem that I have previously reported on in the context of ML-based curation systems [3] and object recognition systems [6] . My preliminary results indicate that providing information on the reliability of the source of a news story is the most promising direction. The advantage of this approach is that is scalable (the number of new news sources appearing is limited), that it can be explained with little effort (based on the history of the news source), and that it can be integrated into the existing workflow of users. A disadvantage of the approach is that news sources frequently mix correct reporting and false reporting. Therefore, the source assessment as an all or nothing approach needs to be situated well. In addition to that, curating the list of reliable and unreliable news sources is a concentration of power and as such politically controversial. To ensure user acceptance, reliable and transparent governance models need to be established, akin to what Wikipedia achieved. I am currently developing a browser extension that augments the interface of existing websites like Facebook and Twitter, which I am planning to evaluate empirically in user studies. I want to understand why people believe in disinformation. Based on my theoretical insights, I want to design and develop solutions that support people in recognizing disinformation. My goal is to provide a taxonomy of influence factors that make people prone to believe in disinformation, thus providing a theoretical foundation for the fight against disinformation. I would be happy to discuss the potentials and risks of the different solutions. Middle-Aged Video Consumers' Beliefs About Algorithmic Recommendations on YouTube The Age of Information Disorder Users & Machine Learning-based Curation Systems Trust in News on Social Media How Fake News Affect Trust in the Output of a Machine Learning System for News Curation More Than Accuracy: Towards Trustworthy Machine Learning Interfaces for Object Recognition Linguistic Signals under Misinformation and Fact-Checking: Evidence from User Comments on Social Media Style Matters! Investigating Linguistic Style in Online Communities Misinformation and Its Correction: Continued Influence and Successful Debiasing Automatic Detection of Fake News The Limitations of Stylometry for Detecting Machine-Generated Fake News Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection Managing the COVID-19 infodemic: Promoting healthy behaviours and mitigating the harm from misinformation and disinformation