key: cord-0426370-y1avlgx6
authors: Fischer, Maximilian T.; Hirsbrunner, Simon David; Jentner, Wolfgang; Miller, Matthias; Keim, Daniel A.; Helm, Paula
title: Promoting Ethical Awareness in Communication Analysis: Investigating Potentials and Limits of Visual Analytics for Intelligence Applications
date: 2022-03-18
journal: nan
DOI: 10.1145/3531146.3533151
sha: a05e5b8e84f4e319a3f16aa1651cc31a5c37b645
doc_id: 426370
cord_uid: y1avlgx6

Digital systems for analyzing human communication data have become prevalent in recent years. Intelligence analysis of communications data in investigative journalism, criminal intelligence, and law present particularly interesting cases, as they must take into account the often highly sensitive properties of the underlying operations and data. At the same time, these are areas where increasingly automated, sophisticated approaches systems can be particularly relevant, especially in terms of Big Data manageability. However, by the shifting of responsibilities, this also poses dangers. In addition to privacy concerns, these dangers relate to uncertain or poor data quality, leading to discrimination and potentially misleading insights. Visual analytics combines machine learning methods with interactive visual interfaces to enable human sense- and decision-making. This technique can be key for designing and operating meaningful interactive communication analysis systems that consider these ethical challenges. In this interdisciplinary work, a joint endeavor of computer scientists, ethicists, and scholars in Science&Technology Studies, we investigate and evaluate opportunities and risks involved in using Visual analytics approaches for communication analysis in intelligence applications in particular. We introduce, at first, the common technological systems used in communication analysis, further discussing the domain-specific ethical implications, tensions, and risks involved. We then make the case of how tailored Visual Analytics approaches may reduce and mitigate the described problems, both theoretically and through practical examples. We show that finding Visual Analytics design solutions for ethical issues is not a mere optimization task, but balancing out and negotiating these trade-offs has, as we argue, to be an integral aspect of the system design process from the outset.

In recent years, the share of human communication transitioning to digital forms has increased significantly, while the advances in big data analysis offer new opportunities concerning the amount of arXiv:2203.09859v2 [cs.HC] 2 May 2022 meaningful information and knowledge that can be harnessed from this data. Consequently, advanced digital systems for analyzing such communication have simultaneously become increasingly prevalent, especially due to the difficulty in managing such large and diverse information either manually or with only rudimentary analysis capabilities. Novel approaches allow for more effective handling, performing sophisticated big data analyses using methods such as social network analysis [92] , meta-data screening [22] , pattern matching [44] , or natural language processing [24] , often assisted through machine learning [33] .

One of the most prevailing domains for such systems is the analysis of communications data for intelligence purposes, namely in criminal investigations, lawsuits, matters of national and international security and in investigative journalism. Specialized systems are used, for example, by the National Security Agency (NSA) as part of global spying operations [77] or law enforcement against organized crime [23] , by lawyers for analyzing case-relevant documents [2] , but also by journalists working on [103] large data leaks such as the Panama Papers. During these operations, large amounts of communication data, like e-mails, chats, posts, or calls are collected, along with associated documents (e.g., attachments) and meta-data like timestamps, locations, and contact networks. In our research context, these domains present a particularly interesting case, as they should consider the often highly sensitive and private character of the underlying operations and data with particular caution. There is no doubt that untargeted mass collection of communication in the name of national security is privacy-invasive and thus highly controversial [71] . However, many ethical challenges remain even relevant for morally more accepted cases like specifically targeted analysis of confiscated organized crime equipment or even practices considered essential to democratic culture and particularly valuable, like data journalism.

For example, privacy issues relating to the separation of irrelevant data, its secure handling, analysis, and deletion have to be considered [6, 10] . Poor data quality, unreliable methods, or biased algorithms may lead to misleading insights [3] , bearing the risk of overlooking critical information, or worse, contribute to discriminatory practices against people of colour [16] , cement existing social inequalities [26] , and may even result in false accusations [81] . Moreover, a lack of transparency can make it impossible to defend oneself against such accusations [94] when the systems supporting (or making) such decisions are considered reliable, but actually do not (consistently) provide complete chains of evidence [61] . Unfortunately, being focused on efficiency and quick results, not all actors consider the arising ethical challenges in this field with a long-term perspective in mind, and even those who try, may be limited by their technical approach and implementation difficulties, for example, through black-box machine learning models or an incomplete understanding of the considerations involved in deriving the result [104] .

The concept of accountability in computer science [78] stresses the need to handle and answer to the harms and risks that can be caused by technology. While this concept has been around for decades already, concrete ways of handling accountability in digital analytical systems have remained vague. Yet, progress has been made in interpretability of machine learning [76, 88] . Whereas the coming into effect of the EU General Data Protection Regulation (GDPR) lead to more awareness on this topic and its implementation [49] , its legal effects on the fields under consideration are limited due to exception clauses in article 2 (law enforcement) and article 85 (journalism) [30] . Even in those areas, however, the need to consider these topics carefully is increasingly prevailing.

Working together closely with criminal investigators from various institutions, we know of a growing awareness of these difficulties, also on the part of the analysts themselves. Concerns about the trustworthiness of their analytics systems and the ethical considerations involved have been expressed. This concern is also reflected in digital communication analytics in general [101] , where the need for more detailed analysis was identified. To date, ethical concerns related to automated communications analysis have been described mainly from either a strictly sociological and/or ethical perspective [46] or in the context of technical capabilities [34, 36, 101] . Much less work, by contrast, addresses the complex techno-ethical tensions and dilemmas that arise in the messy gray areas of sociotechnical feasibility, given the limits and consequences induced by the alleged solutions. Given the lack of overarching work, in this paper we examine ethical considerations in communications analysis for intelligence applications in more detail and propose possible mitigation techniques, which we discuss critically with regard to ethical concerns. Unlike most previous research, we thus bridge ethical considerations, sociological science & technology studies, and a computer science perspective.

We study concrete design approaches and solutions and by analyzing the interfacing problem from an interdisciplinary perspective, we can critically reflect on the opportunities and challenges involved [66] . We argue that designing solutions is not a mere optimization task, but balancing out and negotiating the trade-offs has to become an integral aspect of the design process at the very outset. We further claim that -in light of recent requirements for human oversight [28, 29] -an application design based on visual analytics principles is uniquely suited for such a task:

Visual analytics (VA) [57, 58] combines machine learning methods with interactive visual interfaces to enable human sense-and decision-making. VA features interactive visualization and data processing elements that analysts can engage in, shape and control, aiming at interactively combining human sense-and decisionmaking with computational power, leveraged through a frequent feedback loop. The overall iterative analysis process in VA [57] can be summarized as an explicit process model [89] for knowledge generation. It shows how a user is supported at every stage of the visual data analysis process from exploratory analysis over verification (confirmatory) of hypothesis to knowledge generation and sets this in relation to human sense-making, the information visualization pipeline [17] and the knowledge discovery in databases process [38] from data science. Through these aspects, VA can better handle ill-defined or open-ended tasks -which often occur in criminal intelligence, law, or investigative journalism -than fullyautomated systems. Through human oversight, VA is well-suited for designing and operating meaningful interactive communication analysis systems that consider the ethical challenges outlined above and which we discuss in more detail in the following sections, ease their technical implementation, and allow for an interactive and iterative knowledge generation process [34] .

With this work, we aim to promote ethical awareness of digital communication analysis by turning our focus to the interface, investigating the potentials and limitations of visual analysis for intelligence applications, contributing:

• A detailed discussion on the ethical frictions and tensions involved in intelligence applications, followed by a scenariobased stakeholder analysis of actors and their roles. • A critical reflection of visual analytics design solutions fostering ethical awareness in communication analysis and the involved trade-offs as an integral part of the interface design process.

With this contribution, we aim to inform more ethically-aware approaches to communication analysis in intelligence operations using visual analytics principles.

AI ethics is an expanding field that both drives and responds to the various concerns associated with increasing datafication and automation. Novel technologies used in police intelligence are receiving particular attention in this context, as they are seen as extraordinarily problematic [94] . Their sometimes ill-advised use is being critiqued by both ethics scholars and civil society actors, as illustrated by the fierce backlash against so-called predictive policing technologies (PPTs) [3] . Several key challenges have been identified in the context of this growing debate. Below, we discuss those that are particularly relevant to communications analysis. C1. Discriminatory bias -One of the most urgent challenges are biased algorithms reproducing existing stereotypes and aggravating discrimination of communities (e.g., women, people of color, transgender) [16, 20, 69, 79] . Problematic bias in machine learning may be introduced by human actors (e.g., suspicion based solely on ethnic belonging) or inserted through processes (e.g., replicating stereotypes represented in the training data), for example, in facial recognition [41] systems and pre-trained language models [63] . Correspondingly, fairness principles and metrics are developed to evaluate and mitigate statistical discrimination in AI models [14, 74] . These however may not be enough in societies that suffer from unfair conditions already and require equity before automation [43] .

C2. Privacy -Privacy has been and remains to be a core challenge. Intelligence applications process enormous amounts of data collected from heterogeneous sources. This may include seized data like recorded phone calls, but also online messengers and social media. Due to quantity and heterogeneity, the majority of information is on unrelated third-parties. The storing and processing of such information therefore triggers important legal and ethical questions [75] . While law enforcement is exempt from many legal frameworks regulating data protection and privacy (most notably the GDPR [30] ), this does not give police authorities a free pass. It instead puts more weight on the responsibility to safeguard against misuse of gathered information. Access to such sensible data, for example, has to be clearly defined, explained, and technically implemented in a robust way.

C3. Opacity -The black-box architecture of many deep-learning systems is another novel challenge [84] . Even if the systems are in principle explainable and/or interpretable by experts in the field, analysts are generally not such experts. It is therefore difficult for them to fathom how the systems they use generate their output [8] . In addition, there are the typical problems of public-private partnerships, such as when algorithms used in public domains fall under trade secret protection. This can lead to severe problems, as analysts, judges, the public in general, and those directly affected by the results have little opportunity to challenge the findings. However, these may well be biased or inaccurate. Sometimes with dramatic consequences. [69, 79, 81] .

C4. Exaggerated Expectations -Another strand of criticism deals with the connotation that algorithmic recommendations cannot be disputed because they are mathematical truths. In contrast to the black-box discourse, the discourse that addresses this misunderstanding focuses on the problem of trustworthiness not primarily as a technical, but as an ideological one. Highlighting the opportunities of innovative technologies should not result in practices of "mathwashing" where software is used as a panacea against error-prone or malicious activities of humans [54] . This imaginary is triggered by the portrayal as charismatic machines [4, 5] , with so-called predictive policing technologies being a prime example of this mismatch between promise and actual capability [46] . To counter imaginaries of the flawless machine, STS scholars draw attention to the various forms of subjectivity and intention that are woven into systems and result from collective decisions made by a variety of actors with diverse, potentially conflicting interests [61, 70] . Given the collective work and design decisions that underlie it, communications analysis of intelligence data is anything but neutral [42] . When people belief in the neutrality of tools that in fact serve particular needs and interests, they are deprived of their ability to question the results. This "erasure of doubt" is deeply problematic because it impedes reasoned trust grown from education and experience [7] .

C5. Human-Machine-Configurations -With advanced automation, human machine configurations grow ever more complex [100] . The challenge here is to accomplish good integration and agree on an appropriate level of automation. The goal of automated analysis is to assist investigators by relieving them of scanning through irrelevant and mundane patterns so that they can focus on more useful activities [19] . Sometimes, however, analysts perceive their machine assistants as competition rather than help, and feel disregarded and displaced in their human experience [56] . The critical question is how much automation should be used, how far it should go, and how it should relate to human investigators' activities [99] . What operations need to be supported and guided, for example, by recommending alternative search terms or related individuals, and what agency should the analyst retain in interpreting the machine output? How much and what contextual information should be displayed, how much should be hidden for the sake of privacy and to what extent should interfaces be designed to encourage the user to consider ethical issues, e.g., through nudging? Smooth and well-structured collaboration is a much-discussed topic, as it is seen as a necessary safeguard against AI systems that may undermine human autonomy [27] with sometime detrimental effects.

C6. Accountability -The ethical issues mentioned so far subsequently trigger questions of accountability of the software, its designers, and users. Accountability refers to the willingness or obligation to assume responsibility for actions and decisions of AI systems [1, 65] . In the context of AI, it has to be decided to what degree system users can and should be held accountable for consequential mistakes made if the software failed to meet basic standards of explainability and interpretability. It also needs to be specified what obligations the software provider has to safeguard against ethically-problematic decisions (racially-biased categorization of suspects) and usage (spying on third-parties).

As a prerequisite for the following discussion, we first provide a overview of communication analysis and the common technological systems, before presenting the PEGASUS research project as a case study. We then construct a hypothetical scenario, from which we derive a map of the stakeholders in conflict, forming the basis for our proposition for mitigation.

Digital communication analysis as a research field has no universally accepted definition, with different understandings in different domains. In this work, we follow the definition by Fischer et al. [34] , considering it to encompass the computer-mediated [35] analysis of meaningful digital [91] information exchanges between humans [85] . The analysis relates not only to the actual content (text, audio, or video), but also encompasses accompanying metadata as well as communication network structures. Existing communication analysis approaches rarely consider these aspects holistically [35] , but primarily focus on individual aspects: Most commonly, these are textual analysis through fuzzy search (and increasingly natural language processing (NLP) [72] methods) as well as social network analysis [92] . For example, in intelligence, one of the most commonly used systems [33] is IBM's i2 Analysts Notebook [51] , which has a strong focus on network analysis and information management but has, so far, lacked advanced textual analysis capabilities. However, competing solutions such as Nuix [80] , DataWalk [21] and Palantir Gotham [82] have been gaining ground [35] . Many are primarily large information management systems, using established algorithms (e.g., for centrality calculations in a network) and deterministic filters (e.g., keywords). Novel machine learning-based capabilities used for relevance scoring, person attribution, or facial matching are increasingly used in this context. The reliability of these models, however, the question of hidden bias, and the overall reproducibility (e.g., after updates), remain unclear.

In investigative journalism, tools like New/s/leak 2.0 [103] , as used by Der SPIEGEL, use models trained on public data like Wikipedia for discovering named entities in textual data (e.g., persons or company names). Similarity, the industry-standard spacy [48] uses public corpora and increasingly open web information for model training. While this often results in increased accuracy, concerns about the reliability for less common languages or risks of manipulation (e.g., for datasets extracted from Wikipedia) remain valid.

For a case study on the requirements in intelligence, we specifically focus on the insights gathered through the work in the academic research project PEGASUS, funded by the Federal Ministry of Education and Research of Germany (BMBF). The project aims at improving big data analysis in the context of civil security, also considering the ethical challenges involved. The PEGASUS acronymnot to be confused with the unfortunately equally named PEGASUS spyware -stands for Collection and analysis of heterogeneous Big Data by the police to fight organized crime structures. Organized crime is a transnational and global form of crime, encompassing a broad spectrum of different areas, including human, drug, and arms trafficking, money laundering, smuggling operations, environmental, medical, cyber, and other white-collar crimes. According to Europol, in Europe alone, the number of criminal organizations under investigation is over 5000 (2017) [31] , coming with high economic cost and a destabilizing effect on public security (through, for example, extortion, fraud, trafficking, or bodily harm). Organized crime can be characterized by its organized hierarchies (e.g., clans, mafia structures, shell companies) and sophisticated criminal acts using modern technology, and their ability to adapt quickly to changing circumstances [83] . For example, the COVID-19 pandemic has significantly affected organized crime, which was quick to adapt to new illegal avenues and modi operandi [31] . In conjunction, the seized data is increasing massively, overwhelming traditional (primarily manual) investigation methods. A significant share accumulates as intra-and inter-group communication and can be acquired, for example, when electronic devices are seized. However, the challenges faced are not unique to law enforcement; the goals are strikingly similar to tasks in fields such as investigative journalism and business intelligence, where information and the knowledge derived from it have become more important than natural resources [60] . Tackling the arising ethical issues is challenging because mitigation techniques incorporate numerous tensions and dilemmas that must be carefully weighed between the complex interplay of actively and passively involved stakeholders.

We construct a hypothetical scenario [18] of communication analysis within police intelligence work using current but non-visual analytics software that acts as a reference for our study of ethical challenges and emerging mitigation strategies. The scenario focuses on the challenges and practices of police officers and investigators as the main user community and points out other actors and stakeholders (highlighted as S Name ) in an exemplary way.

S Martin is a police officer at the organized crime unit of the federal police. He currently investigates the selling of fake COVID-19 vaccination passports by an alleged criminal organization named The Medics. The Medics offer the counterfeit certificates to their S customers via the Telegram messenger. Unknown to Martin yet, S Chris , S Carlos , and S Eggert are Medics members, also communicating with their colleagues and suppliers via group Telegram channels while using pseudonyms, sometimes coded language, and images. In their free time, they also communicate with several friends, including their girlfriends, S Sarah and S Marta , who are unaware of their business. Martin's police unit gathers much information about The Medics using traditional investigative methods. This information leads to the identification of the suspect, Chris, who seems to be a low-level member of The Medics. On one evening, Chris is found with blank vaccination certificates during a traffic stop. He is arrested, and his phone is seized by investigator Martin, who aims at using the information on the phone to track down the individuals pulling the strings. After calling judge S Robert to get a search warrant, which is granted, he then searches Chris's unprotected phone, finds the Telegram communication, and extracts it. He recalls that his superior, S Dr. D , asked him to try out the new AutoCommAnalyzer software, which was recently purchased from the multinational company AI-Tech Corp. The software purchase was part of a strong push by the government to digitally optimize work processes at the police forces. Martin looks at the training notes by the head developer S Molly , trying to remember how the machine learning-driven software -trained with texts by S Alf and S Bert -is supposed to direct him to the relevant communication. The software presents him with the most frequent contacts, with Sarah on top. He reads through this communication, as the software has flagged several words like package and hospital, discovering some explicit images but finding that the flags refer to a delivery package and a hospital stay for a broken ankle. In a second chat, the AI highlighted several currency amounts, and manually reading through it, it becomes clear that expensive "stamps" have been sold. Luckily for him, many addresses and names are also included in the chat messages. Searching for all chats that talk about stamp selling, he also finds one with Carlos, including his last name, and one from a person called Big E, which includes an address. Using the nationwide register, he finds a person named Carlos, who used to be a roommate with Chris, and only one person named Eggert is living at the found address. After completing his analysis, he finishes his report and submits it for the trial. During the court proceedings weeks later, Martin is questioned by judge S Muller on his findings. Ultimately Chris, Carlos, and Eggert admit to their guilt and are sentenced for document falsification. The intelligence gathered points to other alleged criminal networks and informs other running investigations.

A stakeholder analysis based on the previous scenario helps to identify, map and describe the different actors and their roles involved in the scenario (see Figure 1) , with the interdependent individuals having potentially conflicting interests. We propose four main groups of stakeholders: civil society, governmental authorities, software provider, and data subjects. This categorization has to be understood as a heuristic with possible overlaps and without claims of being exhaustive. Data Subjects -Following the concept of data subjects defined by the GDPR [30] , we consider the role of natural persons and their data ownership. Immediately apparent becomes the role of the targets ( S Chris ). In many cases, a target is unknown, but one has a list of suspected targets ( S Carlos ), indicating a different degree of certainty. One issue of communication analysis, however, is that communication is not strictly separated and touches on many other data subjects. These can be as of yet unidentified persons (e.g., the customers), but also third parties (e.g., S Sarah ). With the use of machine learning, a fifth subgroup emerges, the training data subjects ( S Alf and S Bert ), whose data is leveraged as part of training the weights in neural networks. Further conflicts of interest arise between uninvolved third parties who usually (and rightly) do not want to be involved in a privacy-invasive investigation, which can also apply to (unwitting) training data subjects. A delicate issue are privacy considerations in the face of imminent suspicion: while target subjects clearly do not want to be investigated either, the reasons and justifications here differ substantially to those of uninvolved third-parties.

Governmental Authorities -In this specific type of intelligence analysis, the opposite of the data subjects are governmental authorities, with their investigating bodies. Here, because this applies to our setting, we assume a democratic political system that follows a separation of executive, legislative and judicial powers. The investigating bodies are primarily part of the executive, with police analysts (users) ( S Martin ) conducting the investigation, overseen by police leadership ( S Dr. D ) and also controlled by regulators like data protection or compliance offices. The judiciary, however, also plays a controlling role during preliminary investigations ( S Robert ), allowing for specific measures, for example, by issuing a warrant. Later, it manages court proceedings ( S Muller ) and questions of legality can ultimately be decided on a constitutional level. The third power, legislative, is not directly involved in investigations but sets the boundary conditions for law enforcement through regulations, usually through parliaments. The area of politics, employs a ambiguous role in this case, influencing decisions but also constituting a part of the executive. Conflicting fields can arise between all government levels. For example, the top levels might put pressure on the bottom to produce results, promoting automated analysis for its efficiency. Analysts, in turn, may use legally questionable methods, the judiciary may be concerned about the failure of legal proceedings in such cases, and regulators may be concerned about established practices that run counter to the intentions of Parliament. A recently observed problem occurs when the relationship between the system implemented as an assistant and the sovereign analyst is reversed. This can lead to effects resembling defensive decision making, where police officers intentionally make suboptimal decisions by following the results of the machine "assistants" even when they disagree. This is mostly explained by pressure from "above" and the need to protect themselves from redress if something goes wrong [9] .

Software Provider -The software provider develops the tools officers use in their investigations. Here, developers (e.g., S Molly ) implement the systems and algorithms. In doing so, it is expected that they know not only the technical details but also being aware of ethical implications. In contrast, the management has to mediate between the investors/shareholders, usually following a profit interest, the cost of implementing ethically flawless systems, and the pressure by the customers (police) to develop usable, efficient and productive systems. It is important to note that the software provider has typically no complete control over all aspects of a software system, as typically external dependencies, models, or training data are being used. Figure 1 : The stakeholders involved in communication analysis from the perspective of intelligence analysis, with conflicting interests giving rise to ethical dilemmas. We propose four main pillars of stakeholders: the civil society, the governmental authorities, the software provider, and the data subjects, each with its own subgroups of stakeholders, such as targets, developers, analysts, or NGOs.

Civil Society -Civil society can materialize not only in the form of mass media through its members, but also in the form of NGOs and human rights groups, arts and culture, street protests, whistleblowers, ethics councils, and so on. As such, it deliberates on what can be considered as acceptable ethical behavior in a given society, which parliament follows (through elections), and which can change over time. On the one hand, civil society can act as a corrective, for example, through critical reporting by the press, or legal advocacy through NGOs or human rights groups. Cases of unfair treatment, when entering the public agenda, can trigger the revisiting of fundamental ethical questions (as in the case of the criticism of the Northpointe recidivism algorithm and the debates it triggered about different notions of fairness and justice [43, 64, 98] ). However, mass media and public deliberation can also proliferate misleading ideas about what algorithms can and cannot do. These "socio-technical imaginaries" [52] have concrete implications for how systems are being used, for the transfer and negotiation of responsibility, as well as public acceptance [13] .

Addressing the ethical issues raised at the outset of the paper and negotiating the conflicting interest of different stakeholders is not a trivial task. Given all the different stakes involved, ethically-aware design of intelligence applications can not reasonably aim at implementing technical solutions to safeguard against all possible pitfalls. Rather it seeks to accomplish a serious consideration and balancing of the inherent trade-offs and inter-dependencies between different concerns, interests, and principles. For example, privacy-by-design may limit possibilities for advanced accountability. It thus needs to be negotiated which good is more important in each specific context and how to best achieve this. In doing so, we propose a socio-technical approach, not looking at possible technical solutions in isolation but as embedded phenomena, interacting with their environment [100] . Human interaction with technology is shaped by increasingly sophisticated and environmentally interwoven interfaces, connecting technical components, humans, and their surroundings [87] . Despite a rising appreciation of milieu-oriented approaches to understanding and designing interfaces [11] , the development of interfaces has traditionally been dominated by technical disciplines and, from an ethico-political point of view, not sufficiently considered the complex socio-technical dynamics at play [40, 50] . To fill this gap, it seems most productive to work with an extended definition of the interface going beyond isolated, technocentric meanings [66, 67] . We thus adopt the proposal to focus on "interfacing" as a joint practice of "becoming-with" of humans, machines, and environments [45] . One could also call this process an intra-action [12] , in which something new emerges, irreducible to its parts. Such an understanding leads to a more compehensive 

In the following, we investigate common interface design methods employed in VA-based systems -the technical side -from an interfacing perspective -including the political, responsibility, intra-active, and ethical dimensions -as part of an intertwined becoming-with one another. When describing relevant aspects, we refer back to the actors from the scenario in Section 3.3. As such, we identify areas where VA, through its human agency approach, is superior to fully-automated systems while also considering the additional burdens through the distribution of responsibility.

Interactive Exploration -One key concept of VA is that instead of merely generating results, such a system supports the knowledge generation process of analysts by enabling them to learn from the data space through supported interactions [57] . As part of this process, ethical mitigation techniques can be integrated. In the context of communication analysis, we introduce some of the critical aspects of VA by the example of HyperMatrix [33] , a conversation topic analysis and probability framework that uses a geometric deep learning approach. In our hypothetical scenario, police analyst S Martin uses the software. Instead of presenting lists of users and topics, it features an interactive design, shown in Figure 2 . The developers, in our scenario S Molly , deployed a matrix-based visualization B for the hypergraph network structure due to increased scalability, which represents model predictions through clustering and color-encoded by confidence. The design can be considered as a form of dimensionality reduction [55] , presenting the complex tensor model in a more comprehensible format. This supports the detection of patterns, while color-encoding facilitates pre-attentive understanding, helping S Martin to distinguish between users communicating about similar topics, like S Chris and S Carlos . Further, a multi-level visual semantic zoom through multiple, more detailed in-line visualizations, shown as insets D , allows for a more-detailed exploration, preventing an initial mental overload of S Martin . Steering is offered by interactive control elements A , allowing S Martin to set methods, cutoffs, and thresholds, thereby granting him agency and creativity in his usage of the system. Similarly, the system features elements from active learning enabling S Martin to interactively modify the model C to create something new and unique for the purpose at hand in the spirit of his analysis "becoming-with" the system. When using the system, the analyst explores the probabilities, refines model parameters, investigates hypotheses, validates change effects as part of an (indeed intra-active and) iterative analysis feedback loop.

The Machine Side -Analysis and Active Learning -An example of an intra-active becoming of investigator and machine is active learning. Figure 3 shows how an analyst provides labeled examples to the system improving its probabilistic accuracy. Labeling everything is tedious and time-consuming when done manually by the user. Here, intelligent labeling techniques can help by only requesting human input when required, relieving analysts from exploring basic or irrelevant patterns [19] . This concept can be applied universally to any number of feedback mechanisms between system and user, affecting data selection, machine learning models, heuristic algorithms, or their parameters. Through active learning, S Martin can integrate its experience-saturated as well as its domain-specific expert knowledge into the analysis process, thereby sharpening the analysis result with regard to the field's unique requirements. Methods such as color coding imply a form of nudging through preattentive processing. Transparency, in turn, can be conveyed by visualizing consequences and effects of the active learning approach (e.g., which entries are subsequently classified differently and by how much). This corresponds to a "what-if" preview , which supports the selection process, but can also act as a "control", e.g., against unintended side effects.

The Human Side -Guidance and Explainability -Guidance describes the interplay between system and user actions and their understanding in the context of machine learning, explainable artificial intelligence (XAI), and knowledge generation. One form of guidance can manifest by the system to nudge the user in the right direction, for example, by showing similar matches or conflicting options. In the context of learning and teaching, it exhibits a wider dynamic, encompassing system teaching, user teaching, system learning, and user learning. As a process, it can be described by the knowledge generation model [57, 89] and by the co-adaptive guidance process [96, 97] . Different forms of guidance can be achieved by a visually abstract visualization as well as conceptual user interaction design. For visualization, abstract representations like glyph [39] can be used for improved recognizability and comparability. In communication analysis, commonly used representations are text, highlighting, concept extraction, and network display. However, depending on the individual needs, they may not suffice. During the design, the aim of the representation, the selection of the appropriate visualization technique, the visual variables employed, and the color-schemes used has to be considered. Inherent biases can play an essential role, affecting values as social biases (e.g., homogeneity bias), actions (e.g., (c) Manually changing the connectivity strength for a hypergraph modem entry through selection, requesting a retraining of the contained machine learning model.

(d) Visualized changes in the model prediction (from negative to no to positive change). Before the changes are either rejected or accepted, the system nudges the analyst to examine the results for unintended side effects. blind spot or Ostrich effect), and perceptions (e.g., illusions or Weber-Fachner-Law) [25] , both during development as well as usage. For interaction, instead of filtering communication texts by keywords (the selection of which might be biased or incomplete), a visual query language [35] can be used where conditions are based on concepts. Here, the system may similarly suggest additional concepts for differentiation or indicate a too restricted search. In implementing such interfaces, care has to be taken regarding a neutral representation while considering the levels of detail and abstraction [53] . Too much abstraction can lead to a loss of context.

Provenance -As systems become more complex and the number of available interactions increases, the number and sequence of necessary steps during the investigation expands rapidly, making it increasingly difficult to fathom and explain them after the fact. In this context, while many systems focus on supporting the process of knowledge generation, few also emphasize how this process is carried out. [62, 68] . For reproducibility, which is crucial for accountability (see Section 4.2), knowing how analysts like S Martin used the system to draw conclusions is essential, but this becomes increasingly complicated when iterative and intra-active processes are involved. Instead of a linear timeline of steps or a list of explored hypotheses, VA applications can store interaction chronicles in context with the data, remembering settings, views, and actions. Through techniques like time-stamping, hashing, and digitally signing information, the chain of evidence becomes proofable. As such, they can provide tamper-proof tracking as well as replay functionality to revisit intermediate steps, while also enabling approaches like provenance graphs as shown in Figure 4 , strengthening the chain of evidence for both analysts and subjects.

In the following we identify advantages but also the limitations and risks involved in using VA for analysis of intelligence data. To do so, we refer back to the various challenges and conflicts identified in Sections 2 and 3. A1. User Agency -First and foremost, VA approaches support user agency and help prevent blind obedience to machine results, which is certainly one of the most common dangers in the face of exaggerated expectations. VA levels the playing field between system and investigator by creating a con-genial joint agency that facilitates a sense of growing-with each other rather than replacing each other. This is due to an approach -inherent in VA systems -that is collaborative rather than hierarchical. Furthermore, rather than encouraging an either-or decision process in cases where human analysts tend to deviate from the system's proposed outcomes, potentially leading to defensive or otherwise suboptimal decision making, explainability can help analysts like S Martin question their own inputs that influence the outcome during active learning. This can ultimately help identify dead ends in the line of inquiry.

A2. Privacy -Guidance allows circumventing privacy issues related to uninvolved third parties because selective presentation allows outsourcing to the system, such that the private life of Martin's girlfriend S Sarah , for example, does not need to be reviewed by natural persons.

A3. Fairness -Using VA as a means to handle heterogeneous data has advantages as opposed to automated systems exclusively trained on past criminal records. The latter frequently transport racial, gender, and other harmful stereotypes into the present and Figure 4 : Example sketch of a provenance history component using a directed acyclic graph (DAG) approach with inline details instead of a linear history, allowing for a more complete picture of the explored steps (1-9) and reverts from dead-ends (3, 4, 7) . future, thus cementing historically grown structures of domination [69] . With VA, the analyst can inspect the reasoning and intervene with his domain knowledge through corrective actions.

A4. Efficiency -Big Data-based VA can help to visualize complex operating dynamics, thus helping to see previously undetectable correlations [73] . VA systems may provide helpful abstractions and powerful analytical capabilities to efficiently find the non-trivial needle in the haystack, supporting investigators identifying even faint tracks. However, each design tailored for more efficiency has to be evaluated for oversimplification or risks of misinterpretation.

A5. Literacy -VA encourages users to engage with and appropriate the systems they use actively and creatively, developing advanced literacy in their usage through daily practice. This literacy, then again, can be shared among colleagues, for example, by the integrated sharing of recipes through VA systems (e.g., Common actions here are. . . ).

A6. Customization -Active learning supports the creation of tailored solutions instead of one-size-fits-all ones. The latter promise rather poor performance in the fight against organized crime, which requires highly localized and specialized approaches as well as detailed domain knowledge of experienced and highly qualified experts [83] . This is not a coincidence, but part of organized crime's recipe for success, which deliberately operates according to a nonlinear, swarm-like logic to confuse investigators and prevent a pattern-based approach from being applied. In such cases, the involvement of highly experienced experts is essential. Here, technical solutions that combine efficiency-enhancing automation with human criminological expertise to produce tailored solutions promise real gains. Furthermore, it is crucial that systems are capable of adapting dynamically to changing circumstances and environments.

R1. Lack of Accountability -Reproducibility is crucial for accountability (C6). This is one of the biggest challenges when considering that in instances of interfacing such as in VA, it is nearly impossible to clearly define and delineate who is responsible in cases of potentially flawed output and decisions since these need to be considered as the result of a joint becoming-with one another of the analysts' knowledge and the system's analysis. Apart from the exact software versions, the starting seeds of pseudo-random generators for non-deterministic algorithms, the exact data used in the training process, what is needed is also detailed information about how analysts like S Martin interacted with the system. Just presenting the final analysis is certainly not enough in such cases, as the decision-making process of S Martin was not strictly linear.

Learning has occurred along the way, and a judge like S Muller wants to be satisfied that all viable hypotheses have been explored, and were not merely neglected. Advanced provenance approaches in VA designs are needed to provide this accountability. R2. Training and Community-Building Among Users -It can also not be taken for granted that police officers possess the necessary background knowledge to understand the inner workings of the applied ML and VA systems, their potential shortcomings, and continual development. On the one hand, developers like S Molly have to mobilize considerable efforts to design explanations and train users in the effective and critical use of the systems. The way forward will be to design such training not as a one-way transfer of how-to-use information, but to curate it as an interactive process between software providers and analysts like S Martin , as well as between the analysts themselves (e.g., through design studies). Possible directions are the privacy-preserving use of gamification [90] inside VA system, where users can compete in practical challenges, or online discussion forums, where users can highlight and socially negotiate limitations and pitfalls. Moreover, the VA system and training activities have to be sensible to the shift of power structures triggered by the introduction of new software [47] . Young police officers with considerable digital literary will likely be favored by the transformation of work practices, whereas longstanding investigators might feel marginalized [56] . To avoid losing valuable experiential knowledge of senior investigators, it seems reasonable to work with technical solutions that make it possible to include rather than exclude these qualitative dimensions.

R3. Prevent Automated Inequality -Individual and institutional racism can be infiltrated into the system through active learning, with individuals influencing the system's learning process with their often unconscious biases [25] . This is a significant risk in the context of collaborative solutions such as VA, which grant investigators and officers increased responsibility in the training and design of automated analysis systems. This risk must be kept in mind and urgently requires further control mechanisms, especially in light of the alarming statistics and research on racial and sexist prejudices (unconsciously) harbored and enacted by many police officers [32, 37, 59, 86] . Active learning and other human in command solutions that result in automation based on user input, like from S Martin , should therefore only be employed if they have been preceded by in-depth anti-discrimination training and education. The same argument may be applied to developers like S Molly , whose biases and inherently (wrong) assumptions may be introduced by an inadequate design of the system. Considering and integrating adequate mitigation measures already at the VA system level may prove especially fruitful, as the invested time is regained through the disseminator effect.

R4. Facilitating critical reflection -The system and user interface should incentivise instances of reflection [15] . On the one hand, these instances of reflection (e.g., through text-based nudges and explanations) can help users recognize their own unconscious biases while interacting with the system, e.g., by showing warning signs after allegedly racist or sexist search queries or by blocking such queries all along (supplemented by an explanation). [102] On the other hand, such reflective features should enable users to challenge outputs and decisions of the system and support a critical use of the software by the investigators. Reflective design [95] can, accordingly, both inform the active learning process and improve the technical literacy of police investigators to interpret and evaluate system results.

R5. Human Oversight -Because of the risks mentioned above, approaches that support shared responsibility and foster a joint agency between analysts and systems must be accompanied by additional instances of human oversight. At a minimum, there needs to be a regular review of whether systems are being trained during the course of use and continuous input to ensure fair treatment, especially of protected groups. In addition, technical mitigation strategies can be considered. A VA provenance system that effectively negotiates responsibilities can remember the individual agents who interacted with the system or made changes through active learning. From a privacy perspective, it needs to secure and protect the identity of these agents but also allow for discussion of problematic decisions among peers and authorities (i.e., specific police officers who have labeled a particular entity differently). This would enable leadership like S Dr. D to validate in a privacypreserving way how much biases occur within their organization. Caution, however, must be taken not to arrive at a culture of control, where long-serving officers feel deprived of their agency [56] 4.4 Negotiating Risks and Advantages Through the Interface VA systems present a particular instance of interface-oriented solutions. As such they conform to data infrastructure platforms, allowing acquiring, handling, leveraging, and storing information and accompanying metadata from heterogeneous sources. Hidden biases in fully automated systems trained on biased past crime data or otherwise problematic, dirty, or insufficient big data sets can be mitigated through more dynamic, real-time, and interactive approaches. However, discriminatory biases can also be reinforced through VA-methods, placing much responsibility on users. Therefore, in-process measures of fairness are urgently needed. To this end, it is absolutely necessary to always ensure the traceability of the process genesis, enable accountability in cases of abuse, and to initiate retraining processes accordingly. The transparency required must also be harmonized with privacy requirements, which presupposes a certain level of abstraction and can only work well with decently trained personnel. In contrast to the manual combination of separate analyses or fully automated approaches, VA can be designed with built-in consideration of ethical issues for the entire knowledge generation process, adjusting user expectations to technical capabilities (C4), counteracting opacities (C3) and meeting privacy requirements (C2). Through human oversight (C5) by different stakeholders, discrimination (C1) can be detected. Simultaneously, by automatically collecting tamper-proof provenance about the system and the human, VA systems can increase trustworthiness and accountability (C6).

We have shown in detail how various VA methods can address ethical challenges in advanced analytical systems. While we have highlighted concrete points to consider, we do not intend to present a fixed set of rules for the ethically conscious design of VA systems. Indeed, in our view, this is always a matter of negotiating trade-offs between conflicting interests, which can vary widely depending on the unfolding interaction dynamics. Therefore, we aim to stimulate a discussion about the consideration of ethical implications as an integral part of the design process from the outset. Although we focus on the case of intelligence applications, many of the results of our work and the ethical discussion are more generally applicable to the design of VA applications. This is because, on a more abstract level, our approach leads us to the insight that one of the main advantages of VA methods is that they take the interface as a starting point for technological innovation. This means that innovation is approached not only in a technical, but rather in a socio-technical way. Useful innovations cannot exist in isolation and should pay attention to their impact on communities and society as a whole. This shifts the focus to embedding technologies into existing social institutions such as police departments and criminal justice agencies. Here, new technologies are most useful when combining the benefits of efficiency-enhancing automation with the experience-saturated knowledge of investigators. Concentrating on the interface has obvious advantages, as it implies a focus on the situated nature of human-computer-configurations.

By capitalizing on instead of trying to stabilize the potential openness, and thus eventfulness of technological interconnectedness, undesirable dynamics can be dealt with much more proactively, while flexibly addressing the situational and individual needs of different cases. The issue is not only what interacts with whom, but also how new phenomena emerge as part of complex intra-active configurations of people and automation systems. Discriminating behavior of individual investigators, for example, might not affect the functioning of the police force as a whole. When multiplied and cemented in automation processes, however, it can contribute to structural discrimination and patterns of unfair treatment on a larger scale. By responding transparently to investigator input, VA can help well-meaning analysts recognize their often unconscious biases by showing them how harmful social stereotypes sometimes cause them to overlook features and stumble down blind alleys.

To avoid risks and increase benefits, it is important to keep in mind that interfacing is not simply the matching of two separate entities, but the creation of something fundamentally new, a hybrid of human and machine. This hybrid requires tailored quality insurance measures such as adapted training, new forms of oversight involving technical, legal, and ethical experts, and also adapted policy and ethical frameworks that focus not solely on the technologies or on the user, but on what emerges as something new in the interaction between these entities.

From Principles to Practice An Interdisciplinary Framework to Operationalise AI Ethics

Predicting Judicial Decisions of the European Court of Human Rights: A Natural Language Processing Perspective

Statement of Concern About Predictive Policing by ACLU and 16 Civil Rights Privacy, Racial Justice, and Technology Organizations

Charismatic Technology

The Charisma Machine: The Life, Death, and Legacy of One Laptop per Child

Security and the Claim to Privacy

Doubt and the Algorithm: On the Partial Accounts of

Seeing Without Knowing: Limitations of the Transparency Ideal and its Application to Algorithmic Accountability

Frequency and Causes of Defensive Decisions in Public Administration

The 'Right to be Forgotten' -Worth remembering?

Technological Environmentality: Conceptualizing Technology as a Mediating Milieu

Diffracting Diffraction: Cutting Together-Apart

Talking AI into Being: The Narratives and Imaginaries of National AI Strategies and Their Performative Politics

Reflective Informatics: Conceptual Dimensions for Designing Technologies of Reflection

Race After Technology: Abolitionist Tools for the New Jim Code. Polity

Readings in Information Visualization: Using Vision to Think

Five Reasons for Scenario-Based Design

Ethical Dimensions of Visualization Research

Design Justice: Community-Led Practices to Build the Worlds We Need

Unique in the Crowd: The Privacy Bounds of Human Mobility

ROXSD: A Simulated Dataset of Communication in Organized Crime

Visual Text Analytics in Context of Digital Humanities

2018. Cognitive Biases in Visualizations

Automating Inequality: How High-tech Tools Profile, Police, and Punish the Poor

White Paper on Artificial Intelligence -A European Approach to Excellence and Trust

Proposal for a Regulation of the European Parliament and of the Council Laying down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts (COM/2021/206 final): Artifical Intelligence Act

Proposal for a Directive of the European Parliament and of the Council on Improving Working Conditions in Platform Work (COM(2021) 762)

Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data

European Union Serious and Organised Crime Threat Assessment (SOCTA): A Corrupting Influence: The Infiltration and Undermining of Europe's Economy and Society by Organised Crime

Stops and Stares: Street Stops, Surveillance, and Race in the New Policing

Visual Analytics for Temporal Hypergraph Model Exploration

Communication Analysis through Visual Analytics: Current Practices, Challenges, and New Frontiers

CommAID: Visual Analytics for Communication Analysis through Interactive Dynamics Modeling

Mapping Digital Communication Systems: Infrastructures, Markets, and Policies as Regulatory Forces. Media

Male Peer Support and the Police Culture

Knowledge Discovery in Databases: An Overview

A Systematic Review of Experimental Studies on Data Glyphs

The Interface Effect

Facial-Recognition Software Might Have a Racial Bias Problem

Raw Data is an Oxymoron

Putting the J(ustice) in FAT

Towards an Integrated E-mail Forensic Analysis Framework

Staying with the Trouble: Making Kin in the Chthulucene

Beyond the Prediction Paradigm: Challenges for AI in the Struggle Against Organized Crime

Flexible Sketches and Inflexible Data Bases: Visual Communication, Conscription Devices, and Boundary Objects in Design Engineering

SpaCy: An NLP library

The European Union General Data Protection Regulation: What it is and What it Means

IBM. 2020. i2 Analyst's Notebook

Future Imperfect: Science, Technology, and the Imaginations of Modernity

Minions, Sheep, and Fruits : Metaphorical Narratives to Explain Artificial Intelligence and Build Trust

Feeding the Machine: Policing Crime Data & Algorithms. William & Mary Bill of

Dimensionality Reduction for Data Visualization

Predictive Policing and the Politics of Patterns

Visual Analytics: Definition, Process, and Challenges. In Information Visualization, Andreas Kerren

Mastering the Information Age Solving Problems with Visual Analytics

Antimuslimische Einstellungen in der Polizei?: Der Zusammenhang von Kontakthäufigkeit und -qualität mit Vorurteilen und Stereotypen gegenüber Muslimen

Power and Interdependence in the Information Age

Thinking Critically About and Researching Algorithms. Information

Storytelling: The Next Step for Visualization

Weight Poisoning Attacks on Pretrained Models

Fair, Transparent, and Accountable Algorithmic Decision-making Processes: The Premise, the Proposed Solutions, and the Open Challenges

Analytik des Interfacing. Zur Materialität technologischer Verschaltung in prototypischen Milieus robotisierter Pflege

Athanasios Karafillidis, and Roser Pujadas. 2022. Theorising Interfaces / Interface Theories

Paths Explored, Paths Omitted, Paths Obscured: Decision Points & Selective Reporting in End-to-End Data Analysis

The Production of Prediction: What Does Machine Learning Want?

The Ethics of Surveillance: An Introduction

Foundations of Statistical Natural Language Processing

Big Data: Die Revolution, die unser Leben verändern wird (1

A Survey on Bias and Fairness in Machine Learning

Where are Human Subjects in Big Data Research? The Emerging Ethics Divide

Interpretable Machine Learning: A Guide for Making Black Box Models Interpretable

Not Interesting Enough to be Followed by the NSA

Computing and accountability

Algorithms of Oppression: How Search Engines Reinforce Racism

Nuix Pty Ltd. 2020. Nuix Discover and Nuix Investigate

Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

The Paradoxes of Organized Crime

The Black Box Society: The Secret Algorithms That Control Money and Information

Human Communication

The Consequences of Race for Police Officers' Responses to Criminal Suspects

Roser Pujadas, Erika Valderrama, and Will Venters. 2020. Interfaces and the Dynamics of Digital Ecosystems: A Study of the Online Travel Ecosystem. International Conference on Information Systems (ICIS)

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead

Knowledge Generation Model for Visual Analytics

Psychological perspectives on motivation through gamification

Mapping Conversations About New Media: The Theoretical Field of Digital Communication

Social Network Analysis

Visual Analytics of Conversational Dynamics

Disparate Impact in Big Data Policing

Reflective Design

Co-Adaptive Visual Data Analysis and Guidance Processes

Learning and Teaching in Co-Adaptive Guidance for Mixed-Initiative Visual Analytics

Inspecting Algorithms for Bias. MIT Technology Review

Interactive Ambiguity Resolution of Named Entities in Fictional Literature

Human-Machine Reconfigurations: Plans and Situated Actions

When Communication Meets Computation: Opportunities, Challenges, and Pitfalls in Computational Communication Science

Toward a Design Space for Mitigating Cognitive Bias in Vis

New/s/leak 2.0 -Multilingual Information Extraction and Visualization for Investigative Journalism

The Trouble with Algorithmic Decisions: An Analytic Road Map to Examine Efficiency and Fairness in Automated and Opaque Decision Making

The authors acknowledge the financial support by the Federal Ministry of Education and Research of Germany (BMBF) in the framework of PEGASUS under the program "Forschung für die zivile Sicherheit 2018 -2023" and its announcement "Zivile Sicherheit -Schutz vor organisierter Kriminalität II". The BMBF had no role in the design and conduct of the analysis or the preparation, review, or approval of the manuscript. The authors declare no other financial interests.