key: cord-0886738-gud4dow5 authors: Kalpokas, Ignas title: Problematising reality: the promises and perils of synthetic media date: 2020-11-09 journal: SN Soc Sci DOI: 10.1007/s43545-020-00010-8 sha: 5f8bfee1f45aab5d92e7bcc71ce5e74a24a2a443 doc_id: 886738 cord_uid: gud4dow5 This commentary article focuses on the emergence of synthetic media—computer-generated content that is created by employing Artificial Intelligence (AI) technologies. It discusses three of the most notable current forms of this emerging form of content: deepfakes, virtual influencers, and augmented and virtual reality (collectively known as extended reality). Their key features are introduced, and the main challenges and opportunities associated with the technologies are analysed. In all cases, a crucial change is underway: reality (or, at least, the perception thereof) is seen as increasingly less stable, and potential for manipulation is on the rise. In fact, it transpires that personalisation of (perceived) reality is the likely outcome, with increasing societal fragmentation as a result. Mediatisation is used as a broad-ranging metatheory that explains the permeation by media of everyday affairs to explain the degree of impact that synthetic media have on the society. In this context, it is suggested that we search for new and alternative criteria for reality that would be capable of accounting for the changing nature of agency and impact in today’s world. for assessing the reality of objects and phenomena still hold and whether reality in the general sense must be reconsidered. Adopting a perspective tentatively affirmative of such a switch, this article explores ways in which the new, synthetic, media can affect human thinking and behaviour without dealing with anything conventionally real. Although the assertion that the media now play an increasingly central role in everyday life has become ubiquitous, the changing nature of the media themselves is commonly overlooked. While discussions would often focus on issues of framing, misrepresentation, or underrepresentation, it is becoming crucial to also focus on the media's generative capacity. The latter refers to the capacity to create synthetic likenesses, personalities, and entire environments solely by way of digital technologies. Therefore, the reality we experience and use as a baseline for future decisions and life plans can easily have no physical counterpart and might be even unique to our own personal experience. To provide at least a tentative account of the transformations pertaining to synthetic media, mediatisation theory is briefly overviewed as a metatheory for conceptualising the media's growing influence. The analysis then focuses on synthetic media, first engaging with the capacity to create synthetic likenesses (deepfakes), then moving onto synthetic personalities (virtual influencers) and synthetic worlds (Extended Reality). This article thereby demonstrates the growing challenges faced by traditional accounts of reality that are biased in favour of physical tangibility. The contention is that the reality of something must, instead, be measured primarily through affective capacity. According to an influential definition, mediatisation refers to the condition whence the media 'have become an integral part of other institutions' operations, while they also have achieved a degree of self-determination and authority that forces other institutions […] to submit to their logic' (Hjarvard 2008, p. 106) . The matter here is, essentially, one of the media's ever-presence, permeating 'all aspects of private, social, political, cultural, and economic life, from the micro (individual) to the meso (organisational) to the macro (societal) level' (Giaxoglou and Döveling 2018, p. 2). In the same vein, the social world of today is 'changed in its dynamics and structure by the role that media continuously (indeed recursively) play in its construction' (Couldry and Hepp 2017, p. 15 ). Hence, the media no longer mediate between the world and the experience of it but increasingly generate that experience. Simultaneously, while previously individuals were confined to their physical location, now one can be immersed in a number of digital worlds and interact with a number of individuals regardless of distance (Couldry and Hepp 2017, p. 90) . Likewise, the media must be seen as constituting 'a realm of shared experience' by offering 'a continuous presentation and interpretation of "the way things are"' and thereby contributing to 'the development of a sense of identity and of community' (Hjarvard 2008, p. 126) , thereby determining the functioning of social relations (Nowak-Teter 2019, p. 5). Of course, the media have always played a community-building and community-integrating role. However, the key difference is this: while previously the media used to perform a somewhat supplementary role, building onto the 'real' world and conveying or explaining it, with varying degrees of fidelity, the current condition is characterised by the media hosting and creating the world that they purport to merely represent (Kalpokas et al. 2020) . No less importantly, mediatisation also implies a certain delegation of agency as 'collectivities [are] created by automated calculation based on the "digital traces" that individuals leave online' (Couldry and Hepp 2017, p. 168) . In this sense, as Boler and Davis (2018, pp. 82-83) assert, algorithms inherent in today's dominant media platforms 'define the spaces of our information encounters, encounters with others, and the status of knowledge'. Simultaneously, attention becomes the scarcest of resources-individuals simply no longer have sufficient means to pay enough of it (Citton 2019, p. 35) . When coupled with algorithmic analysis of trends and user behaviour, attention becomes its own magnet: 'attention attracts attention', i.e. the more people interact with a digital object, the more it rises in the algorithmic pecking order, thereby becoming more visible to others; therefore, even 'looking at an object represents a labour which increases the value of that object', leaving pleasure and labour inextricably entwined (Citton 2019, pp. 47-48, 65) . It then also becomes obvious that whatever maximises audience attention, becomes an attractive proposition for content providers-audience captivation becomes more important than truthfulness, 'reality' in the conventional sense of the term, or any other considerations (Kalpokas 2019) . That also implies a great degree of malleability and adaptability of the social world, as strict adherence to the tangible no longer is a must: for as long as social occurrences can be created and sustained within media ecosystems, they can and should be seen as sufficiently real, leading towards 'primacy of anticipation over content' (Marcinkowski 2014, p. 17) . Such anticipation refers to both the communicators (anticipation of particular audience expectations to be satisfied) and their audiences (anticipation of being satisfied); in this situation, neither side is likely to give the substance of content priority-whatever satisfies expectations, is good enough. And no less importantly, technology now affords increasingly sophisticated ways of decoupling satisfaction of expectations from conventional considerations of reality by producing high-fidelity synthetic reality. Deepfakes are digital content, generated using a Deep Learning technique known as a Generative Adversarial Network (GAN). The production process involves the simultaneous use of two algorithms: one, typically referred to as 'the generator', is tasked with creating artificial content while the second, called 'the discriminator', tries to find fault in the newly-generated content; once such a fault is found, the generator learns from its own mistakes and creates an improved version to be scrutinised by the discriminator, and so on (Chesney and Citron 2019, p. 148; Giles et al. 2019, p. 8 )-this is where the adversarial element of GAN comes from. The end product is arrived at when the pair of algorithms can no longer make any improvements through mutual learning. One of the main fears pertaining to deepfakes is that they can purport to represent events or insinuate behaviours that never took place in order to destroy the reputations of featured individuals (for both political manipulation and private harassment) or potentially even sway the results of elections (Chesney and Citron 2019, p. 148); alternatively, they can lead to an environment of distrust, whereby even 'hard' evidence of crimes or misdemeanours can be easily dismissed as mere deepfakes (Woolley 2020, p. 14) . Deepfakes can also potentially be used for blackmail and extortion, either for financial gain or to manipulate decisionmakers (Hall 2018, p. 52) . Likewise, whereas the creation of simulated public opinion currently requires armies of trolls, deepfakes can automate the process, generating custom-made content coming from custom-generated profiles etc. (Giles et al. 2019, p. 11) . Crucially, deepfakes are democratic in nature: the only things needed are training material for the algorithms and computing power; in contrast to traditional photo or video editing software, no specialist skill is necessary as the process is automated, meaning that even a relative amateur can produce high-quality synthetic content (Chesney and Citron 2019, p. 148) . Currently, the primary use of deepfakes is for synthetic pornography, as in transposing the faces of celebrities or former partners onto the bodies of performers in pornographic videos; however, there are clear threats coming from improvements in the technology itself, such as reducing the quantity of necessary input and increasing the quality of output, and from its pairing with other techniques, including big data-based precision targeting to identify those most susceptible to believing the synthetic content (Paul and Posard 2020) . Although deepfakes can usually still be identified it is only a matter of time until technology catches up with human perception; moreover, as human response to audio-visual content is often visceral and immediate, people will, nevertheless, believe their eyes and ears 'even if all signs suggest that the video and audio content is fake' (Charlet and Citron 2019) . Simultaneously, as communication, particularly online, is turning more and more towards the visual, the capacity to manipulate content in this dominant mode of expression can become a notable source of power (Vaccari and Chadwick 2020, p. 2) . Nevertheless, due to the aforementioned democratic nature of deepfakes, it is unlikely that this power would be concentrated in the hands of a few actors only. A much more likely outcome is dizzying excess, in which it becomes increasingly difficult for information consumers to make up their minds. The net result might be 'a climate of indeterminacy' whereby people have low levels of trust beyond their bubbles (Vaccari and Chadwick 2020, p. 2) . Moreover, this indeterminacy is likely to extend even further, including in domains where objective veracity is prized. One such example would be the legal process, whereby the authenticity of even video evidence will become hard to determine (see e.g. Maras and Alexandrou 2019), thereby further contributing to the undermining of trust. In particular, deepfakes may prove to be dangerous in the runup to elections, as parts of e.g. a smear campaign against an opponent. While unlikely to feature in isolation, they are likely to form an integral part of broader cyber operations, perpetrated by domestic or foreign actors (Whyte 2020) . Extant research already indicates that if deepfakes are targeted precisely, they can considerably reduce the image of 1 an unfavourably depicted politician in the eyes of the target population (see Dobber et al. 2020) . Certainly, the precision-targeting necessary for such an effect necessitates large sets of audience data, which might up the ante for those willing to enter the political manipulation game. Nevertheless, for well-resourced political campaigns and, even more so, for hostile nation-states targeted deepfakes will, in all likelihood, become a new addition to their arsenal. Still, even for the less-resourced, deepfakes may prove to be a viable tool, for example in trying to harass activists of the opposite camp by placing their images in pornographic videos or other types of content that the victims would likely find unpleasant and disturbing (see e.g. Maddocks 2020). For those reasons, it is extremely likely that deepfakes will feature, in some capacity, in the elections to come. Simultaneously, though, the problematisation of reality wrought about by deepfakes extends much further than manipulation or other nefarious uses. For example, as Kietzmann et al. (2020, p. 141 ) asserts, 'we may soon enjoy injecting ourselves into Hollywood movies and becoming the hero(ine) in the games we play' while shopping is going to be transformed by a capacity to create personal deepfake avatars to model different outfits, leading to 'ultimate personalization'. Indeed, deep personalisation is likely to be the next big thing in digital consumer-oriented products more broadly, a continuation of the current drive to put as much personal touch into services as possible. There are also further opportunities for businesses: while data-driven targeting and programmatic ad buying are already de rigueur; the next step would be employing GANs to deepfake segments in anything from news broadcasts to films in real time to deliver targeted advertising and personalised product placement to every viewer. Nevertheless, this ability to place oneself (or be placed) at the centre of the universe and to subject perceived reality to one's interests or tastes (or tasks at hand) clearly points towards an impending future of 'reality' that, instead of being stable and capable of providing a common point of reference, becomes personally tailored and, simultaneously, only personally meaningful, leading to personalised experience cocoons. Ultimately, such synthetic media are going to 'challenge public opinion and what we know as reality in basically all sectors of culture and society' (Woolley 2020, p. 107) . Hence, individuals will be faced with a broad variety of largely (or, at least, immediately) indiscernible truth candidates only to default to their pre-existing opinions, partisan bubbles, or influencers. However, the latter are also becoming synthetic. Recent developments in today's media also involve the creation of synthetic personalities, primarily as virtual influencers (VIs). Like their human counterparts, these are personalities geared for maximum audience impact. However, due to their synthetic nature, VIs provide an unprecedented degree of flexibility and targeting. Hence, it is typical for creators to provide VIs with 'a composite personality based on market research', and then use machine learning-based social listening to adapt to target audiences as effectively as possible (Bradley 2020a) . In contrast to a human influencer, all of the virtual one's characteristics, including 'age, gender, tone of voice and aesthetics' can be tailored to match audience expectations (Bradley 2020a). Therefore, as Bergendorff (2019) Similarly to what has been observed in relation to deepfakes, the synthetic nature of these influencers opens up potential for manipulation, particularly because they can be made so impactful. Campaigners already warn of adverse consequences on matters ranging from body image and a sense of inferiority in comparison to virtual personalities' computer-generated accomplishments to virtual influencers taking a political stance (Booth 2019; Yocom and Acevedo 2019) . And while in case of most, if not all, of the currently popular virtual influencers such concerns are more of a side effect, it is not too difficult to imagine a virtual influencer intentionally created for manipulative purposes; having been created specifically with appeal to a target audience's preferences in mind and specifically designed to evoke trust from that particular segment of the population, virtual influencers could the conceivably become trusted purveyors of information held in high esteem by their followers (Yocom and Acevedo 2019) . While they would be unlikely to develop independent persuasive power in the short or mid-term, virtual influencers could conceivably assist efforts to wrap target audiences within a cocoon of misinformation and this amplify existing campaigns. For brands, VIs offer the usual combination of advertising and audience engagement, but with total control of content and behaviour, unlike the often-erratic antics of real-life influencers (Bradley 2020a) . 1 Moreover, a VI tends to generate around three times more engagement than a human one and acquires followers at a significantly higher rate (Leighton 2019) , possibly as a result of their meticulous tailoring. An additional benefit of VIs is their independence from real-world context: for example, while coronavirus lockdowns issued by governments have significantly constrained opportunities (travel, public appearances etc.) for human influencers, virtual ones can continue regardless (Deighton 2020). The preceding can leave brands wondering why hire a human 'when you can create the ideal brand ambassador from sctratch' (Hsu 2019) . From a societal perspective, however, the situation might be somewhat suboptimal because VIs are less regulated than their human counterparts, leaving brands more leeway in constructing their campaigns; moreover, VIs endorsing products they claim to have tried (which is, of course, impossible) likely contains more than a hint of manipulation (Hsu 2019; Cook 2020) . This problem also extends beyond products and brands as the persuasive power of VIs can also be used for promoting political actors and agendas (Deighton 2020). Crucially, the synthetic nature of VIs might be somewhat liberating: while social media have been used by humans to perform their fake selves, VIs are at least authentically fake (Hsu 2019). Nevertheless, this authenticity can be easily lost. One reason is the interactive capacity of VIs. Making VIs interact both among themselves and with real-life humans allows for storytelling opportunities and manufactured events that can be carefully orchestrated to generate publicity (Sokolov 2019) and captivate audiences through 'emotional storytelling and empathy' (Luthera 2020) . This captivation might preclude followers from maintaining the necessary distance. The second reason, meanwhile, concerns those followers who do retain that distance: here, the creators of VIs may be facing an emotive storytelling gap in talking about specifically human experience while simultaneously being open about not being human (Bradley 2020b) . That might drive the creators of some VIs to be at best ambiguous about the nature of their creations. After all, around 42% of millennials and Gen Z social media users follow or have followed VIs without realising their artificial nature (Cook 2020). The next step in dissolving the boundary between the real and the synthetic will be true AI generation of VIs without requiring major human input (Bergendorff 2019) . Once VIs cease being painstakingly human-made and, instead, become interactively and automatically generated, they will not only become ubiquitous but will also become even more irresistible by automatically adapting to their audiences. And as their impact increases, the question of whether they are human or virtual will become increasingly irrelevant. A further step towards the problematisation of reality is the capacity for immersion in a synthetic environment through Augmented and Virtual Reality technologies, typically referred to collectively as Extended Reality (ER). The problematisation of reality is made particularly acute by the fact that ER is only effective if it causes an illusion of presence (i.e. the loss of awareness of technological mediation) and an illusion of plausibility, whereby a user's experience responds personally to their actions (Pan and de Hamilton 2018, pp. 406-407) . For full immersion, users must also be provided with an 'experiencescape', i.e. a package of 'people, products, and a physical environment' (Hudson et al. 2019, p. 461) . In that way, ER becomes a self-contained world that stands in for 'normal' everyday experience. Moreover, the merging of ER and social media is likely to offer 'far more immersive experiences and the possibility of sharing more of our lives online', affording an even more effective refuge from the physical world (Marr 2019) . Thus far, the primary uses of ER are for gaming and educational purposes, but new and emerging uses involve e.g. virtual attendance of real concerts and sporting events (Rubin 2018), potentially even allowing a band to perform in their studio while their sound is put into a completely virtual concert performed by their avatars-particularly attractive in times of distancing and travel restrictions. Additionally, there is increasing use of ER meeting spaces for both commercial and private use, standing in for travel (Rubin 2018) . In other words, ER can literally offer a (synthetic) world of experience within the confines of one's home. However, virtual co-presence also comes with its own dangers, such as virtual abuse, which can have psychological effects as bad as the 'real' thing (Rubin 2018). Moreover, there also is a threat of manipulation as ER can cause attitude change through experiencing a place, a brand, or a person (Tussyadiah 2018) . In fact, social engineering on ER is already a thing, although the applications currently available (at least those open about their aims) are primarily concerned with empathy, social responsibility etc. (Marx 2019) . Nevertheless, there is no guarantee that such techniques will not be (or are not already) also employed for nefarious purposes. An important sticking point is that ER allows much more extensive data collection (particularly biometric data) than any other type of media (Braun 2019) . These data can also be matched with a record of everything the user sees or hears while the device is on (Marr 2019) . The result is not only potentially detrimental privacy invasion but also unprecedented capacity to tailor the experienced environment by predicting the most visceral of our responses (Hall and Takahashi 2017) . That tailoring might tilt users towards prioritising ER over the non-user-centric physical environment, thus further contributing to the problematisation of reality. Furthermore, the ER of the near future 'will be aware, data-rich, contextual, and interactive' courtesy to the development of both 5G and cloud-based representations of the physical environment, allowing data to be overlaid on the physical world in real time; the net result will be not only a richer experience but also a shift of ER from an add-on to the operating system of everyday life (Koetsier 2019) , particularly as haptic technologies mature. As with the other types of synthetic media discussed above, manipulative potential is rather clear. In the case of ER, this relates not only to the capacity of generating artificial experiences but also its immersive quality that might have serious ramifications. As audiences experience content much more vividly and are, therefore, less immune to the messages promoted, fake news on ER are likely to be more impactful than their broadcast or online forms (Pavlik 2019) . A further issue to be kept in mind is the disappearance of authorship: whereas in traditional content it is easier for the audience to remain conscious of the constructed nature of what they encounter, immersiveness is likely to lead to over-ascribed authenticity (Johnson 2020) . In this way, borders between different versions of reality are likely to blur, thereby completing the slide towards epistemological anarchy. Finally, as the adoption of ER accelerates, that will further affect the perception of the self, not least through the development of digital avatars into effective standins (Marr 2019) , thereby diminishing the importance of the physical self. Concurrently, in line with the development of synthetic persons, it will be increasingly difficult to tell whether one is interacting with an avatar of a human or with an artificial agent (Pan and de Hamilton 2018, p. 411) , thus further stripping reality of its tangible and verifiable character. As shown in this article, the issue of reality has become less straightforward than ever. As we move towards the construction of synthetic likenesses, persons, and entire worlds, that which is real (at least in terms of affecting understanding and causing decisions) might become intangible and personalised. Certainly, people have always had divergent interpretations of the world, including many opinions that were false. However, the present condition simultaneously allows the creation of increasingly realistic synthetic objects and environments and is likely to ensure survival even of those who fundamentally misperceive some of the basic physical characteristics of the world. Nevertheless, the change goes even deeper as we seem set to lose the awareness of-or, indeed, interest in-the source of lived reality while lacking the time, resources, or motivation to assess the exact nature of what is driving us towards certain beliefs, actions, or decisions. Hence, it is advisable that we move towards an affective criterion of reality: an artefact's or an environment's reality to an individual is directly proportionate to the power of affect exerted onto that individual (Kalpokas 2020) . While that might, at first sight, come across as a highly relativist solution, it nevertheless adheres closely to the problematisation of reality discussed above. The net result would be a partly synthetic life that only abides by the conventions of the physical world to a limited extent. Data availability There is no data associated with this article. Why I'd invest in a robot teenager: an investor's perspective on CGI influencers The affective politics of the 'Post-Truth' era: feeling rules and networked subjectivity Fake online influencers a danger to children, say campaigners. The Guardian Even better than the real thing? Meet the virtual influencers taking over your feeds. The Drum Can virtual influencers build real connections with audiences? The Drum Security, privacy, virtual reality: how hacking might affect VR and AR. IOT Tech Trends Campaigns must prepare for deepfakes: this is what their plan should look like. Carnegie Endowment for International Peace Deepfakes and the new disinformation war: the coming of age of posttruth geopolitics Brands are building their own virtual influencers. Are their posts legal? Huffpost The mediated construction of reality. Polity, Cambridge Deighton K (2020) Why use virtual influencers? Raconteur Do (microtargeted) deepfakes have real effects on political attitudes? Mediatization of emotion on social media: forms and norms in digital mourning practices The role of deepfakes in malign influence campaigns Deepfake videos: when seeing isn't believing Augmented and virtual reality: the promise and peril of immersive technologies The mediatization of society: a theory of the media as agents of social and cultural change These influencers aren't flesh and blood, yet millions follow them. The New York Times With or without you? Interaction and immersion in a virtual reality experience Promises and perils in immersive journalism London Kalpokas I (2020) Towards an affective philosophy of the digital: posthumanism, hybrid agglomerations, and spinoza Creating students' algorithmic selves: shedding light on social media's representational affordances Deepfakes: trick or treat Augmented reality is the operating system of the future. AR cloud is how we get there What it means for virtual instagram influencers' popularity rising The dark side of deepfake artificial intelligence and virtual influencers A deepfake porn plot intended to silence me': exploring continuities between pornographic and 'political'. Deep Fakes Porn Stud Determining authenticity of video evidence in the age of artificial intelligence and in the wake of deepfake videos Mediatisation of politics: reflections on the state of the concept The important risks and dangers of virtual and augmented reality Taking virtual reality for a test drive Mediatization: conceptual developments and research domains. Sociol Compass Why and how to use virtual reality to study human social interaction: the challenges of exploring a new research landscape Artificial intelligence and the manufacturing of reality. The RAND Blog Journalism in the age of virtual reality: how experiential media are transforming news Virtual influencer trends: an overview of the industry. The Drum Virtual reality, presence, and attitude change: empirical evidence from tourism Deepfakes and disinformation: exploring the impact of synthetic political video on deception, uncertainty, and trust in news Deepfake news: ai-enabled disinformation as a multi-level public policy challenge The reality game: how the next wave of technology will break the truth and what we can do about it This social media influencer is a robot-but how could this influence the future? The Globe Post Conflict of interest The author has reported no conflict of interests.