key: cord-0611501-3vf6c7pg
authors: Paananen, Ville; Kiarostami, Mohammad Sina; Lee, Lik-Hang; Braud, Tristan; Hosio, Simo
title: From Digital Media to Empathic Reality: A Systematic Review of Empathy Research in Extended Reality Environments
date: 2022-02-23
journal: nan
DOI: nan
sha: 7db819fd5759360013b67486781dda89cffb0000
doc_id: 611501
cord_uid: 3vf6c7pg

Recent advances in extended reality (XR) technologies have enabled new and increasingly realistic empathy tools and experiences. In XR, all interactions take place in different spatial contexts, all with different features, affordances, and constraints. We present a systematic literature survey of recent work on empathy in XR. As a result, we contribute a research roadmap with three future opportunities in XR-enabled empathy research across both physical and virtual spaces.

Empathy can be characterized as an emotional response caused or related to another's emotional state [36] . It serves society through understanding the situation of others, allowing humans to form more meaningful bonds and to act against injustices [37] . With the significant societal changes brought by the rise of computer systems, the empathic effect of digital media has been increasingly investigated. Accordingly, significant research has been conducted on eliciting empathy through virtual simulations. By leveraging immersive technologies such as extended reality (XR), these simulations enable the users to understand the experience of others regardless of their own temporal, cultural, demographic, or spatial context. The way we experience empathy is highly related to our local context. In the domain of ubiquitous computing and Human-Computer Interaction (HCI), context is generally understood as anything and everything that can characterize the situation of an entity. Context serves as the frame of reference through which we make sense of our world. Therefore, understanding context is essential for developing and analyzing computational systems [32] . Among the many components of context, space plays a central role [42] by affecting our perception of the world and our experiences.

Humans form meanings with spaces, such as safety and preference to some places over others [98] . As such, being in a space leads to making and engaging with different contexts. Spaces affect one's understanding of the situation, and thus how they experience empathy [37] . For instance, a person falling in the street has a different meaning to a person falling on a theatre stage. Due to this human connection to spaces, empathy is necessarily bound to the spatial context in which they operate. Despite the spatial nature of humans and its significant relationship to context and empathy, space is still an understudied topic in HCI research [64, 69] . To help make sense of this rapidly emerging field of research, we present a systematic literature review on empathic XR applications under the consideration of their spatial contexts.

Empathy is generally understood as the prosocial behavior that allows humans to feel and understand the experiences of others [37] . While the concept of empathy seems straightforward at surface level, its history and development are multifaceted. The concept originates from the notion of Einfühlung -translating to "feeling into" -developed by German philosopher Robert Vischer, and later brought into psychology by Theodor Lipps in 1903 [37] . In contrast to the interpersonal character trait as it is known today, Einfühlung is based on aesthetic theory to refer to the appreciation of objects and the relationship one has with their surroundings. In more recent literature, empathy carries a complex notion of behaviors and experiences revolving around undergoing or understanding another's experience. Empathy thus encompasses nuanced concepts such as perspective-taking, compassion, sympathy, and pity. Contradictory viewpoints exist as well: it has been suggested empathy is often simplified and misunderstood [12] . By proposing the concept of "radical compassion" as an alternative to empathy, Bollmer argues for a more careful stance in assuming the experiences of the Other.

Empathy comprises two processes: affective empathy, related to feeling, and cognitive empathy, related to understanding [37] . Affective empathy has been shown to be more constant as it is related to the subject's personality, while cognitive empathy is easier to influence through external stimuli [105] . The recent notion of somatic empathy describes the embodied experience of empathy, for instance, wincing at another's experience of pain [107] , and understanding the children's perspective of interacting with everyday objects that are primarily designed for adults [95] . This process has been a prevalent topic in cognitive neuroscience by investigating how the mirror neurons in human brains are a key element in the empathic process [37] .

Empathy is heavily related to the person's own physical and social attributes, culture, and experiences. As such, it is difficult to measure and compare empathy between people. Factors such as race, gender, sexuality, social status, and culture affect how empathy is experienced. Generally, it is easier to form empathy between two people who share some aspects of their character [105] . Over the years, multiple measurement instruments have been developed to assess empathic traits in people. The most popular self-report measures are the Interpersonal Reactivity Index (IRI), the Empathy Quotient (EQ), and the Questionnaire of Cognitive and Affective Empathy (QCAE) [37] . Further, empathy can also be investigated on the neurological level. For instance, modern brain imaging technologies such as fMRI have been able to pinpoint the neurological basis of empathy in the mirror neuron system [37] . However, neuroimaging technologies require specific expertise which is not often available to researchers outside of the field.

Empathy is thus a complex process that has behavioral, neurological, and cultural bases. As such, a robust and all-encompassing definition of empathy is difficult to formulate. The present paper considers "empathy" as a general term describing the process through which one person is "feeling with" another's experience. Further, in the context of computer-mediated empathy described in this paper, a distinction should be made between empathic design and designing empathy tools. Empathic design refers to understanding the user's experience of a product or a service during the design process. It is a typical approach for understanding the user in the domains of design and research [135] .

On the other hand, empathy tools are artifacts designed to elicit an empathic response in users [105] . With the rise of extended reality technologies and their easier development processes, virtual empathy tools have become a popular way to investigate, elicit, and assess empathy [105] . This paper aims to survey the most prominent works on virtual empathy tools, as well as the observations into methods employed to measure empathy. In testing the effectiveness of empathy tools, laboratory-based studies often take the advantage of simulations to elicit empathy in virtual situations.

Conducting empathy studies in the wild brings difficulties in replicability. However, the effect of the designer in a laboratory experiment is a point of contention [105] . To form more robust practices, there is growing literature on different frameworks for empathy in technological contexts. For example, Carey et al. [21] proposed a framework for gathering data on empathy in VR studies, and the framework by W. B. Owais and E. Yaacoub [130] focused on quantifying empathy experiences in immersive virtual environments. We thus classify the studies based on the research methodology, to address the diverse measures.

Virtual environments refer to computer-mediated 3D worlds characterized by immersion, realism, malleability, and scalability [31] . In such environments, virtual entities can be either confined to purely virtual worlds (Virtual Reality, VR) or integrated within physical environments (Augmented Reality, AR) according to the well-recognized virtualphysical continuum by Milgram and Kishino [85] (the upper part in Figure 1 ). In the middle of the continuum, the coexistence of physical objects and virtual entities construct virtual environments, known as mixed reality [116] (MR).

The aforementioned virtual environments, regardless of the degree of virtuality and physicality, can be described by a collective term -Extended Reality (XR: AR/MR/VR). Further, the immersive tools can also be used to remove or obstruct parts of perception, often visual, described as "diminished reality" (DR) [87] .

Virtual environments can be regarded as facilitators of empathy, as depicted in a popular talk by Chris Milk through advocating the prospect of virtual environments as the ultimate empathy machine [86] . VR allows users to feel and understand the difficulties or struggles of other individuals [2] , while AR can augment their surroundings with information of others [5] . Further examples under the virtual-physical continuum are available in Appendix B.

Alongside the development of virtual environments, has been the emergence of the metaverse [76] , which describes diversified virtual environments in everlasting, gigantic, and wide-reaching ways. Although the metaverse is at the nascent stage [76] , the prospect of the metaverse describes that the virtual entities will integrate into many aspects of our daily routines, and people will spend significant proportions of time with such virtual entities. Supported by several ambitious plans of metaverse cities, such as Seoul 1 and Shanghai 2 , virtual environments can become indispensable in our urban spaces. With such prospects and anticipated user numbers, virtual-physical blended urban spaces will become a frontier of metaverse research.

More importantly, such virtual-physical urban spaces should consider the diverse needs of stakeholders, meanwhile, urban designers can leverage virtual environments to inform concerns and resolve conflicts among stakeholders with conflicting needs in shared ways [75] . When virtual environments meet empathic design, we can foresee that empathy can play a vital role in multitudinous virtual environments. As shown in Figure 1 (the lower part), all virtual environments under the continuum of XR can be augmented by a new layer of empathy, namely Empathic Reality (ER).

Through the immersive (virtual) lens on top of our physical spaces, XR technologies promote alternative experiences to citizens along a pathway of empathic spaces. As so, as an extension of our physical reality, XR serves to bring new experiences on top of, or in place of our usual experiences 3 . This survey paper focuses on the whole range of XR technologies and their potential of connecting empathic elements and our physical spaces.

Context is tied with human experiences. In the process of understanding empathy, we analyze context through the lens of spatiality. Being and Time by Heidegger is one of the foundational works on spatial existence. In this book Heidegger claims human existence is first and foremost understood through the process of taking and making space.

The fundamental nature of space for human existence can be seen in the spatiality of human language, memory, and experiences. Following a similar path, Bollnow [13] elucidates the nuanced ways in which humans understand, feel, and embody spaces. On a personal level, meanings we attach to spaces can be positive or negative, influencing whether we seek or avoid such places. These places can be public or private or limited to certain social groups, e.g. residents, employees, or students. Spatiality describes the various ways we inhabit spaces, taking into account the full extent of spatial experience. Spatiality serves as a useful context through which we understand different phenomena, e.g., in the context of the current paper, where, and in what kind of XR spaces do we experience empathy?

Bollnow argues that the psychological mood and the atmosphere of the space are the same. A person's emotional state affects the perception of a space, which in turn affects the atmosphere the space elicits. The connection between the atmosphere of the place and empathy has also been made by others, e.g., through the metaphor of sphere of mutual affection by Sloterdijk [115] , or as applied to geographical software by Widmer and Klauser [134] . Subjects sharing a space are also interacting with each other. A study by Maglio and Polman [80] found that people felt more similarity with people that they were spatially oriented towards [80] . And so, as the psychological distance is not always equal to 3 Note ubiquity of spatial metaphors in languages.

the geometric distance, we need to consider the experience of space as a co-created meaning. In other words, being in a space is a process of empathic interaction, where the space and its inhabitants are in a shared creation of mood.

As this process is holistic and formed through the co-presence of people with their unique experiences of space, the atmosphere of place resists a singular definitive description.

In computing, spatial experiences were first brought into discussion by Harrison and Dourish [53] , by examining the notions of place and space in the context of computer-supported collaborative work (CSCW). 10 years later, Dourish [33] returns to the topic to address conceptual difficulties the spatial notions carry in the face of developing technology.

Building on the works of Harrison and Dourish [53] , Lentini and Decortis [78] develop five dimensions for understanding the experience of space: geographical, sensorial, cultural, personal, and relational. Through these dimensions, the richness of experiences in the urban environment is more readily understandable for developing ubiquitous computing systems. Similarly, proxemics, the study of interpersonal distances, was brought to HCI through the analysis of social and spatial contexts of technology [69] to highlight how modern technologies lack in terms of spatiality. Following the rising popularity of spatial focus, the recently developed notion of Human-Building Interaction (HBI) aims to understand what challenges and opportunities are in the cross-section of architecture and HCI [4] . As a further elucidation of HCI and architecture, Kirsh [64] describes how the fields differ in understanding what interaction means, and HCI could serve well to employ more networked modes of interaction. At the core of these modes of understanding spaces is the social quality of spaces [64] . To this end, understanding spatial experiences from the perspective of everyday spaces has been brought up as a relevant research focus in HCI [98] . Immersive technologies such as augmented and virtual reality open new opportunities to study spatiality in HCI. The immersive capabilities of VR enable users to experience different spaces, while AR augments everyday spaces with digital content. The recent focus on the metaverse serves as a way to produce persistent spaces in the virtual world, mirroring the relative stability of the physical environments.

XR, empathy, and spatiality have all been subjected to various literature surveys in the past. However, the specific intersection of these which is pivotal to empathy research has not been examined earlier. In a recent survey, Pratte et al. [105] surveyed the use of empathy tools in HCI research. From a set of 26 publications, they clarified the language around the audience of empathy tools, analyzed used technologies, and the role of agency, perspective, and sensation for fostering empathy. Finally, they presented recommendations for HCI research for empathy tools. To build on this work, in the present survey we focus on the spatial context of different empathy tools. In the context of caregivers, Brydon et al. [17] reviewed 7 studies using VR empathy tools. The findings supported using VR technologies to elicit empathic behavior in caregivers. In our approach, we also consider XR technologies. Using VR nonfiction (VRNF) media, Bevan et al. [10] analyzed 150 VNRF titles to highlight what narrative techniques were used to elicit empathy. The findings suggest the relatively new media is experimenting with suitable paradigms for interaction, viewer roles, and more. In contrast, we target XR applications that can also have a non-narrative focus. In surveying VR empathy tools, W.

B. Owais and E. Yaacoub [130] formulated a framework for immersive virtual reality environments. However, the scope was quite limited and is not representative of the current state of research. Moving into the spatial realm, Danyluk et al. [26] reviewed how virtual reality tools were used to present and interact with Worlds-in-Miniature (WiMs), which are interactive representations of things ranging from small objects to a planetary scale. The multiple scales afforded by the XR technologies provide a shift in a spatial context, allowing new perspectives. These spatial contexts serve useful lens for the empathic focus in our survey. Lee et al. [75] surveyed the urban spaces as the gigantic immersive scenarios with augmented reality, and further brought up the consideration of various stakeholders in such immersive urban spaces. ("empathy" OR "sympathy" OR "pity" OR "compassion") AND ("urban" OR "space" OR "place" OR "building" OR "city" OR "environment") AND ("virtual" OR "mixed" OR "augmented" OR "diminished") AND "reality" However, their works neglect the empathy in such physical spaces that is evoked with immersive technologies. Finally, Paiva et al. [100] surveyed research on virtual agents and robots with regards to computational empathy. They highlight how virtual agents can serve different roles in empathic interaction, which in turn affects how they are perceived. In contrast, we are not as focused on computational empathy, but rather the tools that can be used to elicit empathy.

This survey reviews recent work on how empathy, spatiality, and XR technologies together enable novel forms of empathy research and experiences. We present the state of current empathy research that uses XR technologies and how different spatial contexts are used in these approaches. As the outcome of the survey, we contribute a research roadmap that builds on identified opportunities within the intersection of the reviewed fields.

We answer the following research questions in the context of empathy research using XR technologies:

(1) What tools or technologies are used and how?

(2) Who is being empathized and by whom?

(3) What perspectives are taken in empathy? (4) In what spatial contexts is empathy elicited? Section 2 describes our systematic review process and data extraction. In Section 3, we describe the different themes brought up through the synthesis of data extraction. Section 4 describes the research opportunities we identified and discusses an array of related issues. In Section 5, we outline a research roadmap for spatial empathy tools that leverage XR technologies. Finally, Section 6 concludes the survey.

To conduct the literature survey systematically. we applied the PRISMA methodology as described by Page et al. [99] .

Web-based tool Covidence was used to handle the process in collaboration with multiple authors.

2.1.1 Keywords. The chosen keywords were in the three dimensions of empathy, spatiality, and XR technologies. To take into account the different scales and contexts of spatiality, keywords "urban", "space", "place", "building", "city", and "environment" were used. Empathy was focused on with the related terms "empathy", "sympathy", "pity", and "compassion". Finally, to cover the wide range of different XR technologies we targeted "VR", "AR", "MR", and "DR" -keywords. The keywords and the resulting general search string is described in Table 1 .

We primarily used ACM Digital Library (ACM DL) and IEEE Xplore Digital Library to focus on technology-and computer science -related literature. For a more thorough review, we also included Web of Science, Scopus, EBSCOHost, and Elsevier (ScienceDirect) to cover fields such as architecture and psychology that relate to spatial contexts and empathy. Using the previously described keywords, we formulated the search strings for the specific databases, as described in the Appendix A, alongside the search results. We note that Elsevier allowed only 8 Boolean operators in the search string, reducing the query length. Also, for ACM DL we targeted the search string to the Title, Abstract, or Keywords for more relevant results.

To extract relevant information from the included articles, we developed a data extraction rubric. Initially, the first author of this survey selected 10 articles pseudo-randomly (sorted by author name: [2, 5, 6, 14, 16, 18, 19, 66, 70, 111] ), and after reading the articles, developed items around the relevant aspects of XR technologies, empathy, and spaces.

The initial data extraction rubric was then evaluated by all authors and finalized into the rubric described in the Table 2 .

Using the finalized rubric, the first and second authors conducted the data extraction for each article, and for any conflicts, consensus was formed through discussion.

The data extraction items DE1 -DE4 are used for general descriptors for the papers -study ID, title, publication type, and keywords. The XR technologies (DE5) described the most general types: AR, VR, VR video (non-interactive),

MR, CAVE, and other. Interaction (DE6) modalities show the range of used hardware and how they can be used in the virtual environment. Locomotion (DE7) describes how the user moves in the virtual environment, whether through their actions (e.g., using a controller), or through a scripted movement (e.g., moving video). Measuring empathy (DE8) was focused on the general scientific measurement methods of qualitative, quantitative, and mixed, and also N/A in the case empathy was not measured or unclear. Measuring the longer-term effects of empathy (DE9) has been recognized as a good practice for understanding the impact of empathy tools [105] . To clarify the language around the subject and object of empathy, we use "user" for the person using the empathy tool and "Other" for the human, animal, or entity we are empathizing. As such, embodiment for the Other (DE10) and the user (DE11) were categorized by general types of human, animal, robot, object, and other. Concepts such as "nature" or "sustainability" were marked as "other".

The items DE12 and DE13 describe how the empathy experience is presented to the user. The point of view (POV) compared to the user (DE12) refers to how objectively the experience is presented, or how much control the user has for experiencing the story (subjective). The narrative point of view (POV) of the user (DE13) categorizes how wide understanding of the situation the user is given. The items DE14 and DE16 are related to the context of the situation, through the notion of proxemics, the study of distances [51] . The type spaces in the experiences (DE14) describe the qualities of the different proxemics distances. To exemplify these, being a remote observer is public, conversation with someone is social, focusing on your near surroundings is personal space, and the intimate is reserved to the spaces most closest to you. Distance in the empathic interaction (DE16) is based on how far the user is from the Other. Empathizing perceptual experiences (i.e. vision or hearing impairment), were marked as "intimate". Another dimension of the quality of spaces is captured by the privacy of the spaces (DE15), ranging from public to private. A semi-public space is a space that is meant for the public, but the access is partially limited (e.g. a shopping mall), and semi-private space is limited to its owner but is occasionally accessed by others by permission (e.g. a backyard). The Other of empathic interaction (DE17) refers to who, and with how many people, are experiencing the empathy. One-to-one means the user is experiencing a single person's experiences in the virtual environment. Accordingly, many-to-many is used for situations where several people are participating in the experiment, and are empathizing many Others. One-to-one, one-to-many, many-to-one, many-to-many, other 

The outline of our systematic review process using PRISMA is shown in Figure 2 . After completing the search, we started the screening process with 588 articles, of which 204 articles were removed as duplicates. We conducted the title and abstract screening process for 384 articles, of which 190 were excluded. After the full-text assessment, we excluded further 125 articles. The justifications and the counts (in parenthesis) for excluding the 125 papers are as follows: no XR technologies (17), is a survey paper (15) , no spatial context (14) , not a paper (14) , duplicate (13), a proposal (13) , no focus on empathy (13) , no empirical study (12) , empathy only a side-effect (5), only a framework (5), full-text not found (2), is a book (1), and non-English (1). Finally, we were left with 69 papers on which we conducted the data extraction. 

We extracted relevant information from the 69 articles that resulted from the screening process. This information was then analyzed and summarized statistically and graphically. Further qualitative insights were brought up in the iterative analysis process. The full list of the included articles is presented in Appendix B. The articles were from years between 2006 and 2021, and the number of publications per year is shown in Figure 3 . It seems there is a general increase in the focus of our survey, however, aspects such as an increase in XR technologies, digital archival of research, and global scientific output can confound the results. As for the publication type, we had 42 (61%) journal papers, 26 (38%) conference papers, and 1 (1%) thesis. As our data extraction took place in mid-2021, the results for that year are not representative of the whole year.

The articles come from various scientific disciplines: besides computer science, fields of medical, educational, and journalism science had employed XR devices for empathy. Most frequent keywords mention "reality", "virtual", and "empathy", after which the frequency of the keywords falls significantly, as shown in Figure 4 . The wide range of the keywords is also visualized in Figure 5 . While the keywords represent the themes of XR technologies and empathy, the theme of spatiality is not immediately present. This can be an effect of spatiality serving as the context for experience and is often less focused on, rather than the experience itself. 

The included 69 papers cover a wide range of topics in empathy and XR research. Alongside the domain of HCI, empathy tools are applied fields such as medical [29] , psychological [22] , educational [19] , journalism [118] , media [45] , religious [60] , and geographical sciences [47] . The following sections describe how our three research themes are present in the articles, leading into the discussion, where we describe the general trends and approaches for fostering empathy with XR. Figure 6 shows the usage of different XR technologies over the years. Using XR for fostering empathy has been increasingly popular, with VR being the most popular technology. Following the release of high-end headsets such as Hololens in 2016, AR technology has been steadily investigated, and studies using multiple technologies have seen a significant increase in popularity. Specifically, VR is the most used technology (43%), followed by VR video (12%), desktop monitor (9%), and AR (7%) approaches. There is also a significant amount of papers (14%) using multiple technologies, to assess the empathic effect across different media. For instance, several papers performed their studies' experiments on different platforms, such as VR and VR video (e.g., [18, 22, 121] ).

The studies using multiple technologies can be categorized in two ways: studies using technologies with different levels of immersion, and studies where technologies with comparable complexity were used. As an example of technologies of different levels of immersion, Bujic et al. [18] used a 360 HMD video, a 360 video viewed on a desktop monitor, and an internet article read on a monitor, i.e., high, medium, and low immersion, respectively. Similarly, P. R. Jones and G. Ometto [97] developed and tested AR and VR technologies for eliciting empathy on visual impairments. As an example of non-comparative technology-induced empathy, Stover [119] employed computer simulation and reading tasks to foster empathy. Furthermore, in the perspective-taking process of understanding the Cuban Missile Crisis, experimenters directed the study participants, students, to employ an active approach to write about their viewpoints. Mixed reality studies used consumer-market hardware such as Microsoft Hololens [6, 103] , or semi-custom HMD solutions with additional modalities [66, 70, 74, 133] to elicit empathy. For example, Lee et al. [74] built an "unconventional driving machine", where the user lies in a steel frame of a small buggy. A 3D camera and light detection and ranging (LiDAR) sensors provide visual feedback to the user through a VR HMD. As such, drivers or vehicle designers can understand the perspective of an autonomous vehicle and how it impacts the urban environment and other people.

In a similar approach, Kors et al. [66] designed an experience around the user's sensory system through audio-visual, touch, and olfactory stimuli. The experience of being a refugee hiding in a truck is enlivened by seating the user in a small wooden crate and accentuating the virtual visual feed with smells, touches, and sounds.

The overview of XR technologies, interaction paradigms, and locomotion in the studied works is shown in Figure 7 .

Interaction describes how the users were able to change the state of the world through their actions and what methods facilitated it. The chosen XR technologies partially dictate interaction methods, and VR is the most popular choice for eliciting empathy, using gestures and devices for interaction. In contrast, VR video describes experiences where the user is witnessing a filmed story, and as such, interaction is limited. However, Radu et al. [106] used controllers to progress the story in a narrative VR video experience. As such, the interaction enables the user to progress the story at their own pace. Furthermore, we observe that many of the VR video experiments did not use locomotion, i.e., were focused on the user being a stand-still observer.

83% of the papers included in the study have used a form of a quantitative assessment of empathy, such as the selfadministered IRI or QCAE -questionnaires, or a subset of them. Around 14% of the papers focused on qualitative assessments of empathy, such as interviews [39, 62] or analysis of behaviour [65, 112] . One-third of the papers used both quantitative and qualitative methods to understand the empathic effect. Further, 17% of the papers did not measure empathy and often focused on some forms of empathy tool prototypes [23] , or the topic was on empathy, but the measurement instruments did not include empathy [49] . 10% of the included studies measured the long(er)-term effects of empathy in their experiments. As shown in Figure 8 , longer-term effects were mostly measured in studies, which used mixed methods to measure empathy. For example, Adefila et al. [2] interviewed a subset of the users after an empathy experiment on dementia, and Herrera et al. [58] focused on the effect of empathy two, four, and eight weeks after the experiment. Out of the 7 papers measuring long-term empathy, 3 articles focused on educational contexts. For instance, Efstathiou et al. [35] found the AR learning condition leads to greater historical empathy as compared to the non-technological approach. Furthermore, out of the 7 papers that measured long-term empathy, 4 articles reported a longer-term qualitative or quantitative difference in empathy from using the empathy tools ( [1, 35, 47, 125] ). However, as the aforementioned studies did not use similar methods, it is not possible to compare the long-term effectiveness between the different approaches.

Most frequently, the users empathized with humans among all the studies we examined. In 84% of the papers, empathy was directed towards humans. In contrast, 13% of the papers focused on empathizing objects and 3% animals.

As an example of empathy towards animals, Clayborn and Delamarre [23] designed a VR experience where the user completes tasks to understand the living relationships of butterflies, plants, and exotic species. Similarly, Kelly et al. Fig. 9 . What is the user embodying compared to the Other [62] built an AR tool to show the world through dogs' eyes, with their limited color accuracy. Objects or concepts that were emphatized included self-driving cars [74] , historical events [112, 121] , and sustainability practises [71, 79, 83, 90] .

Embodiment follows a similar distribution as for the Other: in 86% of the papers, users embodied a human, in 4% an object, and in 1% of the studies they embodied in an animal. It is worth mentioning that 9% of the papers had no clear embodiment and we mark them as N/A, e.g., in the case of many virtual 360 videos where a physical body is not visible.

The relationship between the Other and the embodiment is shown in Figure 9 . In two papers, the users experienced from a non-human perspective to understand a non-human perspective: seeing the world through a dog's eyes [62] , and self-driving cars by acting as one [74] . While there were studies that used multiple human perspectives [117] , no studies were using multiple types of embodiment.

The empathy experiments were mostly subjective (54%) or objective (45%). However, one study by Shin [114] used an omniscient viewpoint, where the users viewed a 360 documentary describing the effects of war on children. As such, the stimulus presents multiple experiences of the children in the documentary.

First POV is the most used viewpoint (91%) to show experiences through the eyes of another. Four studies used the fly-on-the-wall viewpoint [18, 34, 82, 123] . A study by Griffin and Muldoon [47] used the omniscient POV to show the effects of slum tourism through a documentary. Both the first and third-person perspectives were used in the study by Tsai et al. [125] , which aimed at improving emotion recognition in children with autism through a CAVE-like system.

The included publications cover a large range of spatial contexts in which empathy is elicited. For instance, the studies used environments and spaces such as religious sites [60] , slums [47] , museum [93] , library [106] , cafe [40] , school [82] , nature [23] , road [131] , sea [71] , office [118] , truck [66] , doctor's office [29] , and home [52] . A selection of different spatial contexts is presented in Figure 10 .

Almost half of the papers (48%) took place in a public space, as shown in Figure 11 . Accordingly, most of the experiences in public spaces also used public distances. 20% used intimate space, 17% social space, 10% multiple spaces, and 4% personal space.

Nearly half of the papers (48%) took place in a public context, compared to semi-public (17%), multiple (14%), private (9%), and semi-private 5 (7%) contexts, as shown in Figure 12 . In three papers, the privacy of the place could not be determined (N/A) [34, 48, 96] . The private context of the home was used effectively in the study by Hamilton-Giachritsis Fig. 10 . Selection of various spatial contexts in the included papers, ranging from public (top left) to private (bottom right), as follows: a) urban environment [109] , b) nature [43] , c) classroom [39] , d) workplace [128] , e) public transport [58] , f) underwater [83] , g) museum [57] h) elevator [127] , i) doctor's office [29] , j) home [52] , and k) home [110] . et al. [52] . The study had sampled mothers for participants, who then embodied a young child and experienced aggressive behavior from their virtual mother, in the private context of the home.

We mark cases of direct sensory manipulation as intimate distance, e.g., in the case of visual impairment [25, 43, 97, 110, 133] , schizophrenia [61] , or anxiety [127] , due to the immediacy of experience. However, the decisive location of experiences, and by extension, distance from the person experiencing them, is difficult to assess. This aspect of Fig. 11 . Space of the empathic interaction compared to the distance. experience does not translate well into the geometric understanding of space, and more vague descriptions can be more meaningful. In other words, proxemic and empathic distances are not easily understood together.

In most cases (61%), there was a one-to-one relationship between the user and the Other. It means the user was alone in the experiment and was focused on the experiences of a single entity. Then, 19 studies (28%) used one-to-many relationships, 6 studies (9%) used many-to-many relationships, and 2 studies (<2%) used many-to-one relationships. To highlight one-to-many relationships, studies used a classroom bullying scenario through the eyes of the teacher or the person being bullied [117] , experience of multiple occasions of sexual harassment [129] , and an educational approach to teach different religious practises [60] . Many-to-many relationships focused on a more complex setup of users in an empathy situation. Menzel et al. [84] studied poverty by placing the users in different family roles, each with their own goals, to experience poverty and build empathy. In a meditative application, Salminen et al. [111] set up an experience where multiple users participated in synchronized breathing exercises to elicit empathy. In the Cuban Missile Crisis simulation by Stover [119] , the students assumed roles of Cuban, Soviet, or USA officials during the Cold War, and developed empathy through understanding the complex relationships between the different countries, instead of a sole description of individuals, i.e., soldiers' faces with tension and fear. The two papers using many-to-one relationships employed multiple interaction possibilities to feel the autism of a single person [49] and historical empathy where multiple students imagined life in a bomb shelter [112] .

As shown from the description of our included articles, there is clear variance in the use of different technologies, empathy approaches, and spatial contexts. Based on the findings in Section 3, we now tie these different threads together and raise an in-depth discussion on the different dimensions of designing empathy tools for XR.

Understanding Spaces through Context. Spatiality offers an analytical lens for examining empathy tools, as specific spatial contexts can be understood differently by different people. For instance, in the educational approach of presenting different religious sites and practices [60] , the various meanings related to the temples, synagogues, etc., are not readily understandable by all people experiencing the environment. As such, the users of empathy tools are using their present understanding to make sense of the presented, possibly foreign, contexts. To understand how these different modes of experiencing spaces work, we borrow the notion of "context collapse" [28] , which describes how all-encompassing social media makes it difficult to keep an individual's social circles separate.

As people tend to adjust their behavior based on the social context, person may find it difficult to act meaningfully when these different social contexts are mixed. The context can blur either intentionally ("context collusion"), or not ("context collision"). Davis and Jurgenson [28] recognize events such as weddings and funerals as examples of contextual collapses in the real world, where maintaining a singular role and cohesive behavior in a mixed group of friends, family, relatives, and acquaintances can be difficult. While the physical world is tied to the constraints of space and time, the virtual environments are more fluid by definition, enabling more contextual blur. We see use in approaching the continuum of virtual empathy (see Figure 1 ) through contextual collapse, when the real and virtual contexts are intertwined. For instance, the spaces and norms that VR spaces represent in the real world can clash with the "no rules" context of VR [92] . The results of this context clash was seen in the VR metaverse application Horizon by Meta in 2022, where users faced sexual harassment due to a lack of personal boundary in the virtual space, leading to an implementation of an enforceable personal boundary to mitigate unwanted behavior 4 . The lack of pre-existing context in virtual reality applications and the unlimited representations afforded by virtuality lead to a wide array of new contextual implementations, and contextual collapses. On the contrary, AR experiences are much more dictated by the spatial context of our physical surroundings. Considering the number of viral VR "fail" videos where immersed users forget their physical environment and hit themselves or others, it is interesting how similar examples of AR are hard to come by [27] . As such, we posit the contextual collapse is greater with more immersive technologies, that is, VR. In order to truly experience empathy in virtual environments we suggest one needs to suspend their disbelief about the virtual experience and accept the new context it provides.

In this sense, the users needs to give in to empathy in a virtual environment. While a person's characteristics affect the elicited empathy, the virtual environment and its context play a role. A believable XR experience builds a more understandable context and supports the user's willingness to empathize. We can also draw parallels between the privacy of space and context collision. In a private setting, regardless of the level of virtuality, the presence of strangers introduces contextual collision. On the flip-side, in public environments presence of other people can enable contextual collusion, leading to encounters missed in private settings. For instance, placing the user in a public environment, such as a busy metro, but focusing on private matters can bring a contextual collapse that seems unbelievable. Then again, unbelievability can act as a device through which foreign experiences are accepted. One study used the spatial context of a drug store, to highlight the disturbing personal symptoms of schizophrenia [61] . The paper goes on to suggest that "simulations without the context of . . . supplemental resources may lead to outcomes that are not only inferior but are even counterproductive to learning goals", enforcing the need to use necessary contextual support in empathy tools.

Spatial Experience for Eliciting Empathy. The discussion above highlights the contextual understanding of spaces for empathy, but we can approach spaces from another perspective as well. That is, how could a person's spatial experience elicit empathy? In other words, is the way someone experiences spaces something we can empathize about? What kinds of experiences could these be?

In this manner, we see two different sources for empathic spatial experiences. First, starting from the original concept of empathy, Einfühlung, we can see how art and architectural forms can be understood as a source of empathy [44] .

Recent advances in neuroscience have shown links between the visual perception of art and the mirror neuron system 4 https://gadgets360.com/social-networking/news/meta-horizon-worlds-vr-metaverse-personal-boundary-ring-feature-sexual-harassment-report-2750506 [113] . However, this approach requires a distinction between empathy and affect. An impressive architecture can elicit strong feelings, but strictly speaking, it is not empathy however valuable as an aesthetic experience it might be.

Regardless, we see value in the virtual environments, where the structural forms of the environments are not bound to the rules of the physical world, which enables experimentation with new kinds of forms and empathic experiences.

The second way the spatial experience can be emphatized is through someone else's experience of space. As empathy tools use experiences to elicit empathy, we see spatial experiences as another avenue for empathizing. These spatial experiences can include place-dependent experiences, such as safety, accessibility, or aesthetic appreciation. As our understanding and experience of spaces is multilayered and complex and tied to personal history and sensemaking, the richness of experience could serve well to elicit empathy. However, a concentrated study is needed to clarify how we understand and experience these shared meanings of space. Similarly, we could benefit from further explorations on how a given empathy tool works in different spaces. For example, how could a person's experience of social anxiety differ in public and private spaces? Or how the use of multiple POVs could show nuanced aspects of the spatial experience?

Furthermore, real-world spaces are living and the atmosphere can change in an instant, for example in an accident, act of terrorism, or fire. Introducing the layer of empathy and virtuality raises the question of how they can address and convey changes in the atmosphere of the place? Moreover, in an AR system, the user is collocated in the space with others. When the empathy tool elicits the experience of others successfully, how does it change the atmosphere?

Through these explorations, we see architecture and HCI form a more nuanced understanding of spaces.

Frictional Spaces. The XR spaces aimed at eliciting empathy are, in essence, environments for learning. What is unique in these spaces is that they provide connections between humans, and other humans, animals, nature, or objects.

As the goal of the empathy tool designs is often not the artifact in itself, but rather, the experience of empathy, its methods resemble alternative design approaches. The concept of "frictional design" [102] describes how alternative designs have goals that differ from traditional, progressional, concepts of designs. Following this notion, "frictional spaces" can be understood as spaces with an alternative purpose. Dwelling, described as the act of making one's home [55] , works towards building immunity from the outside's unpredictabilities. As such, dwelling is orthogonal to frictional spaces. In contrast, the frictional spaces are not meant as environments for continuous dwelling, but rather, they are experienced similarly as one experiences art.

Following this line of thinking, could one consider the real and day-to-day spaces we share with others as frictional spaces? We do not always prefer coexisting within a space, but it can lead to chance encounters, through contextual collusion. Using the examples of a bus stop and a spatial art installation we highlight the range of frictional spaces from banal to extraordinary. As such, our everyday spaces can indeed be considered frictional spaces. What is emblematic of these frictional environments is that inhabiting them is not necessarily comfortable, or even the end goal. Transitional and liminal spaces also share these qualities but are more defined by their use of moving through or waiting in, for instance.

The virtual spaces offer a simulation of these frictional spaces. Notably, virtuality also leads to a lack of real-world consequences. This might explain the challenges in using virtual environments to elicit real-world empathy. Nakamura [92] argued all spaces in VR are public as there is no privacy to maintain. The non-playable actors in the virtual environment are not in control of the public and private spaces of the environment unless specifically designed. In contrast, AR technologies are grounded in the real spaces and their concept of privacy. Similarly, XR (AR and VR) companies also need to be considerate for protecting users' privacy. For instance, the sensor-heavy XR technologies can gather data from a wide range of human actions and their surroundings, e.g., their behavior, biometrics, speech profile, and other people you share a space with 5 .

Empathizing the Past, Present, and Future. With XR technologies, empathy tools are not limited to contemporary experiences. If there is a perspective known to a sufficient degree, it can be attempted to empathize. Now, the design of these empathic experiences relies on assuming some parts of the experiences of the Other, like all empathy simulations, so the development process should be carefully valued between accuracy and the effect. For instance, take the world of literary fiction, where a skillful writer can convey a host of feelings to the reader, to elicit empathy towards the characters. As an example, a short story "Land Deal" by Murnane examines the relationship between Australian aboriginals and colonizers through an exchange of goods for land. The tensioned dynamic between these two parties resolves in the aboriginals' suspension of reality, and in the belief that they are living a dream. The story is a display of writing to highlight the experiences of the Other.

Following a narrative approach, the studies on historical empathy found in our survey serve as a glimpse to the past [119, 121] . Using historical knowledge, we can present viewpoints that would be otherwise inaccessible to us. These are most often based on some understanding of the past, e.g. through written records, historical knowledge, and more.

However, we see that historical fiction could be a new avenue to explore, as it was not found in the empathy tools in the present survey. In empathizing the past, we can enrich history and bring more nuanced perspectives into our present day.

The benefit of empathizing the present allows the users to relate their experiences more easily, as there is most often more contextual common ground. As empathy tools often aim to foster a behavior change, it could be argued that empathizing the present can be more valuable for the current times. For instance, empathizing hardships of wartime, when living in peace, might not be a directly applicable experience.

Empathy can also be used to target future speculations, although such aspect was not found in the articles considered in our survey. However, Thibault et al. [124] used pastiche scenarios to highlight different aspects of the future of smart cities. These scenarios can elicit empathy and provide viewpoints that are not limited to experiences that have happened.

The literary genre of science fiction is often attributed to future speculation and can provide these views of the future that elicit empathy. These alternative futures can shape our present understanding, and widen our possibilities for the current times. As such, the future prospects offer a non-linear mode of thinking and exploring our present world.

We can also focus on how long we experience the empathic stimulus. Anecdotally, nearly all empathy tools are relatively short experiences (under 30 minutes), but research on longer immersion (e.g. in metaverse) is close to none.

However this can be partially explained as VR technologies can be prone to cause sickness, their use is often limited to shorter durations, especially for the uninitiated. In contrast, AR environments can be a more ecological approach for eliciting empathy in longer durations due to a smaller amount of sensory manipulation and therefore less sickness.

Contrasting these XR technologies with literature, we see that most literature is experienced over a longer period of time, which possibly serves to build a stronger feeling of empathy. However, research on the long-term exposure to the empathy stimulus is limited, and further research would help to elucidate how the duration of the experience affects empathy.

To elicit empathy in a person, the choice and design of the suitable stimulus play a large role. The shape of the empathy tool can be dictated by the desired experience, available technologies, and resources, or the context of the empathy situation. Accordingly, these aspects can act as the driving forces behind developing the empathic stimuli and facilitating XR empathic experiences.

Empathy Technologies in the Virtual-Physical Spectrum. In the studies that compared the empathic effect across different technologies (e.g., [18, 19, 118] , the more immersive approaches often elicited more empathy, as opposed to low immersion conditions such as reading or viewing the experience on a desktop monitor. Similarly, another study expressed this immersion through a model where immersion and related concepts (e.g., presence, flow) are linked with more empathy [114] . While evaluating several technologies is a sensible choice in terms of a study design, it raises the question, to what extent are these experiences are comparable? For example, comparing a reading task to a virtual HMD experience in terms of immersion (e.g., [58] ), naturally the modern understanding of virtual reality immersion is greater in the latter. Understanding the nuanced way these different technologies operate supports designing the empathy stimulus more meaningfully. Furthermore, the same holds true when using different yet complex technologies. In augmenting the physical world with digital content, AR experiences can strengthen empathy in environments familiar to the person through additional information, whereas VR experiences support experiencing empathy in contexts which would be otherwise not "found" in person's own life. It is worthwhile to mention that, albeit this paper has traced back to earlier works in 2006, we rarely find CAVE systems for eliciting empathy, which only appeared in 2013, 2019, and 2021. The CAVE systems refer to some room-size immersive environments, which are primarily employed in industrial applications, such as staff training and consumer product experiences [88] .

The 360 VR videos proved to be an ecological choice for many (12%) study designs ([47, 60, 90, 94, 106, 122, 126, 129] ).

We were surprised by the relative prevalence of VR 360 videos, which in the hindsight are relatively simple to use. The rise of VR as a medium can be explained through the increased adoption of consumer-grade VR headsets, reaching a critical mass [63] . Also, the existence of 360 video platforms, such as YouTube, helps facilitate highly diverse empathy experiences. Even more, the use of playlists allows for a curated selection of empathic material, such as in the case of religious education [60] . If one considers the social benefit of using an empathy tool, an easy-to-use platform with curated experiences can support eliciting empathy on a wider scale. From our observation of various empathic scenes in VR and 360 videos, virtual environments have the advantageous properties of malleability and immersiveness that facilitate various imaginary scenes. It is important to note that certain scenes, e.g., child abuse [52] or drug usage [22] , cannot be replicated in real-life scenarios, due to the high costs of human actors and ethical issues (e.g., mental and physical damages caused by killing and hurting) of replicating such scenes. Thus, virtual environments play an indispensable role in eliciting empathy through putting users in scenes with strong sense of presence [46] . Nevertheless, prior works reveal that the skills or knowledge acquired in virtual environments can be disconnected from the scenarios in physical world. Therefore, augmented reality can play an auxiliary role in visual cues to remind the acquired skills and knowledge during applicable contexts [24] . Similarly, AR can potentially complement VR to extend the entire cycle of empathic journey -VR aims to elicit users' empathy, while AR reminds users of the experienced empathic scenes in matched contexts, achieving transfers of "empathic learning". Even though we only examine limited works leveraging AR, understanding how social, location-based games such as Pokemon Go [5] can elicit empathy or historical empathy for enhanced learning [35] , AR can bring in-context experience and arouse empathy in physical spaces.

Interaction Modalities. Virtual entities can merge with our physical living space and provide various functionalities and utilities [75] . Visual overlays in augmented reality can serve as an effective visualization tool to inform users regarding e.g., privacy threats, along with delivering empathic content to users. Thus, virtual spaces could serve as a more explicit and swift communication than traditional 2D UIs [9] .

On the other hand, researchers in the broadest domain of virtual environments (i.e., not limited to empathy-related studies) attempted to explore multiple interaction modalities to enrich the user experience, e.g., gestural interaction plus haptic feedback [8] . In such empathic immersive environments, the users had used one or more interaction modalities, including tangible devices like keyboards and joysticks, as well as embodied interaction like gestures and speech ( Figure 7) . As stated in prior work on city-wide user interaction with immersive environments [75] , user bandwidths, i.e., the user ability to convert their intents into actions in virtual environments, drops significantly in virtual environments.

When both the interaction modalities and the levels of empathic virtual spaces are considered together, two questions remain unexplored -(1) What is the threshold of user bandwidth that guarantees the experiences of eliciting empathy ;

(2) Do richer interaction modalities also elicit higher levels of empathy?

User Agency in Experiencing Empathy Tools. Our collected articles reveal that empathy can be taught (i.e., aroused) in virtual environments. For instance, examples include the training of staff in medical services and education institutes [65, 123] . Virtual environments allow the staff to be situated in the role of victims, e.g., physical or verbal violence.

Through such perspective-taking approaches, the staff had gained awareness of violent behaviors and understood the perspectives, and perhaps experiences, of the persons being abused.

Noticeably, the content designers of virtual environments bring a level of agency to their audiences. In virtual environments, agency refers to the extent to which users can interact and modify objects in the scenes [59] . Users with virtual environments of high agency levels can employ a more active approach when interacting with the empathic content to experience empathy. Therefore, eliciting empathy is subject to agency, creating either curated virtual experiences [90] with low agency or free-to-wander experiences [57] with high agency.

Apart from the agency that describes the flexibility and richness of virtuality, we consider that locomotion (Figure 7 is a critical user-centric factor that empowers agency. Locomotion, through various interaction types, enables greater agency for the user in the virtual environment. Users leverage interaction techniques (gestures and keyboards) to explore (e.g., walk and search) the environments and the intended experiences regarding empathy.

For instance, the underwater diving simulation [71] allows the users to freely explore a seascape in VR. The user can control a virtual Diver Propulsion Vehicle (DPV), to locomote underwater. The game focuses on exploring the diverse seabed scenes to elicit empathy towards underwater sustainability. As a side-effect, the authors note the safe VR experience can be used to surpass users' phobias and physical disabilities.

As the collected articles employ highly diverse scenarios, but inconsistent evaluation methods, we cannot state to a definitive conclusion on how agency and users' active participation correlates to the effectiveness of eliciting empathy.

Nevertheless, it is important to note that the learning process of empathy can be facilitated by a process of active participation (see e.g., [119] ), supported by a high agency level. In contrast, a low agency level only allows users to receive information in less interactive yet unidirectional flows. Therefore, the relationship between the agency and the easiness of experiencing empathy requires further investigation. A very limited number of studies reflect the accuracy of user experience in such virtual empathy situations, and we have very limited knowledge of how faithful to the original experience is the simulated experience. In addition, as the existing works have no comparable virtual scenes and different empathy goals, the connection between designing fictional events and the process of empathizing with users' personal experiences poses significant research opportunities.

Embodiment and Empathy. The included papers represent embodiment beyond humans and include animals, nature, and more. As such, they provide a perspective into non-human entities as well. As no paper in our survey used multiple types of embodiments, we highlight this as a future research potential for studying empathy. To this end, we present two relevant explorations. First, Krekhov et al. [68] studied how non-human representations can be used in VR. Their experiment with three animals (rhino, scorpion, and bird) implies an illusion of virtual body ownership (IVBO). Through the experiments with different animals, they recognize more work is needed in both hardware and software to produce believable and enjoyable VR experiences with animals. Second, the experimental video game "Everything" 6 by artist David O'Reilly empowers the player to embody numerous amounts of non-human entities. The player can embody and control things as small as particles, systems as large as galaxies, and everything in between. In this process, the game focuses on the interconnected nature of the universe and builds perspective-taking on a massive scale. The game serves as a useful frame of thinking in multiple scales and contexts for empathy.

To this end, XR serves as a fruitful enabler of experiences with varying embodiments. Through VR, we can craft new experiences, which are less related to the spatial context of the physical world, and are more focused on the unique possibilities of world-making and experimentation with scale. As shown in MR studies in our included articles, the additional sensor and interaction technologies allow a more immersive and embodied empathy [74] . As the AR technologies operate in the spatial context of the real world, its inclusion of non-human empathy can help to ground us in our environments. The AR enjoys a strong potential to present augmented information in its context. In this sense, while in our view AR is often related to human embodiment, it can show the world through adjusting the perception of real-life (e.g., see [62] ).

Finally, we can consider empathy beyond the scope of a single study. We understand and use empathy through different means and with different goals in mind. Who designs these empathic experiences matters and what values are represented therein. Through careful consideration, we evaluate empathy and its virtual possibilities as a fair mechanism for self-reflection, for the society at large.

Developing empathy tools raises the question: who designs them and with whose experiences? As empathy experiences are most often concerned with personal viewpoints, using someone's experiences for empathic stimulus is challenging. The level of fiction in the representation of the emphatized experience affects how close to the "source material" one is. In some cases, representing someone's experiences can be understood as a process of sustaining stereotypes [92] . For instance, while racist experiences can be used to elicit empathy towards their victims, they also reproduce those events. As such, being considerate and respectful of someone's experiences is essential, and a critical approach is needed to produce empathy that serves the common good. While empathy is most often seen as a positive aspect of human social behavior, it is the methods with which we produce and reproduce empathy, that can also harm. Another concerning aspect we found in our surveys as well as the varying respect of privacy. Filming vulnerable populations such as children, underrepresented, low-income, homeless, etc., has an inherent risk of exploitation. Respecting others' experiences also requires respecting their privacy and choice not to be used as means for empathy. To engage with the Looking from the perspective of the users reveals another point of discussion: for what purposes do people use the empathy tools? In the realm of entertainment media, we can see how empathic games can provide engaging entertainment. Then again, serious games have goals less related to entertainment [30] , and more related to education and perspective inducement. As only a few empathy tools presented here are purely entertaining by nature, we see designing these approaches to have more similarities with serious games. The goals of serious games are in a way directed outward, towards other people. As such, we can see empathy through the lens of doing social good, through nuanced behavioral change. In contrast to the financial care towards others in social good like charity, behavioral adjustment with empathy tools could serve as another form of tool-assisted altruism. To see empathy as the social glue also requires us to recognize moral aspects of understanding another, and ultimately, how should person's character be developed. Granted, this line of thinking can lead to wicked problems about the way person's behavior should be changed, and who has a say in the matter.

It is also relevant to consider what relationships are fostered, and what understanding is transmitted when designing and using empathy tools. The designer, the user of the empathy tool, and the person whose experiences are used for empathy are in an interesting dynamic of meaning-making. We can represent these relationships diagrammatically through Figure 13 . Notably, the relationship between the empathy tool designer and the Other are not easily explainable.

To what extent does the designer participate with the Other in designing the experience? Conversely, how does the Other benefit from this relationship? Having these multiple lenses means understanding empathy purely from a single viewpoint in a robust fashion can be difficult. For instance, the designer expects and hopes to elicit empathy, which can inadvertently affect the user's experiences, which can be challenging in the academic context. Further, the person whose experiences are used might be compensated for their input but is never actually empathized in the real life. Similarly, the user can experience great empathy, while they might never encounter a similar situation in the future. This echoes Bateson's statement how in the academic realm we often remove participants from their context but rarely place them back in [7] . To this end, we need to take care in how we respect the various stakeholders and the contexts they live in.

Often empathy tools are one-off interventions. Accordingly, like with other one-off interventions, there is a lack of engagement with cultural systems in a deep enough way. To produce a lasting impact, there needs to be a culture around eliciting and fostering empathy. It is this systemic way of approaching empathy that leads us to consider, how 

The prevalent trends in XR media indicate potential for fostering empathy. On top of the technologies supporting virtual environments, we need to thoroughly examine the scope and effects of XR-driven empathy. A key challenge is to provide an experience that successfully elicits empathy and, if possible, inspires taking action. Shifting from recognizing the experiences of the Other, towards more action-driven outcomes serves to extend benefits beyond the simulation in the virtual environment. The articles in the present survey did not find meaningful differences in long-term empathy, and to this end, more work is needed to ascertain what factors could support empathy over a longer time period. Moreover, the methodologies deployed in the studies are not useful in comparing the effectiveness between different approaches. Further research into the various stakeholders in empathy tool design could serve to promote more sustainable approaches for eliciting empathy. To this end, constructing a framework for developing and measuring empathy tool experiences with XR technologies would support proper practices, and help create new ones.

Accordingly, the designers of empathy experiences have a great responsibility. For instance, a collaboration between social scientists, psychologists, anthropologists, architects, and designers is needed to develop a meaningful, responsible, and sustainable empathy.

Open Question 1: What methods support long-term empathy? More knowledge is needed on producing a lasting impact with empathy. As our findings show (see Figure 8 ), the long-term impact of empathy intervention was measured only rarely, and usage of the empathy tools was often limited to one-off instances. To support building long-term empathy, we need a more systematic approach to facilitating empathy, and the empathic reality approach described below is a step in that direction. How the empathy experiments can work over time is ripe for more longitudinal work.

Open Question 2: How to foster non-human empathy? Empathy is mostly focused on the experiences of fellow humans (see Figure 9 ). With the rising focus on Animal-Computer Interaction (ACI) in computing (see, e.g., [81] ) and more-thanhuman approaches [41] , we recommend a wider understanding of empathy for a more sustainable future. The direction in which we develop technology, and by extension our environments, should work with the multi-perspective lens of flora, fauna, and humans. In essence this approach encourages diversity through the cultivation of multiple perspectives, and helps to promote actions towards environmental challenges such as the climate change and sustainable behaviors [15] . As shown by the limited count of non-human empathy, we promote the challenge of facilitating empathy beyond our species. While the reciprocal nature of empathy is difficult to ascertain, we suggest the empathic position towards flora and fauna builds awareness of co-existence.

Spatiality is a beneficial lens for analyzing empathy. However, current research does not consider how the space affects empathy, and conversely, what possibilities empathy has for spatiality. To this end, we recognize two research opportunities.

First, more empirical knowledge is needed on how spaces modulate empathy. As XR technologies have a wide range of input modalities and immersive capabilities, generalizing spatial experience is difficult. However, we suggest that proposing technology-specific guidelines for developing with spatial experience in mind would lead to actionable results. Further, with empathic content that is available and accessible in the spectrum of virtuality, we can open up new application areas for research in terms of spaces. For instance, how could these traditionally one-off interventions be installed in a physical space in the city? Alternatively, what kind of empathy could be fostered in the privacy of one's home? As such, the space and its context play into building a more nuanced understanding of empathy. As empathy tools are focused on immersing the user, we could see them serve a similar purpose as movie theaters or libraries. Further, the historical or futuristic forms of empathy are applicable in the educational contexts of museums and schools. New applications for empathy are in line with trends such as the empathic city [11] that have future-oriented strategies that move from the technocratic focus of smart cities into a more human centric focus. To this end, contextual collapse allows us to recognize how the contexts of physical and virtual spaces are brought together. This leads to a great challenge in the research of empathy tools. How are these different contexts interpreted, and re-interpreted?

The designer of the empathy tool uses the experience of the Other, and reframes it in a technological tool, using the designer's understanding of the context. Then, the user of the tool experiences using their contextual frame, and hopefully experiences empathy in this process.

Second, we see potential in examining how spatial experience can be used to elicit empathy. A person's experience of space with regards to aspects like safety, accessibility, aesthetic experience, personal meanings, and social interactions can be fruitful triggers for empathy. This spatial empathy serves to form empathic relationships through the experience of space. As spatial experience is related to an individual's sensemaking, there is great potential for communicating and empathizing these personal experiences of space. We see a few relevant focus for research into spatial empathy.

Clarifying the relevant aspects of the spatial experience for eliciting spatial empathy highlights the potential for perspective-taking. Also, researching methods for spatial empathy would build understanding towards more effective empathy studies. For example, Heiss [56] developed a prototype named "Empathy Vest", which is an interactive vest that connects the user to the remote environment of someone else, through multiple sensors. We see this somatic approach of empathy as fruitful for eliciting spatial empathy, due to the holistic way we experience spaces, alongside cognitive and affective forms of empathy. Further, like the included articles in our survey, spatial empathy is not limited to contemporary experiences. Memory and data help to empathize the past, and speculations can focus on the future.

Besides the linear understanding of time, we can follow Lefebvre's method, by approaching space through its temporal cycles and rhythms [77] . What hourly, daily, monthly, seasonal, or yearly experiences of space are there? What rhythms are not linearly understandable or measurable? Most importantly, how can these nuanced aspects of time help to evoke spatial empathy? Spatiality and temporal experiences provide challenges as well as the potential for eliciting empathy.

Open Question 3: How can spatiality be better understood in XR?. The present understanding of spatial experience in everyday physical environments is not directly applicable to the wide range of extended reality technologies. The current research in VR immersion etc. is partially related to spatial experience, but more focused on the believability of the virtual experience, and what design aspects are most profitable. As such, a successful technical implementation leads to a more immersive experience, enabling further spatiality. Different virtual applications are still quite implementationspecific. Further, as different aspects of spatial experience, and especially perception, have been studied in virtual reality (e.g., scale and gravity [104] and distance [80] ) more information is needed on how this knowledge is applicable in real-world environments. In the range of XR, the level of virtuality challenges developing an all-encompassing theory of spatiality. To this end, we posit that exploration into different spatial experiences in XR is conducive to building more impactful and believable empathy tools.

Open Question 4. What tools can support interaction designers prototyping temporal and spatial empathy? Both temporal and spatial elements can significantly impact the XR experiences and the delivery of empathic spaces. Nevertheless, the required efforts for blending our spaces are high, primarily due to the readiness of XR prototyping tools. In brief, the XR user experience is not yet well studied by interaction designers, while the development time and the lead time of collecting feedback are long. As reflected in an interview of 26 AR/VR designers and developers [67] , the current processes of XR developments cannot effectively bring synergy between the technical AR/VR developers and content creators, while presenting limited communication channels to collect feedback from end-users. Researchers have proposed various prototyping tools to address the issue, such as SpatialProto [89] , but none of the existing works, to the best of our knowledge, serves prototyping XR in empathic contexts specifically. Remarkably, the knowledge of collaborative interactions (many-to-many) and user agency also requires further research (Sections 3 and 4), and we call for diversified tools to promote the understanding of users' empathy in XR, especially when designing interaction for empathic reality requires doubled efforts compared to usual XR applications, for the investigation of both users and the Other.

From the exhaustive list of studies considered in this paper (Appendix B), we see significant research efforts on pursuing empathy development in people through XR technologies. The collected works demonstrate empathic content going beyond personal, temporal, cultural, and geographical limitations. Empathic content can be presented through virtual environments that are characterized by plasticity, expansibility, and scalability. Virtual environments enable the users to understand the perspectives of others by stepping into one's shoes. As such, it results in enhanced empathy and a better understanding of others' sufferings. Users with virtual headsets can gain a sense of presence and realism [46] , in which a first-person view can bring the users an illusion of experiencing someone else's suffering. Still, such empathy can be limited to solely immersive worlds with VR headsets. We acknowledge that the rising availability of consumer-grade mobile VR headsets also gives a strong incentive to attract research attention on VR-driven empathy. Meanwhile, limited options and high prices of AR headsets have led to limited interests in facilitating empathy with virtual-physical blended environments (AR).

It is also worth mentioning that the current interest in the metaverse will accommodate multitudinous new content and numberless content consumers. As stated in [76] , the metaverse situates at the stage between Digital Twins and Digital Natives. The ultimate phase of the co-existence of physical objects and virtual entities projects that the metaverse eventually will become ubiquitous in our daily life through a highly blended virtual-physical reality. Though the current metaverse, defined by mostly technology giants (e.g., Meta and Microsoft), refers to sole virtual environments, we foresee that the metaverse will result in inclusive environments and simultaneously provide building blocks for building the metaverse towards empathic dimensions. With very limited knowledge of AR-driven empathy, the metaverse will hardly reach the ultimate phase. To this end, it is necessary to link the sole virtual environments with virtual-physical (blended) environments [72, 73] . As a prior study revealed that the combination of AR and VR can further engage the users [24] , more investigation should be done to achieve the potential of the empathic metaverse.

As the majority of studies only reveal a snapshot of enhanced empathy after the participants had experienced the empathic virtual environments, we cannot give a definitive conclusion on what will be the exact role(s) and limitations of such empathic virtual environments in the metaverse. However, our observations among the collected articles give obvious cues that the virtual environments have strong efficacy in cultivating care towards others, e.g., the environment [94] . Albeit we do not have a piece of solid evidence showing that virtual environments give long-term empathy to the users, immersive technologies could support the very first step of building up a harmonic and mutual-understanding society, through inclusive-or empathy education. When metaverse technologies can support such scenarios blended into our physical world, our immersive urban environments should own a capacity of reflecting multiple, and perhaps conflicting, interests of various stakeholders, as well as their difficulties in a shared and open community [75] .

Open Question 5. What possibilities exist for planet-scale virtual-physical blended environments for empathic reality? The empathic experiences in immersive virtual environments described in this paper have been constrained by the state of immersive technology. As such, these experiences are confined to small-scale virtual and physical spaces, with a single user at the core of the experience. Besides, many of these experiences focus on fully virtual worlds, with little physicality. Recent advances in extended reality technology are enabling many larger-scale experiences, whether in terms of physical space, digital space, or several concurrent users. Planet-scale virtual and mixed environments are under concrete considerations through proposals such as the metaverse. Such a shift in technology is likely to affect the design of empathic experiences. Enabling planet-scale (digital or mixed physical-digital) experiences shared between a multitude of users enables greater flexibility in the environments and actors, multiplying the possibilities for empathy development.

Open Question 6. What are the vital roles of autonomous agents when 'empathic' reality goes wrong? Besides extended reality, advances in artificial intelligence enable the creation of digital agents that can dynamically adapt to the user's behavior, to guide the experience and ensure accomplishing the primary goals. We foresee the future of empathic experiences to be more natural and open-ended, relying on the interaction between multiple simultaneous users and digital agents. However, open-ended experiences raise the question of accomplishing the primary empathy goals. An empathic experience where most actors are players may rapidly turn into a game where users collaborate towards turning an unpleasant experience into entertainment. For instance, a simulation of war conditions may turn into a collaborative survival experience which would negate certain of the intended aspects such as distress or loneliness.

At the other end of the spectrum, it is necessary to address phenomenons such as cyber-bullying, which may further distress users placed into uncomfortable experiences. A delicate balance will have to be achieved, where the simulation adapts to users' behaviors to avoid unforeseen actions, and perhaps autonomous agents characterized by stewardship and trustworthiness can audit varying empathic experiences in many virtual worlds in the metaverse era.

This article surveys the area of empathy-driven immersive extended reality (XR) technologies, analyzing different ways by which virtual environments can simulate and trigger empathy. We especially focus on the spatiality and the contexts that these empathy tools take place in. Consequently, we bring an understanding of how users can enhance their awareness and empathy towards others' experiences. Our findings suggest that the current trends of empathy tool implementations have a limited impact on users, and more research is needed to establish good practices for eliciting long-term empathy. We discuss the current opportunities in the research conducted to date on empathic virtual environments, coined as Empathic Reality (ER). Under the domain across XR technology, empathy, and spatiality, our survey article ends by presenting a research roadmap and several open research questions in the wide scope of digital media and the metaverse.

Effectiveness of Using an Immersive and Interactive Virtual Reality Learning Environment to Empower Students in Strengthening Empathy and Mastery Learning

2016. myShoes -the Future of Experiential Dementia Training?

Experiencing Nature: Embodying Animals in Immersive Virtual Environments Increases Inclusion of Nature in Self and Involvement With Nature

Introduction to Human-Building Interaction (HBI): Interfacing HCI with Architecture and Urban Design

Gotta Catch 'em All: Exploring the Use of Pokémon Go to Enhance Cognition and Affect

Empathy Tool Design-Eye Disease Simulator Based on Mixed-Reality Technology

Small Arcs of Larger Circles: Framing Through Other Patterns

Exploring Button Designs for Mid-Air Interaction in Virtual Reality: A Hexa-Metric Evaluation of Key Representations and Multi-Modal Cues

PARA: Privacy Management and Control in Emerging IoT Ecosystems Using Augmented Reality

Ultimate Empathy Machine": On the Composition of Virtual Reality Nonfiction Experiences

From smart to empathic cities

Empathy machines

Empathy Toward Virtual Humans Depicting a Known or Unknown Person Expressing Pain

Empathy, place and identity interactions for sustainability

Virtual Reality Clinical-Experimental Tests of Compassion Treatment Techniques to Reduce Paranoia

Virtual Reality as a Tool for Eliciting Empathetic Behaviour in Carers: An Integrative Review

Empathy Machine": How Virtual Reality Affects Human Rights Attitudes

Impact of Immersing University and High School Students in Educational Linear Narratives Using Virtual Reality Technology

Increasing Awareness, Sensitivity, and Empathy for Alzheimer's Dementia Patients Using Simulation

Toward Measuring Empathy in Virtual Reality

A Virtual Reality Simulation of Drug Users' Everyday Life: The Effect of Supported Sensorimotor Contingencies on Empathy

Living Room Conservation: A Virtual Way to Engage Participants in Insect Conservation

Transfer of training-Virtual reality training with augmented multisensory cues improves user experience during training and task performance in the real world

Design of Visual Deficit Simulation for Integration into a Geriatric Physical Diagnosis Course

A Design Space Exploration of Worlds in Miniature

Bad Breakdowns, Useful Seams, and Face Slapping: Analysis of VR Fails on YouTube

Context collapse: theorizing context collusions and collisions. Information

Do Medical Students Respond Empathetically to a Virtual Patient?

From game design elements to gamefulness: defining "gamification

3D Virtual Worlds and the Metaverse: Current Status and Future Possibilities

What we talk about when we talk about context

Re-space-ing place: "place" and "space" ten years on

Cyberball3D+: A 3D Serious Game for fMRI Investigating Social Exclusion and Empathy

An Inquiry-Based Augmented Reality Mobile Learning Approach to Fostering Primary School Students' Historical Reasoning in Non-Formal Settings

Empathy and its development

Assessing Empathy

A Pilot Study of the Impact of Virtually Embodying a Patient with a Terminal Illness

Rolf Nordahl, and Stefania Serafin

Using Virtual Reality to Facilitate Empathy Between Hearing Impaired Children and Their Parents

Physical and Social Presence in Collaborative Virtual Environments: Exploring Age and Gender Differences with Respect to Empathy

Towards a More-than-Human Approach to Smart and Sustainable Urban Development: Designing for Multispecies Justice

An Immersive Approach to Visualizing Perceptual Disturbances

Motion, Emotion and Empathy in Esthetic Experience

What Can Virtual Reality Teach Us about Prosocial Tendencies in Real and Virtual Environments?

The Use of Virtual Reality Alone Does Not Promote Training Performance (but Sense of Presence Does)

Exploring Virtual Reality Experiences of Slum Tourism. TOURISM GEOGRAPHIES

2020. E-Mpathy and the Phantom Limb Sensation: A Multisensory Experience for Embodiment of Amputation

Feel Autism VR -Adding Tactile Feedback to a VR Experience

Conceptual Knowledge and Sensitization on Asperger's Syndrome Based on the Constructivist Approach through Virtual Reality

The hidden dimension: man's use of space in public and private

Reducing Risk and Improving Maternal Perspective-Taking and Empathy Using Virtual Embodiment

Re-place-ing space: The roles of place and space in collaborative systems

Being and time

Building Dwelling Thinking

Empathy and the Space Between

Stop Catcalling -A Virtual Environment Educating Against Street Harassment

Building Long-Term Empathy: A Large-Scale Comparison of Traditional and Virtual Reality Perspective-Taking

Effects of Emotion and Agency on Presence in Virtual Reality

Using Virtual Reality and 360-Degree Video in the Religious Studies Classroom: An Experiment

The Virtual Doppelganger: Effects of a Virtual Reality Simulator on Perceptions of Schizophrenia

Our Dog Probably Thinks Christmas Is Really Boring": Re-Mediating Science Education for Feminist-Inspired Inquiry

Who Are Virtual Reality Headset Owners? A Survey and Comparison of Headset Owners and Non-Owners

Do architects and designers think about interactivity differently?

Understanding Empathy Training with Virtual Patients

On the Design of an Empathy-Arousing Mixed-Reality Game

Current Practices, Challenges, and Design Implications for Collaborative AR/VR Application Development

Beyond Human: Animals as an Escape from Stereotype Avatars in Virtual Reality Games

Sensitizing concepts for socio-spatial literacy in HCI

Alzheimer's Eyes Challenge: The Gamification of Empathy Machines

A VR Game to Teach Underwater Sustainability While Diving

M2A: A Framework for Visualizing Information from Mobile Web to Mobile Augmented Reality

A2W: Context-Aware Recommendation System for Mobile Augmented Reality Web Browser

Who Wants to Be a Self-Driving Car? A Mixed-Reality Data-Trust Exercise

Towards Augmented Reality Driven Human-City Interaction: Current Research on Mobile Headsets and Future Challenges

All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda

Rhythmanalysis : Space, Time and Everyday Life

Space and places: When interacting with and in physical space becomes a meaningful experience

Promoting Sustainability through Virtual Reality: A Case Study of Climate Change Understanding with College Students

Spatial Orientation Shrinks and Expands Psychological Distance

Animal-computer interaction: a manifesto

Through the Eyes of a Bystander: Understanding VR and Video Effectiveness on Bystander Empathy, Presence, Behavior, and Attitude in Bullying Situations

Virtual Reality, Augmented Reality, Mixed Reality, and the Marine Conservation Movement

Effectiveness of a Poverty Simulation in Second Life®: Changing Nursing Student Attitudes toward Poor People

A taxonomy of mixed reality visual displays

How virtual reality can create the ultimate empathy machine

A survey of diminished reality: Techniques for visually concealing, eliminating, and seeing through real objects

Virtual reality and the CAVE: Taxonomy, interaction challenges and research directions

SpatialProto: Exploring Real-World Motion Captures for Rapid Prototyping of Interactive Mixed Reality

Communicating Sustainable Consumption and Production in 360°Video

Collected Short Fiction. Giramondo

Feeling Good about Feeling Bad: Virtuous Virtual Reality and the Automation of Racial Empathy

Augmenting Museum Communication Services to Create Young Audiences

Virtual Reality as a Tool for Environmental Conservation and Fundraising

HandMorph: A Passive Exoskeleton That Miniaturizes Grasp

Virtually Old: Embodied Perspective Taking and the Reduction of Ageism under Threat

Degraded Reality: Using VR/AR to Simulate Visual Impairments

Investigating Human Scale Spatial Experience

The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

Empathy in Virtual Agents and Robots: A Survey

Neuroanatomical Basis of Concern-Based Altruism in Virtual Environment

In Tension with Progression: Grasping the Frictional Tendencies of Speculative, Critical, and other Alternative Designs

Empathic Mixed Reality: Sharing What You Feel and Interacting with What You See

The Plausibility Paradox for Resized Users in Virtual Environments

Evoking Empathy: A Framework for Describing Empathy Tools

Using 360-Video Virtual Reality to Influence Caregiver Emotions and Behaviors for Childhood Literacy

The Cognitive, Affective, and Somatic Empathy Scales (CASES) for Children

Initial Evaluation of Accessibility and Design Awareness with 3-D Immersive Environments

Virtual Superheroes: Using Superpowers in Virtual Reality to Encourage Prosocial Behavior

Simulating Illness: Experiencing Visual Migraine Impairments in Virtual Reality

Bio-Adaptive Social VR to Evoke Affective Interdependence: DYNECOM

Towards the Design of a Virtual Heritage Experience Based on the World-as-Support Interaction Paradigm

Pictures of pain: their contribution to the neuroscience of empathy

Empathy and Embodied Experience in Virtual Environment: To What Extent Can Virtual Reality Stimulate Empathy and Embodied Experience?

Foams : plural spherology. Semiotext(e)

What is Mixed Reality?

Designing a Virtual Environment for Teacher Training: Enhancing Presence and Empathy

To Be There When It Happened: Immersive Journalism

Simulating the Cuban Missile Crisis: Crossing Time and Space in Virtual Reality

Visuospatial Transformations and Personality: Evidence of a Relationship between Visuospatial Perspective Taking and Self-Reported Emotional Empathy

Using Augmented Reality and Virtual Environments in Historic Places to Scaffold Historical Empathy

Embodying Senescence Performing Agedness in the Space of Virtuality

Can Virtual Reality Simulations Be Used as a Research Tool to Study Empathy, Problems Solving and Perspective Taking of Educators? Theory, Method and Application

Transurbanism: Smart Cities for Transhumans

Inclusion of Third-Person Perspective in CAVE-like Immersive 3D Virtual Reality Role-Playing Games for Social Reciprocity Training of Children with an

Walking in Small Shoes: Investigating the Power of VR on Empathising with Children's Difficulties

HAVE Experience: An Investigation into VR Empathy for Panic Disorder

Virtual Reality Perspective-Taking Increases Cognitive Empathy for Specific Others

How Does It Feel to Be a Woman Victim of Sexual Harassment? The Effect of 360 Degrees-Video-Based Virtual Reality on Empathy and Related Variables

Quantifying Empathy in Virtual Reality: An Outline

Educating Bicycle Safety and Fostering Empathy for Cyclists with an Affordable and Game-Based VR App

The Effects of a VR Intervention on Career Interest, Empathy, Communication Skills, and Learning with Second-Year Medical Students

Empathizing Audiovisual Sense Impairments: Interactive Real-Time Illustration of Diminished Sense Perception

Foams of togetherness in the digital age: Sloterdijk, software sorting and Foursquare

Empathy and Experience in HCI

Parkinson's Disease Simulation in Virtual Reality for Empathy Training in Medical Education

OR "sympathy" OR "pity" OR "compassion") AND ("urban" OR "space" OR "place" OR "built environment" OR "building" OR "city" OR "environment") AND ("virtual" OR "mixed" OR "augmented" OR "diminished") AND "reality") OR Keyword:(("empathy" OR "sympathy" OR "pity" OR "compassion") AND ("urban" OR "space" OR "place" OR

OR "sympathy" OR "pity" OR "compassion") AND ("urban" OR "public space" OR "social space" OR

empathy" OR "sympathy") AND ("urban" OR "place" OR "city" OR "environment") AND ("virtual" OR "augmented") AND "reality