key: cord-0565009-hlz898m9 authors: Constantinides, Marios; Quercia, Daniele title: The Future of Hybrid Meetings date: 2022-05-10 journal: nan DOI: nan sha: 53bd76588f9a4c3b95cc6487f43a644077750c92 doc_id: 565009 cord_uid: hlz898m9 Meetings are typically considered to be the fuel of an organization's productivity -- a place where employees discuss ideas and make collective decisions. However, it is no secret that meetings are also often perceived as wasteful vacuums, depleting employee morale and productivity, likely due to the fact that current technologies fall short in fully supporting physical or virtual meeting experience. In this position paper, we discuss the three key elements that make a meeting successful (i.e., execution, psychological safety, and physical comfort), and present new tools for hybrid meetings that incorporate those elements. As past research has focused on supporting meeting execution (the first element), we set the roadmap for future research on the two other elements: on psychological safety by articulating how new technologies could make meeting useful for all participants, ensure all participants give and receive appropriate levels of attention, and enable all participants to feel and make others feel comfortable; and on physical comfort by dwelling on how new technologies could make the meeting experience comfortable by integrating all human senses. We also discuss the potential danger of these technologies inadvertently becoming surveillance tools. . Three key elements that make a meeting successful along with key research questions for future research: (a) Execution: whether the meeting had a clear purpose, structure, and resulted in actionable items; (b) Psychological safety: whether participants felt listened to and were motivated to contribute during the meeting; and (c) Physical comfort: whether participants felt physically comfortable by, say, being in an environment with good air quality and sufficient natural light. Most innovations in meeting technologies have focused on execution, and future work should focus on the two other aspects -psychological safety and physical comfort. A sample of key research questions are reported in this infographic. A meeting's perceived experience has been evaluated through standardized questionnaires, or ad-hoc post meeting surveys. For example, the Event Performance Indices (EPI) [22] is a six-item questionnaire that focuses on a meeting's overall performance results, and, in turn, measures meeting effectiveness. The questionnaire prompts participants' satisfaction levels including, for example, whether the meeting was worth the time investment, and whether participants were personally motivated. In the fields of Management and Organizational Science, meeting productivity has been directly linked with a meeting's agenda, structure, and purpose [15, 26, 40] . Inclusiveness [6, 7] , dominance [36] , peripheral activities [29] , and psychological safety [8, 20] are aspects that have also been found to influence meeting experience and productivity. The physical comfort of the workplace environment is an additional factor that makes up the list [2] . For example, a recent survey found that light and outdoor views were among the most popular perks employees craved for [28] , whereas stuffy and stale air offices tended to reduce productivity [3, 4, 41] . More recently, to capture the factors that generally make a meeting successful, Constantinides et al. [17] designed a 28-question survey and administered it to 363 individuals whose answers were then statistically analyzed. The survey covered an array of themes, previously identified in the Organization and Management Science literature, including a meeting's psychological experience (e.g., contribution, balance, turn-taking, attention), its structure, and its content (e.g., agenda, physical environment, use of technology). The survey responses were then statistically analyzed using a Principal Component Analysis, with 11 items of the 28-question survey explaining 62% of the total variance. These 11 items loaded on three factors -the three factors that make a meeting successful: Execution. Whether participants ultimately "got things done"-more specifically, whether the meeting had a clear purpose, structure, and resulted in actionable items. Psychological Safety. Whether participants felt listened to and were motivated to contribute during the meeting. Physical Comfort. Whether participants felt physically comfortable by, for example, be in a meeting room with good air quality and sufficient natural light. Next, we discuss recent research aiming at supporting these three key elements in either physical or virtual meetings. Execution refers to whether the meeting had a clear purpose, structure, and resulted in actionable items. Besides the mainstream communication tools of the likes of MS Teams, WebEx, Skype, Zoom, just to name a few, a large body of scientific work has been focused on supporting meeting execution through contextual information, text, audio, and video support. For example, one work focused on detecting key decisions in dialogues [23] , while another on generating an "action items" list from these dialogues [27] . Using an agenda planning technique, Garcia et al. [19] developed a tool that allows meeting participants to vote for agendas items. As the balance of conversational turn-taking is important for group performance [46] , technologies were developed to create awareness. This was done by highlighting salient moments and visualizing participants' contributions [24] . Content recording was also a focus for technologies supporting execution. NoteLook [12] exploited video streams to support note taking. Catchup [42] automatically identified the gist of what was missed in a meeting, allowing people to join the meeting late and still participate effectively. Video Threads [11] supported asynchronous video sharing for geographically distributed teams. Finally, Banerjee et al. 's playback system allowed for revisiting a recorded meeting [10] . Psychological safety refers to whether participants felt listened to and were motivated to contribute during the meeting. As Edmondson described it, psychological safety refers to "the absence of interpersonal fear that allows people to speak up with work-relevant content" [18] . In face-to-face interactions (e.g., during a social encounter or a meeting), we primarily read faces through visual cues and read bodies through a multi-sensory integration. All these cues, to a great extent, allow us to understand the dynamics in a conversation (e.g., whether one feels listened to and receives the appropriate attention from his/her peers). In virtual interactions, however, these natural cues are often lost, flattening the psychological and social experience. Furthermore, current meeting technologies need to deliver this experience in an attention-enhancing way as opposed to the current attention-depleting way: currently, in a virtual meeting, attendees need to pay attention not only to what is being said, but also to peripheral modes of communication (e.g., text messages, raising of virtual hands). To bring the offline meeting experience into the virtual world, recent research developed an app called MeetCues. This captures three types of cues, aiming at augmenting psychological safety in hybrid meetings. First, the MeetCues app allows meeting attendees to provide real-time feedback [5] . The app enabled attendees to react on what was being discussed by tagging key points and action items. These virtual crowdsourced tags were translated into an emoji cloud visualization, which allowed attendees to infer the overall atmosphere. In this way, these crowdsourced cues served as indicators of moments during which, for example, participants agreed with what was being said or whether further clarification was needed, cultivating a safe environment for contributions. However, these crowdsourced feedbacks did not fully capture the multi-sensory integration that we are attuned to in physical meetings. A case in point is the almost universally acceptable non-verbal cue of nodding [31] . In face-to-face interactions, it is natural to observe bodily expressions to understand, for example, whether one agrees on what is being said (nodding), or a clarification is needed. However, the picture is different in virtual meetings. Participants of virtual meetings who often turn off their cameras, leave the speaker staring at a sea of black squares, feeling psychologically disoriented. That is why the second type of cues MeetCues captured was body movements. It did so by integrating wearable devices with the MeetCues app, which captured participants' heart rates, head and hand movements, and changes in postures [13, 30] . These body cues were predictive of the meeting's vibrancy and multi-tasking activities, and ultimately of the meeting's success; these cues were even more predictive than the meeting's emotional content derived from the meeting's transcript. For example, head movements such as nodding served as a proxy for (dis)agreement, while changes in postures served as a proxy for (dis)comfort. In particular, these two body cues metrics proved useful in helping attendees infer the levels of psychological safety "in the room". The third type of cues MeetCues captured was types of conversations [14] (e.g., a heated discussion resulting in a conflict eventually being resolved, a supportive conversation, an exchange of knowledge). By analyzing more than four thousand minutes of conversations from eighty-five real-world meetings, Zhou et al. [51] found that conversation types were more predictive of meeting success than traditional voice and text analytics. By monitoring these conversations during a meeting (or after it), one could potentially measure specific aspects of organizational productivity, and proactively take actions for improvement. To see how, consider conflict resolution. Having new real-time ways of marking indicators of, for example, conflicts, not only would increase attendees' awareness but would also help cultivate an approach of conflict resolution. Physical comfort refers to whether participants felt physically comfortable by, for example, be in a meeting room with good air quality and sufficient natural light. To see what we mean by the word 'comfort', let us draw a parallel with architecture. In that area, comfort mainly describes four main qualities that a good building must possess: visually pleasurable, thermally livable, acoustically tolerant, and respiratory humane (in terms of air quality) [2] . To paraphrase comfort in the meeting context, recent research has initially focused on the specific aspects of good air quality and natural light. A team of researchers, in particular, developed miniaturized devices, called Geckos, which are fitted with cheap-to-produce sensors capturing light, temperature, and the presence of a broad range of gases such as volatile organic compounds (VOCs). They integrated Geckos with the MeetCues app [5] , and created a new indoor environmental sensing infrastructure ComFeel [17] . At the end of each meeting, the MeetCues app asks each participant: (a) whether the meeting had a clear purpose and structure, and resulted in a list of actionable points, and (b) whether each participant felt listened to and was motivated to contribute. To explore the extent to which air quality alone determined whether a meeting was perceived to be productive or not, they deployed ComFeel in a corporate office and gathered data from 29 meetings in different rooms. As one expects, productive meetings were those in which participants felt safe to contribute: the probability of a meeting being productive increased by 35% for each standard deviation increase in psychological safety. Surprisingly, the results for air quality were dramatic too. The productivity probability increased by as much as 25% for each standard deviation increase in room pleasantness. Furthermore, among all the sensors, the air quality one was the most important: indeed, room pleasantness was achieved through improved temperature (with a relative contribution of 25%), lighting (30%), and air quality (45%). These results suggest that, if a meeting takes place in a stuffy conference room, even if it is run well, people will still struggle to pay attention. To fix that, one just needs to do a handful of things, from manually or automatically adjusting ventilation, lighting, temperature, which could increase a meeting's productivity by a considerable extent. Most innovations in meeting technologies have focused on supporting physical or virtual meetings independently, and they did so by extensively focusing on how to support meeting execution. That is why future work should focus on the two other overlooked aspects (Figure 1 ): on psychological safety (Section §4.1) by making meetings useful for all participants, ensuring all participants give and receive appropriate levels of attention, and enabling all participants to feel and make others feel comfortable; and on physical comfort (Section §4.2) by integrating all human senses, and, as such, making the meeting sensory experience pleasurable. At the same time though, future work should also consider how these new technologies could inadvertently become surveillance tools (Section §5). The design space of meeting technologies could be well enriched with features that fully support all three facets of psychological safety [17] : (a) making the meeting useful and satisfying for all participants, (b) ensuring all participants give and receive appropriate levels of attention, and (c) enabling all participants to feel and make others feel comfortable. First, to make a meeting useful and satisfying, existing tools already provide analytics in the form of summaries or real-time feedback [5] . However, these analytics could be enriched with aspects that are hard to quantify such as the types of conversations happening in a meeting [51] , nuanced emotional states [52] , or other language markers that are hidden in conversations (e.g., presence of stress, engagement in emphatic conversations) [35, 39, 50] . Second, as turn-taking and balance in conversation helps to cultivate a safe environment for contribution [46] , future meeting tools need to ensure that all attendees give and receive the appropriate levels of attention. For example, some meeting technologies already do provide indicators about "speaking up" time, and often do so through attentiondepleting notifications. However, future technologies must ensure attention-enhancing ways of notifying users. As Weiser and Brown argued in their seminar work The Coming Age of Calm Technology, "the information display should move easily from the periphery of users' attention to the centre, and back" [44] . Future design elements need to smoothly capture the user's attention only when necessary, while calmly remain in the user's periphery most of the time. For example, while current meeting tools allow various modes of communication (e.g., voice, text) and notifications (e.g., emojis) in a virtual meeting, personalized or timely delivery of those notifications could remove unnecessary information overload. Imagine a scenario during which five attendees need to focus on the speaker, yet one of them types a question and another raises his/her virtual hand. If these notifications were to be delivered straight away, they may well overload the speaker and distract the audience. More work needs to go into identifying the right moment for delivering a notification, or that for delivering a notification to the best person in a group, allowing for personalized delivery. Third, it is also important for meeting technologies to facilitate an inclusive environment where all attendees feel comfortable, and can make others feel comfortable. One way of achieving it is to create social interactions that scaffold learning, and then use computational methods to weave these interactions into the fabric of meeting tools. For example, as demonstrated in massive online classes (MOOCs) [33] , future meeting technologies could coach people skills that foster inclusion and diversity, and collaboratively help them reflect on their behavior. Another way of achieving inclusivity is through broadening the design space. Design elements such as emojis allow, to a great extent, attendees to non-verbally communicate and express their emotional state; it can be seen as a way of developing trust or empathy for others. Future meeting technologies could also borrow concepts from biophilic design [45] and embrace new types of visual cues [32] (e.g., the use of different symbols, imagery, and artificial artifacts), thus making additional layers of non-verbal communication available to users. Aligned with the Human-Building Interaction vision [1] (an emerging area aiming at unifying HCI research in the built environment), physical comfort goes well beyond air quality, integrating all the human senses. To begin with, let's take sight. Could we make the physical environment visually pleasurable? Recent studies showed that adjusting light conditions in a room could serve as a stress reducing intervention mechanism, or even as a biofeedback relaxation training [34, 47] . Moving to hearing. Could we design ubiquitous technologies that facilitate meetings anywhere and everywhere? Consider a knowledge worker who attends meetings from a variety of places: from the office to home to a cafeteria to even an autonomous car [25] . How could future technologies support smooth transition between different places, and provide a pleasurable sound experience? Smart earable devices can be one of the answers wherein an array of sensors could be embedded in them to control not only what sound the end user is hearing, but also how that sound is delivered in a specific physical setting [21] . While addressing sight and hearing is to some extent easier, smell and touch require future research efforts. Which odour is more relaxing and energizing during a meeting, and, to what extent, certain odour could be adapted as a meeting unfolds to facilitate better execution? How could we stimulate the sense of touch to increase physical comfort during a meeting? For example, as most of white collar jobs require employees to spend long hours sitting, previous studies explored the design of ergonomic chairs that would increase physical comfort levels. While these studies typically employ various objective methods to capture physical comfort such as pressure sensors measurement [48] or an array of sensors including, for example, temperature, gas, illuminance, VOC, CO 2 [49] , new sensing modalities such as Electromyography (EMG) could prove useful to fully incorporate the sense of touch in the design space of these chairs. While these technologies hold the promise of enabling employees to be productive, report after report has highlighted the outcries of workplace AI-based solutions being biased and unfair, and lacking transparency and accountability. During the COVID-19 pandemic, systems were being used to analyze footage from security cameras in workplace to detect when employees are not complying with social distancing rules 1 ; while there is a handful of good intentions behind such a technology, the very same technology could be used for tracking employees' movements, or time away from desk. As we move towards a future likely ruled by big data and powerful AI algorithms, important questions arise relating to the psychological impacts of surveillance, data governance, leadership and organizational culture, and compliance with ethical and moral concerns. The historian and philosopher Yuval Noah Harari argued that digital platforms such as those that allow us to work from remote need to follow three basic rules 2 to protect us from "digital dictatorships". First, any data collection on people should be used to help people rather than to manipulate, control, or harm them. In the meetings context, this translates into providing analytics that help employees reflect on their experience rather than causing them to receive low performance review in the event of a meeting not being executed well. Second, surveillance must always go both ways. That is, whenever an organization increases surveillance of individuals, at the same time, organizational accountability needs to increase. If organizations could establish processes to monitor their workforce, they may well establish processes to audit their own actions as well. Third, data should not be concentrated in a single entity, not least because data monopolies are the recipe for dictatorship. In the workplace context, this translates into newly created divisions in a company that oversee data collection and, ideally, ensure that data about their workers is in the hands of the workers themselves. In a global modernized workplace, new meeting tools are likely to be developed, facilitating and augmenting the experience of hybrid meetings. As a result of the COVID-19 pandemic, fully remote or hybrid meetings removed any physical barriers and often contributed to improved work-life balance [37] . On the downside, tools for supporting hybrid meetings may inadvertently becoming surveillance tools, compromising employees' privacy [16] . To unpack the AI ethics of workplace technologies, Constantinides and Quercia [16] conducted a crowdsourcing study to understand how employees judge such technologies and determine which ones are desirable, and why. They considered 16 workplace technologies that track productivity based on diverse inputs (e.g., tracking audio conversation during virtual meetings, tracking text messages in collaboration tools), and asked crowd-workers to judge these scenarios along five moral dimensions. They found that workplace technologies were judged harshly depending on three aspects (heuristics) participants used to assess the scenarios. In increasing importance, these aspects reflected whether a scenario: 1) was not currently supported by existing technologies (hard to adopt); 2) interfered with current ways of working (intrusive); and, more importantly, 3) was not fit for tracking productivity or infringed on individual rights (harmful). Tracking eye movements in virtual meetings and the visited websites in remote work, despite being possible, were considered to be "on the way" of getting the job done (they were easy to adopt but intrusive). By contrast, tracking text messages in collaboration tools such as Slack was considered to not interfere with work (unobtrusive). Finally, tracking audio conversations in virtual meetings was considered to be possible (easy to adopt), and not interfere with work (unobtrusive), yet it was considered to be harmful, as it entailed tracking not only whether a meeting took place but also its content, causing a loss of control. The above heuristics offer a guide on how workplace technologies are likely to be morally judged. Having a technology that is easy to implement and does not interfere with work is not necessarily a technology that should be deployed. Tracking facial expressions (even beyond the nefarious uses -of dubious effectiveness -of inferring political orientation or sexual preferences [43] ) is possible and could be done in seamless ways (e.g., with existing off-the-shelf cameras), yet it would be still considered harmful and unethical. Tracking eye movements, task completion, or typing behavior was considered a proxy for focus (harmless) yet intrusive as it would "get in the way". Tracking social media use in remote work was considered not only intrusive but also harmful, as it infringes on privacy rights. On a very pragmatic level, there is a handful of reasons as to why organizations opt for employee surveillance (e.g., maintaining productivity, monitoring resources used, protecting the organization from legal liabilities). Critics, however, rightly argue that there is a fine line between what organizations could be monitoring and what they should be monitoring. If this line is crossed, it will have consequences on employees, affecting their well-being, work culture, and productivity [9] . If future meeting tools incorporate any kind of employee monitoring, they need to preserve individual rights, including that of privacy. New meetings tools need to also ensure non-discrimination, while promoting inclusivity, and, at the same time, provide explainable and understandable outputs (e.g., the decision upon which the system decided that a meeting was successful or not). Hybrid meetings also create asymmetries of interactions stemming from the social and cultural contexts. As Saatci et al. [38] found, the most challenging asymmetry is the diverse experience between co-located and remote meeting participants. Remote participants often feel isolated, while co-located participants dominate the interaction. Differences in language and accent, cultural behaviors, digital literacy, physical location, and the boundaries between work and personal life [37] also contribute to interaction asymmetries. To overcome such challenges, new meetings tools should focus on making meetings more inclusive for everyone by maximizing psychological safety and optimizing physical comfort. We thank those who actively supported this research at Nokia Bell Labs; in particular, Sagar Joglekar, Bon Adriel Aseniero, and Jun-Ho Choi for their active role in the development of the meeting companion app MeetCues; and Mark Introduction to Human-Building Interaction (HBI) Interfacing HCI with Architecture and Urban Design Comfort: A Coordinate of User Experience in Interactive Built Environments Stale Office Air Is Making You Less Productive Associations of Cognitive Function Scores with Carbon Dioxide, Ventilation, and Volatile Organic Compound Exposures in Office Workers: A Controlled Exposure Study of Green and Conventional Office Environments MeetCues: Supporting Online Meetings Experience 6 Reasons to Get Better at Leading Meetings How to Design Meetings Your Team Will Want to Attend How to Respond When You're Put on the Spot in a Meeting Workplace Surveillance: An Overview The Necessity of a Meeting Recording and Playback System, and the Benefit of Topic-level Annotations to Meeting Browsing Video Threads: Asynchronous Video Sharing for Temporally Distributed Teams NoteLook: Taking Notes in Meetings with Digital Video and Ink KAIROS: Talking Heads and Moving Bodies for Successful Meetings Ten Social Dimensions of Conversations and Relationships Use Subtle Cues to Encourage Better Meetings Good Intentions, Bad Inventions: How Employees Judge Pervasive Technologies in the Workplace. arXiv ComFeel: Productivity is a Matter of the Senses Too Psychological Safety and Learning Behavior in Work Teams Cutting to the Chase: Improving Meeting Effectiveness by Focusing on the Agenda How to Save a Meeting That's Gotten Tense Earables for Personal-scale Behavior Analytics The 6 Post-Event Survey Questions That Will Reveal Your Meeting's Effectiveness Learning About Meetings Meeting Mediator: Enhancing Group Collaboration Using Sociometric Feedback The Future of Work and Play: From Automated Vehicles to Working from Home Leading Great Meetings: How to Structure Yours for Success. Meeting for Results More to Meetings: Challenges in Using Speech-based Technology to Support Meetings The #1 Office Perk? Natural Light Recurring Meetings: An Experiential Account of Repeating Meetings in a Large Organization WellBeat: A Framework for Tracking Daily Well-Being Using Smartwatches Nonverbal Communication in Business Negotiations and Business Meetings HeartBees: Visualizing Crowd Affects Mentor Academy: Engaging Global Learners in the Creation of Data Science Problems for MOOCs LightSit: An Unobtrusive Health-promoting System for Relaxation and Fitness Microbreaks at The Language of Dialogue is Complex Meeting Analysis: Findings from Research and Practice Eworklife: Developing Effective Strategies for Remote Working During the COVID-19 Pandemic Hybrid meetings in the modern workplace: stories of success and failure Extracting Medical Entities From Social Media 8 Ground Rules for Great Meetings Air Pollution Is Making Office Workers Less Productive Catchup: A Useful Application of Time-travel in Meetings Deep neural networks are more accurate than humans at detecting sexual orientation from facial images The Coming Age of Calm Technology Biophilia Evidence for a Collective Intelligence Factor in the Performance of Human Groups DeLight: Biofeedback Through Ambient Light for Stress Intervention and Relaxation Assistance Are Pressure Measurements Effective in the Assessment of Office Chair Comfort/Discomfort? A Review Hilo-wear: Exploring Wearable Interaction with Indoor Air Quality Forecast The Language of Situational Empathy The Role of Different Types of Conversations for Meeting Success Predicting Meeting Success With Nuanced Emotions Clougherty, Sean Kennedy, Michael Eggleston, and Marcus Weldon for their guidance during the development. We also thank Nigel Oseland for sharing his research at the intersection of environmental psychology and workplace strategy.