key: cord-0179349-fs7k8vte
authors: Bodala, Indu P.; Churamani, Nikhil; Gunes, Hatice
title: Creating a Robot Coach for Mindfulness and Wellbeing: A Longitudinal Study
date: 2020-06-09
journal: nan
DOI: nan
sha: b72e8dd99e35453f329f2514967bb739b8a555c3
doc_id: 179349
cord_uid: fs7k8vte

Social robots are starting to become incorporated into daily lives by assisting in the promotion of physical and mental wellbeing. This paper investigates the use of social robots for delivering mindfulness sessions. We created a teleoperated robotic platform that enables an experienced human coach to conduct the sessions in a virtual manner by replicating upper body and head pose in real time. The coach is also able to view the world from the robot's perspective and make a conversation with participants by talking and listening through the robot. We studied how participants interacted with a teleoperated robot mindfulness coach over a period of 5 weeks and compared with the interactions another group of participants had with a human coach. The mindfulness sessions delivered by both types of coaching invoked positive responses from the participants for all the sessions. We found that the participants rated the interactions with human coach consistently high in all aspects. However, there is a longitudinal change in the ratings for the interaction with the teleoperated robot for the aspects of motion and conversation. We also found that the participants' personality traits -- conscientiousness and neuroticism influenced the perceptions of the robot coach.

The need for mental wellbeing interventions for general populations is ever increasing [1, 29] . Researchers have investigated the efficacy of mindfulness training in alleviating anxiety and depression, improving resilience in students towards stress, fostering emotion regulation strategies and lowering depressive moods [12, 38] . Particularly, the current COVID-19 climate has caused many people to experience a sense of vulnerability, loss of control, and challenging emotions such as fear and grief [17] . Mindfulness practice has been recommended as one potential solution to this situation by encouraging to pay attention to present moment experiences with curiosity and compassion. Mindfulness is also gaining prominence as a wellbeing approach and a life-skill that can be attained with regular practice rather than just being a treatment or therapy.

Despite the benefits, learning and practising mindfulness can be challenging and inaccessible due to lack of trained teachers and training programs, misconceptions about the methods involved and difficulties with establishing a guided arXiv:2006.05289v2 [cs.HC] 11 Jun 2020 regular practice. Virtual conversational agents, in the form of mobile applications and chat-bots, have been developed to improve the accessibility of mindfulness training [18, 27] . However, majority of these agents are not interactive in nature and when they are, they rely on text-based and non-adaptive communication. Social robots with multi-modal interaction capabilities of speech, gestures and vision, constitute a promising solution.

In this study, we investigate the use of robots to deliver mindfulness sessions. For this purpose, we conducted a 5-week mindfulness study where two independent groups received mindfulness sessions, once a week, from an experienced human coach and the Pepper robot 1 teleoperated by the human coach respectively. Interaction sessions with the teleoperated robot were made as natural as possible by replicating the upper-body and head movements of the human coach on the robot, in real-time. The robot also maintained a coherent conversation with the participants where the human coach teleoperating it could speak and listen to the participants in real-time while looking at them. In this paper, we present the results of our experiments, comparing the changes in perception of the participants, over time, towards the human and the teleoperated robot coach.

The design and use of social robots to support wellbeing is an active field of investigation [25] . Interactions with the Paro robot [34, 32] , showed positive effects on wellbeing, social facilitation as well as coping with physical and cognitive impairments in elderly populations. Additionally, children with Autism Spectrum Disorder (ASD) also benefited from similar robot-assisted therapies that enable embodied interactions, increasing engagement and attention and decreasing social anxiety [8] . Robot coaches were also employed to instil behavioural changes in people while dieting where the robot can help individuals keep a track of their weight loss [19] .

Despite positive results, the field of Socially Assistive Robots (SAR) still faces a major challenge with respect to acceptance by various stakeholders. The primary concern comes from the psychologists' lack of confidence in using robots in their practice [7, 9] . Autonomous robots also raise ethical concerns and may not be accepted by the general public [6] . To address some of these concerns, Winkle et al. [40] proposed the mutual shaping approach to the design of SAR and measured significant shift in participants' acceptance of robots pre-and post-study. Cao et al. [4] proposed a supervised autonomous therapy system that enables therapists to take full control of the therapeutic environment, using the robot as an effective tool, rather than being replaced by it. Other aspects of the robots such as their appearance, embodiment, personality, engagement and adaptation to interactions were also studied to make the design of social robots more acceptable [15] .

Longitudinal studies are extremely useful to investigate changes in user behaviour and experiences over time. The motivation behind the investigation of long-term effects in human-robot interaction is that current robots and virtual agents lack social capabilities to engage users over extended periods of time. Some of the early long-term studies showed that the novelty effect quickly wears off and people lose interest and change their attitudes towards robots [13] . This may result in a decreased use of robots and adherence to the interventions being delivered [35, 39] . Leite et al. [21] presented a detailed survey on the use of social robots for long-term interactions and discussed future directions and design considerations to promote their long-term use. De Graaf et al. [14] investigated reasons for the refusal and abandonment of social robots through questionnaires and interviews. Their findings suggest that a key challenge in designing robots for long-term use is to make them easy to use in the short-term as well as functionally-relevant in the long-term. Longitudinal studies also provide us an opportunity to study and design advanced strategies for robots' continuity and incremental behaviours, affective interactions, and memory and adaptation to create a framework that engages users over continued interactions.

The data collected over a longitudinal interaction study is characterized by inter-and intra-individual variability in the overall values as well as the patterns of change. Hence, Generalised Linear Model (GLM) methods such as repeated measures ANOVA may not accurately explain longitudinal changes as they only look at the changes in group means and variances [36] . Moreover, it is important to understand that the total variance cannot be interpreted as random error in longitudinal data. Growth modelling can address these issues as we can look into how individuals change over time and whether there are any differences in their change patterns [24] . Random Coefficients Modelling (RCM) is such an approach that can model group and individual level variability [3] . Univariate longitudinal models can also be extended to analyze multivariate and more complex models.

Our objective is to study the differences in perceptions and the overall effectiveness of mindfulness practice experienced by participants when delivered by a human coach versus a teleoperated robot coach. We conducted a betweensubjects study where participants were randomly assigned to two groups -one group taught by an experienced Human Coach (HC) and the other taught by the Pepper robot teleoperated by the same human coach (RC) (see Section 3.2).

The study consisted of 5 mindfulness sessions, with one session administered each week for around 40 minutes. Each session focused on a specific topic, and the 5 sessions together were designed to provide an introduction to mindfulness techniques and suggest how to integrate mindfulness into daily life. These sessions were structured along several types of pedagogical strategies necessary for effective coaching of mindfulness meditation: didactic, to convey the conceptual basis of mindfulness; experiential learning, via guided meditations; and dyadic interactions designed to maintain participant engagement via peer and coach dialogue customized for each session with the use of external props.

We recruited staff and students across the University of Cambridge, splitting them in two groups; HC, consisting of 2 males and 7 females guided by the human coach and RC, consisting of 6 males and 3 females guided by the teleoperated robot coach. They were offered an incentive in the form of Amazon vouchers upon attending all the sessions. 8 out of the 9 participants in the HC group, and 5 out of the 9 participants in RC group attended all the 5 sessions and the others missed one session each at random.

The experiment design was reviewed and approved by the Ethics Review Board of the Department of Computer Science and Technology, University of Cambridge. All the participants provided informed consent for the recording of audio and video data during the session for further analysis. Participants were advised that they should not be undertaking other professional mental health related treatments or medication if they plan to take part in this study. We also asked the participants to fill PHQ9 [20] and GAD7 [28] questionnaires to assess depression and anxiety scores, respectively. We reviewed the scores to make sure none of the participants were experiencing high anxiety or depression levels.

The participants filled a 20−item personality questionnaire at the beginning of the study [31] . After every session, the participants filled a 'Session Experience' questionnaire about their human or teleoperated robot coach. For the RC sessions, the questionnaire is adapted from a combination of the Godspeed [2] and the human-robot interaction questionnaires [26] . While the former has items focuses on observing the behavior of the robot (Anthropomorphism, Animacy, Likeability and Perceived Intelligence), the latter focuses on how the user interacts with the robot (Robot Motion, Conversation and Sensation). A shortened version, with relevant questions (Likeability, Perceived Intelligence, Conversation and Sensation) for a human coach, is used as the session experience questionnaire for HC sessions. Questionnaires for both RC and HC sessions also measure how the participants felt in the beginning and end of a session with items: Anxious -Relaxed and Agitated -Calm to measure the effect of each session on them. The human coach filled the NASA TLX questionnaire [16] after each session recording the workload experienced during delivering the session both directly and while teleoprating robot.

The human coach teleoperated the Pepper robot to deliver RC sessions. We implemented 3 important components in the setup to enable real-time interactions between the teleoperating human coach and the participants. We established a secure, encrypted local network using a router to connect all the components needed for the transmission of video, audio and pose data between the robot and the human coach (see Fig.1 ).

The human coach was seated in front of a camera used to capture and estimate his upper-body and head pose using the 'Lifting from the deep' algorithm [30] . The algorithm initially worked for a single image was modified to estimate frame-by-frame pose in real-time. The estimated pose coordinates is interpolated from 3 Hz to 30 Hz to obtain smoother movements. Joint angles The head yaw was fixed to 0 • to rectify the inaccuracies in estimating head pose by the algorithm. Real-time image frames were acquired from the camera on the forehead of the Pepper robot using the NAOqi vision API 3 , recording at 30 FPS. The obtained frames are stacked horizontally with slight superposition (percentage of the pixels superposed estimated empirically) to adapt the robot's vision to be projected onto the HTC Vive VR headset 4 . This allows the human coach teleoperating the robot to see 'from the robot's eyes'.

We established a SIP protocol-based audio call using Jisti 5 over a local network between the teleoperator and the robot for the human coach to talk to the participants (through the robot) and listen to what they are saying. Due to bandwidth restrictions and audio feedback issues resulting from using the mic and speakers on the Pepper robot, we used an external speaker/mic placed behind the Pepper, hiding it from the participants, giving the illusion of the robot speaking with the participants.

For analysing longitudinal changes in perceptions towards the human and robot coaches, we use the data collected from the Session Experience and the 20-item personality questionnaire filled by the participants along with the NASA TLX filled by the human coach for each session, each week.

We divided the questions in the RC Session Experience questionnaire into the 8 sections, namely, Anthropomorphism, Animacy, Likeability and Perceived Intelligence (constituting Impressions); Feelings in the beginning vs. the end; Robot Motion, Conversation and Sensation (constituting Interactions). Similarly, we divided the questions in the HC Session Experience questionnaire into four sections -Impressions, Feelings in the beginning vs. End, and Interaction -Conversation and Interaction -Sensations. For further analysis, we used average scores for all items in each section. Impression and Interaction scores are combined together as Perception Scores. We use the scores for Feelings in the beginning vs. the end to study the effect of mindfulness sessions on the participants. Using these evaluations, we investigate the following hypotheses: Statistical Analysis To investigate H1 and H2, we used Random Coefficients Modelling (RCM) which was found to be very useful in analyzing longitudinal data with both inter-and intra-individual variability and missing data points. We assume that the data points are missing at random and therefore does not lead to bias in parameter estimates. We follow the model building approach illustrated by Bliese et al. [3] . We present the results from random intercept model which models the individual variability in the overall levels of their perception ratings. The univariate random-intercept models of the perception scores are extended to test the interaction effect of the personality scores on the intercept variation, to study the effect on overall individual level and slope variation, to study the effect on growth patterns of the perception scores.

Interactions Interactions with the Robot Coach (H2): The longitudinal changes in the Impression score in terms of Anthropomorphism, Animacy, Likeability and Perceived Intelligence are shown in Fig. 2b . Qualitatively, we see that the participants rated the robot low on Anthropomorphism and Animacy but high The longitudinal changes in the Interaction scores, that is, Robot Motion, Conversation and Sensations are shown in Fig. 2c . We found a significant effect for Robot Motion (t = 4.71; DF = 31; p < 0.05) and Conversation (t = 2.57; DF = 31; p < 0.05) with both the scores increasing over time. This suggests that with time, participants found that the robot moved and interacted more consistently as well as easier to converse with.

Mindfulness sessions to promote relaxation and calm (H3): We used the average ratings of two items, namely, Anxious-Relaxed and Agitated-Calm scores, at the beginning and end of each session. We conducted a two-way repeated measures ANOVA with one factor as the Time (5 weeks) and the other factor as Instance (beginning vs. end). We found a significant effect of Instance, in that the scores at the end of the session were significantly higher than at the beginning for both HC (F = 9.365; p = 0.018) (see Fig. 2d ) and RC groups (F = 24.64; p = 0.008) (see Fig. 2e ) suggesting that the sessions made the participants more relaxed and calm. There is no observed effect over time.

Interaction between Perception and Personality Scores: To investigate the effect of participants' personality on their perception scores, we conducted a separate analysis by including each of the personality traits (extroversion, agreeableness, conscientiousness, neuroticism and openness), into the previous random intercept model of each attribute of the perception scores. We modeled the effect of personality traits on both the intercept and slope of the perception scores. Out of the five personality traits, Conscientiousness and Neuroticism showed significant effects on the perception scores. We found a significant negative effect of Conscientiousness on the intercept variability of Robot Motion (t = −2.618; DF = 7; p = 0.032). This suggests that people with higher conscientiousness scores gave low overall ratings for robot motion. However, no significant effect was found with respect to the changes across time. Additionally, we found a significant negative effect of Neuroticism on the intercept (t = −2.789; DF = 7; p = 0.027) and slope (t = −2.968; DF = 7; p = 0.021) variability of Sensation and a significant positive interaction effect between Neuroticism and time on the slope variability of Robot Motion (t = 2.17; DF = 30; p = 0.038). Neuroticism or low emotional stability negatively effects on the overall session experience and also effects how the perception score for robot motion changes with time.

Qualitative workload comparison: The NASA TLX scores collected from the human coach in both HC and RC, indicate an increased perceived workload for the RC sessions (see Fig. 2f ). Although, no statistical analysis was conducted on this data obtained from only one coach, our discussions with the coach highlight this increase as a direct consequence of having to adapt to the use of additional equipment (such as the VR headset) in the RC sessions.

In this paper, we report on research work that bridges two important enablers for improving wellbeing: telepresence robotics and mindfulness coaching. We follow an iterative design process, with the ultimate goal of creating a fully autonomous and adaptive mindfulness robot coach. The first version of the robot coach was implemented using teleoperation with multimodal interaction capabilities (voice and head/arm gestures) that enables us to test our initial hypotheses and better understand the context as well as the expectations and the challenges faced.

Our results show that the impression scores corresponding to items adapted from the Godspeed questionnaire did not change with time for both the human and the teleoperated robot coach. These ratings correspond to characteristic attributes of the coaches and therefore the perceptions regarding these attributes do not change for similar type of interactions across time. Godspeed ratings have usually been used to compare different robots or the same robot performing different tasks [37] and might not be suitable for evaluation of long-term HRI.

The robot coach is rated high for Likeability, showing characteristics such as pleasantness, voice-only communication, synchronous movements which are rated highly likeable [5] ; and Perceived Intelligence, as it is being teleoperated by the human coach [37] . However, Anthropomorphism and Animacy ratings were low for the robot coach. In our future work, we would like to focus on customized generation of gestures where there is a congruence between the speech and movements of the robot to enhance natural perception. For example, Ondras et al. [22] demonstrated that viewers preferred machine-like movements generated to match machine-like synthetic speech of the robot.

We also found a longitudinal increase in the interaction ratings for Robot Motion and Conversation. We did not observe the novelty effect, that is, a drop in ratings of the robot over time. Our post-study interviews with the participants from the RC sessions revealed a preference for the robot that could move synchronously while speaking and could maintain eye contact during conversations. The participants appreciated that they can maintain a coherent conversation with the robot where it checked on them periodically and they could give feedback about how they were doing. These results suggest that participants accepted the use of a robot coach to deliver mindfulness sessions over time but expect functionalities such as generation of natural language and expressive gestures for interaction from an autonomous platform [11, 23] . We also found significant links between participants' personality traits and their perception scores which suggests that person-specific customization should be included in the design of an autonomous coach.

The human coach is rated consistently high in terms of both impressions and interactions. We also did not observe any novelty effect, that is, drop in ratings with time towards the human coach. Van Alderen et al. [33] demonstrated that characteristics such as the embodiment of a non-judgemental and compassionate stance as well as empowerment and non-reactivity play an important role in the acceptance of a mindfulness teacher. The mindfulness sessions in our study are delivered by a certified and experienced coach who has personally been practising mindfulness for many years. The sessions were therefore reported to be beneficial to the participants in both the groups.We note that care should be taken while moving towards designing an autonomous platform, where translating the experience of the human coach into robot delivered sessions is only possible if the human coach has supervisory control over the robot's delivery. The human coach in our experiments reported that although teleoperating gets easier with time, it is still tiresome compared to delivering the session directly. These suggestions motivate us to investigate strategies for creating platforms with supervised autonomy [10, 4] .

An examination of mindfulness-based programs in US medical schools

Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots

Growth modeling using random coefficient models: Model building, testing, and illustrations

Robot-enhanced therapy: Development and validation of supervised autonomous robotic system for autism spectrum disorders therapy

Effectiveness of robot communication level on likeability, understandability & comfortability. In: Proc. IEEE ROMAN

A survey of expectations about the role of robots in robot-assisted therapy for children with ASD: Ethical acceptability, trust, sociability, appearance, and attachment

Are future psychologists willing to accept and use a humanoid robot in their practice? Italian and English students' perspective

Towards interactive robots in autism therapy: Background, motivation and challenges

The clinical use of robots for individuals with autism spectrum disorders: A critical review

How to build a supervised autonomous system for robot-enhanced therapy for children with autism spectrum disorder. Paladyn

Natural language generation for social robotics: Opportunities and challenges

A mindfulness-based intervention to increase resilience to stress in university students (the mindful student study): A pragmatic randomised controlled trial

Designing robots for long-term social interaction

Why do they refuse to use my robot?: Reasons for non-use derived from a long-term home study

Exploring influencing variables for the acceptance of social robots

Development of NASA-TLX (Task load index): Results of empirical and theoretical research

Mental health strategies to combat the psychological impact of COVID-19 beyond paranoia and panic

Enhancing mindfulness-based cognitive therapy with a virtual mindfulness coach

Robots at home: Understanding long-term human-robot interaction

The PHQ-9: Validity of a brief depression severity measure

Social robots for long-term interaction: A survey

Audio-driven robot upper-body motion synthesis

Generic method for generating blended gestures and affective functional behaviors for social robots

Longitudinal research: The theory, design, and analysis of change

Robotics technology in mental health care

Testing a fully autonomous robotic salesman in real scenarios

A review of interactive technologies as support tools for the cultivation of mindfulness

A brief measure for assessing generalized anxiety disorder: The GAD-7

Mindfulness in organizations: A cross-level review

Lifting from the deep: Convolutional 3D pose estimation from a single image

The short IPIP-BFM-20 questionnaire for measuring the big five

Relational artifacts with children and elders: The complexities of cybercompanionship

The role of the teacher in mindfulnessbased approaches: a qualitative study

Living with seal robots-its sociopsychological and physiological influences on the elderly at a care house

When in Rome: the role of culture & context in adherence to robot recommendations

Repeated measures analysis

Meta analysis of the usage of the godspeed questionnaire series

Improving emotion regulation and mood in teacher trainees: Effectiveness of two mindfulness trainings

Social robots for engagement in rehabilitative therapies

Mutual shaping in the design of socially assistive robots: A case study on social robots for therapy