key: cord-0273259-e8tpwjmv
authors: Suomalainen, Markku; Mimnaugh, Katherine J.; Becerra, Israel; Lozano, Eliezer; Murrieta-Cid, Rafael; LaValle, Steven M.
title: Comfort and Sickness while Virtually Aboard an Autonomous Telepresence Robot
date: 2021-09-09
journal: nan
DOI: 10.1007/978-3-030-90739-6_1
sha: bd4869b7bc5ec31ff8611a75a8e0f07f7d4d4197
doc_id: 273259
cord_uid: e8tpwjmv

In this paper, we analyze how different path aspects affect a user's experience, mainly VR sickness and overall comfort, while immersed in an autonomously moving telepresence robot through a virtual reality headset. In particular, we focus on how the robot turns and the distance it keeps from objects, with the goal of planning suitable trajectories for an autonomously moving immersive telepresence robot in mind; rotational acceleration is known for causing the majority of VR sickness, and distance to objects modulates the optical flow. We ran a within-subjects user study (n = 36, women = 18) in which the participants watched three panoramic videos recorded in a virtual museum while aboard an autonomously moving telepresence robot taking three different paths varying in aspects such as turns, speeds, or distances to walls and objects. We found a moderate correlation between the users' sickness as measured by the SSQ and comfort on a 6-point Likert scale across all paths. However, we detected no association between sickness and the choice of the most comfortable path, showing that sickness is not the only factor affecting the comfort of the user. The subjective experience of turn speed did not correlate with either the SSQ scores or comfort, even though people often mentioned turning speed as a source of discomfort in the open-ended questions. Through exploring the open-ended answers more carefully, a possible reason is that the length and lack of predictability also play a large role in making people observe turns as uncomfortable. A larger subjective distance from walls and objects increased comfort and decreased sickness both in quantitative and qualitative data. Finally, the SSQ subscales and total weighted scores showed differences by age group and by gender.

In immersive robotic telepresence, as seen in Fig. 1 , a user wearing a Head-Mounted Display (HMD) embodies a physical robot in a distant location. Besides the visual input through the HMD, the user can communicate with people around the robot through bidirectional audio, and can command the robot to move. This technology creates opportunities to visit museums or nature for people with limited mobility or who otherwise cannot, or grandparents attending grandchildren's birthdays far away. Overall, the technology enables meetings which mix physically present and remotely attending people, such that, for example, a single remote participant can join a physical meeting and actually feel as if she was really there. The robot also allows for touring real facilities or office buildings, helps join physical conferences remotely and facilitates the important impromptu corridor discussions during remote work [50] .

There is evidence that telepresence robot users prefer the robot to handle lowlevel motions autonomously, such that the users only choose the target location for the robot to move in a map or within visible area [41, 1] . Most of current commercial telepresence robots (with standard camera streaming into a standard screen, in other words, not employing an HMD) have a joystick or similar as the main control method, with autonomy functionalities only being rolled out at the moment in products such as the Double 3 robot. However, with the increased immersion and embodiment of an HMD-based telepresence robot, any autonomous path is not suitable but the motions the robot makes must be carefully planned to avoid adversarial effects; pure Virtual Reality (VR) research and applications usually enable and encourage the use of teleportation to avoid VR sickness, often caused by sensory mismatch rising from seeing motion in the HMD while staying stationary. Thus, there is a very limited amount of research on how an immersive telepresence robot should move to make the embodied user feel comfortable.

It is unclear which aspects of autonomous motions make an HMD user feel uncomfortable or experience VR sickness [26, 5] , besides the known result of rotations causing more sickness than translations [22] . For example, distance to walls and objects modulates the optical flow linked to sickness, but simply staying as far as possible from objects and walls is often not a suitable path planning strategy for a robot in complex environments. Also, even though there is evidence that performing turns faster may reduce cybersickness [51, 2] , such fast motions may not feel comfortable for the immersed users and there may be more factors in turns besides the speed that modulate comfort and sickness.

In this paper we make an attempt to disentangle comfort and VR sickness experienced by subjects aboard an autonomous immersive telepresence robot and advise what aspects in the trajectory of such a robot should be paid attention to. First, we present an unexpected result: the amount of VR sickness suffered by users does not affect their choice of most comfortable path, even though paths are shown to induce different amounts of VR sickness. As there is a correlation between experienced VR sickness and comfort, it was expected that people who suffer more from sickness would prefer paths that are more comfortable. Then, we present several results regarding the robot's turns; the Likert-scale answers show that perceived turn speed does not correlate with VR sickness or perceived comfort. However, turns are very often mentioned in open-ended questions as reason for discomfort or the choice of most comfortable path; further analysis of the open-ended questions reveals that even though turn speed was the most mentioned keyword, predictability and length of turn also play major roles in making a turn comfortable or uncomfortable. Additionally, distances to walls and objects were weakly correlated with comfort and sickness, which was also reflected in the open-ended questions, meaning that distance should be kept whenever possible. Finally, we report the effect of age, gender and gaming experience on the results, as well as the Simulator Sickness Questionnaire (SSQ) subscales, finding that women in our study experienced higher levels of sickness on average than men, adding to the currently conflicting results on the topic [13, 39] . Contrary to a recent meta-analysis [45] , in this study older subjects suffered more from the effects of VR sickness. We discuss why subjects in our experiments suffered more from VR sickness when compared to roller coaster studies and find ways to reduce the VR sickness caused by VR-based telepresence to make it a viable option for the general population.

The main contributions of this paper are 1) quantitative analyses of the relation between sickness, comfort, and other variables related to immersive telepresence, with suggestions on turn lengths and speeds, and 2) in-depth analysis of open-ended questions regarding turns, leading to findings that turn predictability and length play major roles in making turns comfortable while retaining the non-sickening abilities of performing the turns fast. Additionally, we present results on VR sickness; carryover effects, effect of demographics and comparison to similar studies, to contribute to the literature on the topic from the perspective of autonomous motions in VR. We note that we performed the experiments purely in VR to avoid confounding factors (such as shaking of the robot) that would arise from using a physical robot. However, we are planning to test the results also on a physical robot to confirm the results.

Telepresence, a term originally coined by Minsky [33] and sometimes referred to as tele-embodiment [38] , is classically, and in this paper, defined as embodying a robot in a remote location. Most of the work on robotic telepresence considers seeing the remote environment through a standard naked-eye display [28, 40, 41, 35, 11] ; these works present the potential of telepresence robots in, for example, conferences and classrooms, with also medical applications being an often researched topic [16, 47] . Additionally, there is research for more personal, intimate, and extensible use cases, such as sharing outdoor activities [15] or more personal long-distance relationships [52] .

There is an increasing number of works that demonstrate the potential of increased immmersion created by an HMD in telepresence, such as better task performance [12] and situational awareness [30] . An interesting motivator is also the finding that in a group work task, users telepresent through a traditional display speak less and perceive tasks as more difficult than the physical participants [48] ; this is exactly the sort of issue that the increased feeling of presence by the user [44] , facilitated by the increased immersion of an HMD, can remedy. A few other researchers have also noticed the importance of robotic telepresence using an HMD: Baker et al. [1] let the user choose the destination similarly as in teleporting and then make the robot move to the destination autonomously. Zhang et al. [53] explored using redirected walking on a telepresence robot. Finally, Oh et al. [36] used such a robot on a tour with several pre-selected destinations.

However, as always when using an HMD, VR sickness can severely deteriorate the experience for many users. A major cause for VR sickness is a conflict between the visual and vestibular senses when self-motion is seen through the HMD but not sensed by the vestibular organs in the ears [42, 5] . In virtual environments, the use of teleportation is frequently used to avoid VR sickness [3] , but in telepresence teleportation would have a significant delay and is thus not a realistic option. Thus, in virtual environments, continuous autonomous motions are often avoided, with perhaps the notable exception of roller coasters often used in sickness studies due to their more sickening effects [8, 34, 31, 18] . Though there is a recommendation that VR sessions should not last longer than 55-70 minutes to avoid overwhelming sickness levels [25] , in the roller coaster studies many people could not even complete a 15 minute session. A meta-analysis found that 15.6% of participants across 46 VR studies dropped out due to sickness effects [45] .

Even though sickness plays a major part in the comfort of an embodied telepresence user, it should not be the only criterion considered when designing motions for a telepresence robot. Becerra et al. [2] showed that the use of piecewise linear paths (meaning that the robot is only rotating in place, and during forward motions there was no rotation) can decrease the VR sickness on embodied participants and make the path more comfortable when compared to a traditional robot path where the robot can rotate and move forward simultaneously; however, they did not explore more detailed questions on why such turns were preferred, or other path aspects such as speeds and distances to objects. Moreover, due to the varying susceptibility to VR sickness [43] , other aspects should not be completely ignored in favor of reducing VR sickness, as some methods to reduce VR sickness (such as driving extremely slowly) also deteriorate the overall experience. To the knowledge of the authors, there are no studies that would consider aspects such as closeness to objects and speeds as factor in immersive telepresence robots motion planning.

To collect results from VR studies that give a more accurate reflection of effects in the general population, demographic information and individual differences must be taken into account. Though there have been conflicting findings in the VR literature regarding gender differences in response to VR sickness [13] , a recent meta-analysis found that SSQ effects were systematically associated with the number of men and women in the study [39] , such that when there are fewer women participants, SSQ scores tend to be higher. In regards to age differences in VR sickness effects as measured by the SSQ, a 2020 meta-analysis of 55 VR articles analyzing VR sickness across types of content and individual factors found that older subjects had lower SSQ scores, and in particular older subjects had significantly lower SSQ scores on the disorientation subscale as compared with younger subjects [45] .

The study was run on a university campus in February 2020 before COVID-19 caused any local restrictions. The users were seated during the study, as shown in Fig. 2 , and shown pre-recorded panoramic videos with an Oculus Rift S using the Virtual Desktop application; as the video was recorded as a 360 video, the users were able to look around in the virtual environment as the virtual robot was moving. The test environment was designed with Unity. To create a more realistic optical flow and experience for the users, there were various paintings on the walls and statues in the gallery of the museum.

Three paths were considered, presented in Fig. 3 . Two of the paths, Pareto Least Turns (PLT) and Pareto Shortest Path (PSP), were piecewise linear, and in [2] they were shown to be less sickening and more comfortable than the third path, the Rapidly-exploring Random Trees (RRT) path. PLT and PSP were paths chosen from a Pareto front across multiple criteria; the Pareto front of a multiobjective optimization problem is the set of solutions from which any objective score cannot be improved without deteriorating another objective. The Pareto front was computed with the objectives of minimizing the number of turns, minimizing the distance to goal and minimizing the amount of time an object was closer than 2m from the subject. The first path, the PLT, (duration 76s, length 72m, min. distance to walls 1m), minimized the number of turns along a piecewise linear path (2 turns), and the second path, the PSP, (67s, 62.6m, 0.4m), minimized the length of the path (4 turns). Both the PLT and PSP had a constant turn speed of 90 deg/s and mean forward speed of 1m/s. The RRT path had smooth turns and was generated by the RRT algorithm [27] , widely used in robot motion planning. This algorithm respects the dynamics of the Differential Drive Robot (DDR) base, which means that the actual path exists in a 5-D space, from which only two dimensions are plotted here; this is why simply curating the path to avoid unnecessary curves, or finding a more optimal path, is infeasible. As the RRT is a sampling-based algorithm and does not provide an optimal path, the algorithm was run 1000 times and the path with the least amount of changes in direction was chosen as the third path (129s, 67.2m, 0.3m, avg. turn speed 18.8 deg/s, avg. forward speed 0.52m/s).

Subjects were recruited from a university campus and the surrounding community. Of the 45 participants, altogether nine subjects were excluded; three quit the experiment due to excessive sickness symptoms (we will discuss possible reasons for this high number of dropouts in Section 5), and the rest were excluded due to technical failures, not completing questionnaires, or for feeling severe sickness symptoms before the study began. Thus, the results are from 36 subjects, age 20-44 with mean 28.25, divided equally between men (n = 18) and women (n = 18). Regarding video game use, 25% of the subjects (n = 9) reported that they never played video games on PC, mobile, or console, 14% (n = 5) rarely played them, 28% (n = 10) played them weekly or often, 19% (n = 7) played them daily, and 14% (n = 5) did not respond.

The Simulator Sickness Questionnaire (SSQ) [21] was used as a measure of VR sickness symptoms. Before seeing any of the path videos, subjects completed a baseline SSQ which was used for screening purposes. They also filled out SSQ's immediately after seeing each video; the order in which the videos were shown to users was counterbalanced to avoid order effects. Then, the subjects completed a questionnaire created for this study after each path. The questionnaire consisted of several questions that were rated on a six-point Likert scale, followed by some open-ended questions asking why they gave that rating. The Likert-rated questions asked subjects how comfortable their experience was on the path (from very uncomfortable to very comfortable), how well they could find their way back to where they started in the museum (from not easily at all to very easily), how they felt about the distance between themselves and the walls or objects in the museum (from too close to too far), and how they felt about the speed of the turns (from too slow to too fast). The open-ended questions asked why a path was uncomfortable if they had given that rating, and if they had any comments. After they had seen all three paths, they answered additional questions asking them to select which of the paths was preferred and most comfortable, and the reason for that choice.

Upon arrival, each subject was given an information sheet about the study and asked to sign a consent form if they wanted to participate. A baseline SSQ was administered, and then the subject was seated in the experiment chair (Fig. 2) . The experimenter read the instructions out loud for the subjects, and then the subject put on the HMD and the first path video was played. After the video, the subject removed the HMD, filled in an SSQ, answered the Likert scale and open-ended questions and drew the path. This process was not timed, but the duration between videos was approximately five minutes. Then, the second video was played and the procedure was repeated. After the third video, the subject completed the final questionnaire comparing the paths, and then reported demographic information and gaming experience. Once they had completed this, they were given a debriefing about the study, copies of the consent forms to take home, and a coupon worth €2 for a coffee from the local cafe.

Data from the same experiment has earlier been used in [2, 32] , where the focus has been on both the technical implementation, comparison of the paths and the naturalness and preference of the user. In contrast, in this paper we do not focus on comparing the paths and path-planning methods, but instead analyze other interesting results accross all paths. The main focus is about perceived user comfort, to present the community more information about the use of autonomous motions in VR-based telepresence. The total number of comments on turns and distances for the "why was the path uncomfortable" question have been reported in [32] , but without the more detailed analysis presented in this paper regarding the actual contents of the open-ended comments. Also, even though individual results on questions regarding comfort across paths have been reported, the relationship analyses between variables reported here have not been presented. Finally, we note that [2] found PSP and PLT causing less VR sickness and being rated more comfortable than the RRT.

Exploratory analyses were conducted with two-tailed significance levels for alpha set at 0.05 and confidence intervals set at 95%. When multiple tests were run, Bonferroni correction within each test was used. Post-hoc power analyses and observed effect sizes [6] were calculated using G*Power [10] , Psychometrica freeware [29] , or by hand [49] . Thematic analysis with an inductive approach [37] was used to classify the responses to open-ended questions by two independent coders. Results from Likert-scale questions, forced-choice questions and SSQ are first presented in Section 4.1, after which responses to the open-ended questions are analyzed in Section 4.2.

Sickness had a negative correlation with comfort. The more the subjects suffered from VR sickness, the less comfortable they felt. A Spearman's rankorder correlation test was run between the Likert comfort ratings, from very uncomfortable (1) to very comfortable (6) , and the total weighted SSQ scores after each path. There was a statistically significant moderate, negative correlation (rs(108) = −.400, p < .001); as the total SSQ scores increased, the Likert comfort ratings decreased.

High sickness scores did not influence people's choice of preferred or most comfortable path. The motivation for checking for this relationship is the assumption that people who suffer more from VR sickness would put a higher weight on how comfortable the path is. To crudely rank the sickness sensitivity of the participants, the highest of the three total SSQ scores for each individual was selected as an index. These scores were then separated into three equal-sized groups of 12 people. Highest total weighted SSQ scores under 15 were in the low sickness group, between 15 and 40 in the medium sickness group, and over 40 in the high sickness group. The relationship between sickness groups and choice of most comfortable path was analyzed using crosstabulation and a Fisher's exact test (two-sided) to account for the small sample size. There was no statistically significant association in choice of the most comfortable path by sickness group, p = .164, w = 0.85. Similarly, there was no statistically significant association between choice of preferred path and sickness group, p = .873, w = 0.58.

Increase in perceived distance to walls and objects had a weak correlation with sickness and comfort. The distance to walls and objects in virtual museum from 1 (too close) to 6 (too far) was compared to Likert ratings for comfort (higher numbers mean greater comfort) and total weighted SSQ scores (higher scores mean more sickness symptoms) using a Spearman's rank-order correlation test. There was a statistically significant weak, positive correlation (rs(108) = .302, p = .002) between the distance to walls and objects and comfort. The closer to walls and objects the paths were rated, the less comfort subjects reported experiencing. Compared to the SSQ scores, there was again a statistically significant weak, negative correlation (rs(108) = −.277, p = .004). The closer to walls and objects the paths were rated, the more sickness symptoms subjects reported experiencing.

There was no statistically significant correlation between the speed of the turns, rated from too slow (1) to too fast (6) , and the Likert ratings of comfort (rs(108) = −.157, p = .105) or the total weighted SSQ scores (rs(108) = .117, p = .228).

There were no carryover effects on sickness with 5 minute breaks between videos. The videos of the museum were counterbalanced by gender and by the order that they were seen in. This counterbalancing was used to allow comparison between paths regardless of whether carryover effects had an impact or not. However, to see if there were carryover effects, a Friedman's test on the SSQ total weighted scores after each video by order was run. There was no statistically significant difference in the distributions of total weighted SSQ scores after the first, second, and third videos, χ 2 (2, 36) = .775, p = .679, W = 0.01. Thus, the recovery time subjects had with the headset off, involving answering questionnaires for about five minutes after each video, appears to have been sufficient for participants to recover from these VR sickness effects. In one extreme case, for example, the subject's total weighted SSQ score after their second video was 153.34, and the score after their third video was zero. Whereas several papers report longer carryover effects (for example [46] ), a large variation across studies has been observed, with lowest recovery times being 10 minutes [9] .

Older people suffered more from VR sickness. Age differences in sickness were examined by splitting the subjects into three similar-sized groups for analysis (small variance in group sizes result from not splitting same-aged subjects to different groups): Under 26 (13 subjects), 26 to 30 (12 subjects), and Over 30 (11 subjects), collapsing across paths, and testing the difference between these groups on each of the SSQ subscales and total weighted scores. A Kruskal-Wallis test showed for the nausea subscale (NS), χ 2 (2, 108) = 19.02, p < 0.001, η 2 = 0.16, the disorientation subscale (DS), χ 2 (2, 108) = 9.23, p = 0.010, η 2 = 0.07, and the total weighted score (TS), χ 2 (2, 108) = 11.48, p = 0.003, η 2 = 0.09, there were statistically significant differences in scores between the age groups. Posthoc pairwise comparisons all showed the same pattern, with the Over 30 age group significantly higher than the 26 to 30 group (NS p = .001, DS p = .019, TS p = .012), and the Over 30 group significantly higher than the Under 26 group (NS p < .001, DS p = .030, TS p = .007), but no difference between the Under 26 group and the 26 to 30 group. The exception to this pattern was on the oculomotor subscale, where there was no statistically significant difference between the age groups, χ 2 (2, 108) = 4.296, p = 0.117, η 2 = 0.02. Mean SSQ subscales and total scores for each age group are shown in Fig. 4 .

Previous VR experience did not influence VR sickness. Subjects were split into three groups based on how often they used virtual reality to test whether or not previous virtual reality experience had an impact on sickness. Ten subjects had never tried a VR HMD (28% of the sample), 15 subjects had tried VR between one and nine times or a few times (43% of the sample), and 10 subjects tried ten or more times or used VR regularly (28% of the sample). One subject did not give a response about his previous VR use and was excluded from these analyses. There were no statistically significant correlations between VR usage frequency and the nausea subscale (rs(105) = .011, p = .915), the oculomotor subscale (rs(105) = −.030, p = .764), the disorientation subscale (rs(105) = .038, p = .697), or the total weighted SSQ score (rs(105) = .008, p = .933).

Women suffered more from VR sickness. To investigate potential differences in sickness by gender, scores across paths were collapsed and compared each Subjective wayfinding ability was weakly correlated with comfort. The idea for testing this correlation is reports that loss of wayfinding ability can cause discomfort [7] . Likert ratings for comfort (higher numbers mean greater comfort) were also tested against the Likert ratings for subjective wayfinding (answer to the question "If you had to go back to where you started in the museum, how well do you think you could find your way back?", with a high number meaning more likelihood of finding the way back) using a Spearman's rank-order correlation test. There was a statistically significant weak, positive correlation (rs(108) = .205, p = .034). The better that subjects believed they could find their way back to the beginning of the museum, the greater the comfort experienced. 

Turns was the most commented aspect making a path uncomfortable, with fast turns being the most commented within turns but unexpectedness and length of turns also playing major roles. After the Likert-rated question regarding how comfortable each path was, subjects were asked an open-ended question, "if it [the path] was uncomfortable, why?" Their answers were coded with keywords based on the text that they provided. A full breakdown of the distribution of all codes for each path can be seen in Fig. 7 . The largest group of comments was in relation to the turns, with 59 comments across all paths; fast turns had the greatest number of individual comments, with surprising turns, sharp turns and many turns also getting at least several mentions. This is in contrast to turn speed not being correlated with the Likert-scale value for comfort.

Whereas fast and surprising turns are quite evident, looking closely into the comments saying "sharp turns" reveals that the word "sharp" is often associated with the "length" of the turn. For example, the comments "SHARP (90 DEGREE)TURNS ARE VERY UN REALISTIC" and "The turn is very sharp and rapid. Turn is like 90 degree, so I feel uncomfortable" were typical. Moreover, the degrees of the turn were mentioned also when turns were classified as "fast", such as "The 90 degrees turns felt very fast.". It is likely that because there is no clear and concise everyday word, especially within non-native speak- Fig. 6 . The frequently found codes from the question "If the path was uncomfortable, why?" which was asked after each path that a each user watched.

ers, to talk about the degrees a turn makes, the "large turns" category is in fact larger than it appears to be in these results.

Turns also had the largest influence on the choice of the most comfortable path. The "size", speed and predictability of the turn seem to have a large effect. The largest number of individual comments were under the code "good turns". However, it become evident that people had varying preference of turns when looking at the responses together with the chosen path. For example, subjects chose the PSP because "It was smoother, there were not so many forced turns" and "The walking speed was not too high, and again, the number of uncomfortable turns was not too high." and "Turn is not rapid and sharp. The distance is also OK.", even though PSP had the same amount of 90°turns as PLT, and additionally two 45°turns. This implies that, even though long turns at once should make people less sick, users would still prefer not to have such long turns and 45°turns do not bother users as much. Additionally, even though only one subject used the word "predictable", terms such as "forced" (above) and "abrupt" ("There were not so much abrupt turns.") were used on the paths not chosen as the most comfortable, indicating that predictability plays a bigger role in making such turns comfortable than what could be deducted from the coding alone.

Small distances to walls and objects were considered a major source of discomfort, and an important factor when choosing the most com- Fig. 7 . The frequently found codes from the question "Of the three paths, which one was the most comfortable? Why?" which was asked after the subject had seen all three paths.

fortable path. There were 26 comments across all paths regarding the distance group of responses in the "why uncomfortable" question, most of which were related to close walls (18) or corners (6) , with only two responses mentioning being too close to objects. A very related category is fear of collision (10 mentions), where most quotes were related to distance to walls ("Sometimes it felt I would bump into the walls"); these responses were detected in all paths, with PSP and RRT being most mentioned but also PLT gathering several mentions, even though PLT had a larger minimum clearance (1m) than the other two paths (0.3m and 0.4m). The responses reveal that it was considered problematic that the 1m clearance was at a 2m wide corridor, which was deemed too narrow ("in the end when we went trough a very narrow path, it felt like the walls were closing in."); indeed, a good distance to objects or walls was often mentioned as a favorable property when choosing the most comfortable path (Fig. 7) , with almost all mentions being together with the PL. When inspecting the comments regarding most comfortable path, the distance to objects is mentioned as often as distance to walls. However, from the five mentions of "distance to objects", only one considered that the objects were close enough ("going near to the objects"), whereas another one simply stated that distance was "good" and the rest preferred staying as clear from objects as from walls and corners.

We expected to find a stronger link between the perceived comfort and SSQ scores than what the results showed (no relationship between the SSQ and the choice of "most comfortable path," and a moderate correlation between the SSQ and the Likert-scale comfort); we expected people with higher SSQ scores to prefer paths that, in general, caused less sickness (PLT and PSP). The lack of this connection shows that there are more facets to a user's comfort than only the often used SSQ; even though experiencing VR sickness can have the most impact on users, sickness symptoms may not be the most significant part of the VR experience for the users. Several subjects stated this outright, such as "It was not sickening but not comfortable in the way the (virtual) robot moves". The importance of this is highlighted by a strong relationship between perceived comfort and preference reported in [32] , meaning that people can like things that make them a little sick. Also, comfort is a more subjective feeling than sickness, and discomfort does not necessarily equal sickness; all of this should be taken into account when designing VR experiences and new methods for quantifying the sickness effects and comfort of VR exposure.

We were also expecting that perceiving a fast turn speed would not have an effect on the SSQ, based on [51] finding that performing the same turn faster than slower makes people less sick; the data confirmed this. However, it came as a surprise that even though the turns were most frequently mentioned in the open-ended questions regarding why users felt a particular path was uncomfortable, there was no statistically significant correlation between Likert-scale turn speed ratings and comfort. A likely reason is that we specifically asked about the turn speeds in the Likert-scale question; thus, the Likert-scale question did not consider any other aspects of the turn besides speed. Besides the earlier mentioned quotes where the 90°turns were specifically mentioned, also other quotes with fast turns had additional adjectives hinting towards the surprise element, such as "the movement was slow but, the turns were fast a jarring". Another subject, who chose PSP as the most comfortable path, stated that PSP "Didn't contain sharp turns and lot's of unreasonable moves. ", even though it contained the same amount of 90°turns as PLT, and additionally two 45°turns; this suggests that the 45°turns were not considered as uncomfortable as the longer turns. These findings indicate that there could be even a contradiction between perceived comfort and experienced VR sickness during turns; long turns are considered uncomfortable, even though they reduce sickness. We believe this should be taken into account when designing the motions of a telepresence robot, and make a more controlled experiment about turn speeds and lengths in VR, where besides sickness also comfort is queried. Even though the results are only preliminary, we suggest avoiding 90 degree turns at once and making the turns more predictable. We still suggest keeping the turns fast, at least close to the 90 deg/s, since there is known evidence of effect on VR sickness but contradictory evidence with comfort.

Distance to walls and objects had a correlation with both comfort and sickness, even if weak, and was also mentioned frequently in the open-ended ques-tions. This was expected, as a closer passing distance increases the optical flow, which is linked to increase in VR sickness. Additionally, fear of collision was mentioned a few times, which, as a strong feeling causing anxiety, may have a strong effect on comfort and sickness [14] . Based on the results, 1m passing distance did not cause users any discomfort unless in a narrow corridor, but such a passing distance would make robot motion planning complicated. However, it would be beneficial to verify the result on a real robot in a real environment, with more variance of passing distance.

The observed relationship between subjective wayfinding ability and comfort is not surprising, since there is evidence in the literature that loss of wayfinding ability can cause discomfort [7] . The environment used in this study was too simple to test objective wayfinding ability, and the correlation is only weak, with also the possibility that this is only a coincidence; however, the observed connection and evidence in literature indicates that a more focused study would be useful to explore whether the wayfinding ability can be increased by careful planning of the robot's motions.

The subjects succumbed to VR sickness quickly considering that the videos were not very long (67s-129s) and the robot moved slowly (0.5m/s − 1m/s); however, the subjects also recovered faster than suggested by earlier literature. For example, Kourtesis et al. [25] state that VR sessions can be comfortable up to 55 to 70 minutes, but we experienced three dropouts (two during their first videos, which were RRT and PLT, and one while watching RRT as second video after already finished PLT) with videos lasting less than two minutes. This supports the proposition that continuous motions, without a visual, stationary cue that moves with the user, are a significant contributor to VR sickness. However, the surprisingly fast recovery time (no carryover effects detected with 5 minute breaks, whereas the lowest suggested required recovery time from literature is 10 minutes [9] ) balance this; these results hint that VR sickness with continuous motions raises fast but also decreases fast. However, more focused research on VR sickness for continuous motions would be required to confirm this.

Comparing to the SSQ scores caused by roller coasters in two other studies (both using HTC Vive), the results from this study were comparable or slightly higher when adjusted for the duration of the stimuli; Islam et al. [18] had no dropouts in a 15-minute roller coaster ride with a mean SSQ of 55.45, and McHugh et al. [31] had no dropouts in a 5 minutes 50 seconds exposure with a mean SSQ of 34 (visually approximated from figure), whereas our mean SSQ was 20.3; thus, it seems that the telepresence experience provided a higher variance of sickness to test subjects than a roller coaster, since in our study there were more dropouts but less average sickness. In fact, there are a few considerable differences between telepresence and a roller coaster, both adding and reducing the potential for VR sickness. A roller coaster moves much faster, which increases the optical flow and the probability for sickness, but it has a constant frame of reference (the car) which is known to decrease VR sickness [4] . Also, most of the scenery does not move with the speed of the roller coaster if it travels higher than on ground level, reducing the optical flow; a reduction of sickness has been reported when a plane flies higher above the ground [20] . Additionally, the turns are predictable -the user can always see the track ahead, which has also been shown to reduce VR sickness [24] . We also observed individual comments in open-ended questions on turns being unpredictable ("the turns were too fast, but thankfully pretty predictable this time.") and, in regards to why the path was uncomfortable, "The turns were very strong (sudden)" and "It was not sickening but not comfortable in the way the (virtual) robot moves. Sudden turns very close to the corners of the walls." Thus, future ways to combat VR sickness in autonomous telepresence motions could be visualizing the path to the user so that movements are less unexpected and providing a constant frame of reference, such as visualizing the robot.

Women experienced greater sickness symptoms during the study than men, which has been found previously in VR research [43] ). In this study, previous VR experience did not influence the severity of VR sickness, which is in agreement with [25] , even though such an effect through habituation [17] has been reported elsewhere [43] . Regarding higher sickness in older subjects, we were unable to find similar work in the literature, and in fact found conflicting results [45] . One explanation could perhaps be related to differences in lens flexibility in the eye across the lifespan, making it easier to focus on things very close to your face when you're younger and sometimes resulting in the need for reading glasses later in life. It will be an interesting topic for future research whether there is something specific in continuous motions that causes this discrepancy.

We established in this study that the design of the robot's turns are very important for increasing comfort and reducing VR sickness. Based on the results, we can provide initial suggestions for designing robot motions, but to draw decisive conclusions more research is needed. For example, it seems that a Likert-scale question about the length of the turns should have been asked, to pinpoint with more accuracy which aspect of the turn makes them so uncomfortable. Moreover, we want to test how big a role predictability plays, to make the experience closer to a slow speed roller coaster. The predictability could be done either explicitly, such as using arrows or a path on the ground, or by modifying the velocity profile of the turns; for example, the common trick used in animation to employ high-order derivatives of speed, such as jerk, could be useful to increase predictability by making the turn start slower but then increase the speed to take advantage of the less-sickening nature of performing turns fast [51] .

The different durations of the videos could have had an effect on the SSQ measures. However, as the travelled distance was still similar, this means that the higher speed had less of an impact on the experienced VR sickness. The slower RRT path made some unnecessary turns, but it is a de facto standard robot motion planning algorithm, and therefore we believe this to be a useful comparison. However, a more direct comparison between a path similar to the PSP but with smooth turns could also be useful to pinpoint the effect of rotation in place. We did not measure head motions, but as they have been successfully used in predicting VR sickness [19] , it would have been interesting to test their correlation to VR sickness; ad-hoc observations from the study instructors indicate that an increase in sickness was observable by a decrease in the subjects' head motions.

We intentionally had art in the museum to make the study more realistic, both in terms of ecological validity and optical flow. Besides an expected effect on the subjects' preferences, the art also affected the perceived comfort; users reported in open-ended questions issues such as how comfortably they could see the art ("Many times I was just facing the wall and couldn't see the displayed items"). Whereas it is important to acknowledge the effect that the situation and environment have on comfort, it is difficult to quantify. In the future we plan to run another study in an empty museum and compare the results; this will give us a proper, quantifiable metric on the importance of context, at least between the cases of museum and a simple path from A to B.

The study was run in a virtual environment, even though the ideas are meant for a telepresence robot streaming real video footage. Although there is some evidence that 360 • videos make people more sick than virtual environments [45] , there is no single study comparing these two cases on sickness (although it has been shown that teleoperation increases stress over a virtual environment, which in turn can increase discomfort and sickness [23] ); moreover, as there is no reason to believe that the mechanisms causing sickness would work differently in a 360 video than in a virtual environment, we expect our results to extend to real video capture. Nevertheless, we will also run a study with a real telepresence robot to verify these results.

The number of subjects in the study was small. We tried to account for this by using statistical tests that are more sensitive for small samples when appropriate, and by calculating the effect sizes and then running post-hoc power analyses. One final limitation was the choice of a six point Likert scale rating instead of a standard five or seven point scale. This scale was selected so that subjects would need to make a decision either way on the ratings and could not simply select a neutral choice. In future studies, a more traditional scale will be used.

We presented an analysis focusing on the interplay between and possible causes of comfort and sickness of VR telepresence users with regards to the path taken by the robot. In essence, discomfort and sickness are distinct yet overlapping concepts, and more focus should be put on addressing the comfort of the user instead of simply targeting VR sickness since reducing only sickness may cause other issues that degrade the whole experience. As autonomous motions are rarely considered in virtual worlds, this paper will hopefully spark further interest in research from other investigators both in telepresence and in autonomous continuous motions in virtual worlds.

Towards an immersive user interface for waypoint navigation of a mobile robot

Human perception-optimized planning for comfortable vr-based telepresence

Locomotion in place in virtual reality: A comparative evaluation of joystick, teleport, and leaning

Visually-induced motion sickness reduction via static and dynamic rest frames

Virtual reality sickness: A review of causes and measurements

Statistical Power Analysis for Behavioral Sciences. Routledge, 2nd edn

Spatial orientation, wayfinding, and representation

Comparing the onset of cybersickness using the oculus rift and two virtual roller coasters

Can simulator sickness be avoided? a review on temporal aspects of simulator sickness

G*power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences

Are we there yet? comparing remote learning technologies in the university classroom

A natural interface for remote operation of underwater robots

Are modern head-mounted displays sexist? a systematic review on gender differences in hmd-mediated virtual reality

Influence of video content type on users' virtual reality sickness perception and physiological response

Geocaching with a beam: Shared outdoor activities through a telepresence robot with 360 degree viewing

A review of telepresence, virtual reality, and augmented reality applied to clinical care

Characteristics of habituation to motion in a virtual environment

Automatic detection of cybersickness from physiological signal in a virtual roller coaster simulation

Automatic prediction of cybersickness for virtual reality games

Introduction to and review of simulator sickness research

Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology

Axis rotation and visually induced motion sickness: the role of combined roll, pitch, and yaw motion

Spatial presence, performance, and behavior between real, remote, and virtual immersive environments

Simulator sickness in virtual environments

Validation of the virtual reality neuroscience questionnaire: maximum duration of immersive virtual reality sessions without the presence of pertinent adverse symptomatology

Virtual reality

Randomized kinodynamic planning

Now, i have a body: Uses and social norms for mobile remote presence in the workplace

Calculation of effect sizes

Immersive 3-D teleoperation of a search and rescue robot using a head-mounted display

Investigating a physical dial as a measurement tool for cybersickness in virtual reality

Analysis of user preferences for robot motions in immersive telepresence

Correlating reaction time and nausea measures with traditional measures of cybersickness

From being there to watching: Shared and dedicated telepresence robot usage at academic conferences

VR based robot teleoperation interface for virtual tour

Qualitative research. Encyclopedia of statistics in behavioral science

Social tele-embodiment: Understanding presence

Mind the gap: The underrepresentation of female participants and authors in virtual reality research

Bodies in motion: mobility, presence, and task awareness in telepresence

Robotic telepresence at scale

Motion sickness

Individual variation in susceptibility to cybersickness

From presence to consciousness through virtual reality

Factors associated with virtual reality sickness in head-mounted displays: a systematic review and meta-analysis

Virtual environment sickness: adaptation to and recovery from a search task

Mobile robotic telepresence solutions for the education of hospitalized children

Wait, can you move the robot? examining telepresence robot use in collaborative teams

The need to report effect size estimates revisited. An overview of some recommended measures of effect size

Exploring use cases for telepresence robots

Assessing postural instability and cybersickness through linear and angular displacement

Our house: living long distance with a telepresence robot

Detection thresholds for rotation and translation gains in 360 video-based telepresence systems