key: cord-0594240-hr4ge1i4
authors: He, Kerry; Simini, Pradeepsundar; Chan, Wesley; Kuli'c, Dana; Croft, Elizabeth; Cosgun, Akansel
title: On-The-Go Robot-to-Human Handovers with a Mobile Manipulator
date: 2022-03-16
journal: nan
DOI: nan
sha: 739edf5bae4bf27e1e0c4b7e86c426b4f8947ccc
doc_id: 594240
cord_uid: hr4ge1i4

Existing approaches to direct robot-to-human handovers are typically implemented on fixed-base robot arms, or on mobile manipulators that come to a full stop before performing the handover. We propose"on-the-go"handovers which permit a moving mobile manipulator to hand over an object to a human without stopping. The on-the-go handover motion is generated with a reactive controller that allows simultaneous control of the base and the arm. In a user study, human receivers subjectively assessed on-the-go handovers to be more efficient, predictable, natural, better timed and safer than handovers that implemented a"stop-and-deliver"behavior.

Today's robots are most commonly found in manufacturing industries and warehouse automation, and typically operate in isolation from humans. The introduction of robots that coexist and interact with humans is creating an opportunity for new use cases, in both manufacturing and service robotics. One such use case is delivery robots. Robotic deliveries can either be performed indirectly by leaving the object in the vicinity of the human receiver [1, 2, 3] or directly by handing over the object to them. The majority of research in robotic handovers is focused on direct handovers using fixed-base manipulators [4, 5] . On the other hand, mobile robots allow for more control over when and how to approach a human receiver for fetch-and-carry tasks. While more recent works implement handovers on fixed-base robotic arms, earlier works feature humanoid robots [6, 7] and mobile manipulation platforms [1, 3, 8, 9, 10, 11, 12] . In this work, we focus on robot-to-human handovers performed with a mobile manipulator.

Drawing parallels with human-to-human handovers, we would expect a human handing over an object to be able to perform the handover without completely stopping in their path. The handover is more efficient as the giver can move onto their next task quicker. This observation motivates us to transfer this behavior to robot givers. Our recent survey on robotic handovers revealed that none of the prior works proposed keeping the robot's base moving during the physical exchange stage of the handover [13] . We identified only two papers that considered the mobility of the handover agents. Mainprice et al. [14] plans a path considering the mobility of a human receiver; however, the robot comes to a stop at the object transfer point. Kupcsik et al. [15] considers cases where the transfer point moves for a human receiver who may be walking or running; however, the handover is performed with a fixed-base robotic arm. In this paper, we consider mobile manipulators and exploit their full motion capability in robot-to-human handovers. We present on-the-go handovers, in which the robot's base keeps moving even during the physical exchange stage of the handover, as illustrated in Fig 1. To achieve on-the-go handovers, we adopt the reactive controller by Haviland et al. [16] , which generates motions by moving the robot's base and its arm simultaneously. Our work investigates the subjective perceptions of human receivers for on-the-go handovers compared to stop-and-deliver handovers. To this end, we conduct an in-person user study in which we compare two handover behaviors (on-the-go vs stop-and-deliver) and two agent types (robot vs human). The contributions of this work are twofold:

• We propose on-the-go handovers in which the robot's base continues to move during physical transfer. To the best of the authors' knowledge, this is the first reported study of mobile manipulation to perform object handovers throughout all phases of the handover. • User studies that confirm the benefit of the on-thego handovers compared to standard stop-and-deliver handovers in terms of subjective assessments by human receivers.

As on-the-go handovers are faster, we expect people to perceive it as more efficient and better timed. We also expect on-the-go handovers to be perceived as more natural and predictable as it conforms more closely to what a human would do, and more competent as it requires the robot to multi-task. In addition, we expect people to perceive the handover as safer as the motion is smooth and blended together. Furthermore, we expect that a robot giver will emphasize the differences between two handover behaviors more than a human giver would, as people are less familiar with robot handovers and will therefore be more attentive and critical towards it. Stop-and-deliver On-the-go Therefore, we consider the following hypotheses:

• H1: On-the-go handovers will be perceived more positively than stop-and-deliver handovers in all subjective measures. • H2: Compared to stop-and-deliver handovers, on-the-go handovers will have a larger positive effect on subjective metrics for a robot giver compared to a human giver.

Robot-to-human handover experiments are implemented on a Fetch Mobile Manipulator which consists of a robotic arm mounted onto a non-holonomic mobile base, and a head which can pivot about two axes.

A 2D occupancy map of the experimental environment is pre-computed by using a SLAM algorithm which fuses base laser scanner and depth camera measurements [17] . This map is then used to perform on-line localization using a particle filter algorithm [17] . The position of the seated human is assumed to be known within this map.

The robot is controlled using an optimization-based reactive controller [16] , which allows the simultaneous movement of the robot's base and the robotic arm. Target end-effector task-space velocities are computed to drive the end-effector in a straight line towards a waypoint defined relative to the mobile base. The optimal control problem is augmented with an additional control objective, formulated as a hard equality constraint, for the mobile base to meet a specified linear and angular velocity. These target velocities are calculated using a proportional controller to drive the base towards the final goal. The robot head is controlled independently using a similar proportional controller.

To choose the robot arm configurations at the start and at the transfer point of the handover, we use the findings by Cakmak et al. [8] who propose using contrast in the robot's actions for handovers. We adopt their idea of spatial contrast, by making the robot's hand-over pose distinct from other things that the robot might do with an object in its Tucked

On-the-go Stop-and-deliver hand. The tucked and handover configurations of the arm are fixed a priori, and defined relative to the robot base. These configurations are chosen to be within arm's length of the receiver, able to be quickly and smoothly transitioned to by the arm, and avoids colliding with the table and other obstacles.

The robot begins with its arm in a tucked configuration (0.3 m forwards, 0.6 m high) oriented away from the participant so that the object is not accessible by the participant (as shown in Fig. 3 , left image). When a handover is performed, the end-effector is outstretched and tilted down slightly towards the participant (as shown in Fig. 3 , middle and right images). This change in configuration aims to maximize the spatial contrast between the robot's states to clearly communicate to the participant when the object is ready to be grasped. In addition, a simple gaze behavior is adopted where the robot looks at the participant while it approaches them and performs the handover. This was inspired by a user study by Zheng et al. [18] that found that for robot-to-human handovers, people tend to reach for the offered object earlier when the robot provides a continual gaze cue towards the receiver's face. The robot then looks away towards the goal once the handover is complete.

The sequence of behaviors for the robot handovers is described below, and visualized in Fig. 2 :

• On-the-go (OtG): The robot begins with its arm in a tucked position and the mobile base begins moving down the corridor. Once it is within 2.5 m of the participant, the mobile base gradually slows down to half its original speed, and the arm is extended towards the participant (0.3 m forwards, 0.5 m to the side, 0.8 m high). Once the robot has passed the participant, the mobile base speeds up to its original speed and retracts its arm to its original tucked configuration. For human-to-human handovers, the human giver mimicked the robot's behavior as closely as possible. The role of the human giver was performed by the same experimenter in all experiments for consistency.

The participant, who was seated at a table next to the corridor as depicted in Fig. 4 , was tasked to receive an object from the robot. The robot approaches the participant from their right side, following Dautenhahn et al. [19] who found that the right side was the approach direction people are most comfortable with, with no significant differences in preferences being found between handedness. The giver starts at one end of the corridor where the participant cannot see them, so that they cannot initially know whether it is a robot or a human performing the handover. The giver is tasked with navigating to a goal at the opposite end of the corridor while performing the handover. A soft toy is used as the handover object and is loosely held in the gripper such that participants can easily take the object from the robot's gripper anytime, without the robot needing to detect and respond to when the person is taking the object.

Prior to the trials, the participant was told that there will be two handover behaviors and two handover givers, and that the handover agent will move down the corridor while performing the handover. Each participant is exposed to all 4 types of handovers, as visualized in Fig. 5 . The order in which participants experience these handovers is randomized, and they are not aware of which handover will happen before each trial. Each experimental trial began after the participant gave a verbal cue to the operators when they are ready. After each trial, the participant was asked to fill out a questionnaire regarding their experience with the latest trial. More details about the survey are given in the next section.

To gauge the experiences of each participant during the experiments, participants were asked to fill out a survey after each trial to test the hypotheses listed in Sec. I. After each one of the 4 trials, the participants responded to the following 6 statements using a 5-point Likert scale:

1) The giver was efficient in completing the handovers.

2) The interaction with the giver felt natural.

3) The giver's timing was appropriate. 4) The giver was competent in giving objects to me. 5) I felt safe during the interaction. 6) I was able to predict what the giver was going to do. In addition, at the end of the 4 trials, participants were asked about which handover behavior they preferred (On-The-Go/Stop-and-Deliver/No Preference):

• Which robot giver handover behavior did you prefer?

• Which human giver handover behavior did you prefer? At the end of the study, participants were asked to provide any additional comments they might have about their experience.

We recruited 15 participants from the lab and University premises 1 , including 11 male and 4 female participants between the ages 20 and 31 (µ = 22.9, σ = 3.56). All participants had some prior experience working with robotic platforms. There were 3 out of the 60 trials that were repeated for the following reasons: an error in the experimental setup, a localization error of the robot, and a handover failure due to a participant's uncertainty about how to interact with the robot on their first trial (a stop-and-deliver robot-to-human handover). All following statistical tests are performed to a 5% significance level. The distribution of all responses are shown in Fig. 6 and the user preferences are shown in Fig. 7 .

Due to the ordinal nature of the Likert scale variables, a non-parametric test was applied. To test H1, a singletailed Pratt modified Wilcoxon signed-rank test was ran for paired differences between perceptions of the on-the-go and stop-and-deliver handovers. As we believe that a robot agent compared to a human agent fundamentally changes the nature of the handover, we do not treat our experiment as a 2×2 factorial design, and instead analyze the robot-to-human and human-to-human experiment results independently. The statistical test results are summarised in Table I For human-to-human handovers, on-the-go handovers were rated significantly better than stop-and-deliver handovers with respect to all survey questions. This was expected, as it is natural for people keep moving as they are handing over objects.

For robot-to-human handovers, on-the-go handovers were found to be significantly more efficient, natural, safer, predictable, and to have better timing. These results affirm H1 to an extent. Interestingly, we did not find a statistically significant improvement in the perceived competence of the robot, therefore H1 was not fully affirmed.

To test H2, we define two new sets of variables as the difference between on-the-go and stop-and-deliver handover ratings for each of the robot and human agent experiments. Single-tailed Pratt modified Wilcoxon signed-rank tests are then used to test for H2 for each subjective metric. The results for this statistical test are summarized in Table II . No subjective metric featured a statistically significant larger positive effect when using on-the-go handovers over stop-and-deliver handovers when compared between the two handover agents.

From the survey data shown in Fig. 6 , we observe that all participants felt safe with the on-the-go robot-to-human handover (all either agree or strongly agree). We believe factors including the robot slowing down when it is closer to the participant, and approaching from the side as opposed to a front-on approach, helped to achieve this result. However, out of all of the subjective metrics for on-the-go robot-tohuman handovers, participants rated the handover's naturalness and competence the lowest, with only two participants strongly agreeing that the robot exhibited each of these traits. This is likely influenced by the specific handover behavior implementation used, such as the timing of the handover, positioning of the handover point, gaze patterns, and other possible expected verbal or non-verbal cues. Future work is recommended to fine-tune the behavior of the onthe-go handover make the handover feel more natural and competent.

We also observe that receivers subjectively preferred human givers over robot givers (p ≤ 0.023 for all subjective measures and givers using a Pratt modified Wilcoxon signedrank test). This is somewhat in contrast to the study by Unhelkar et al. [20] which did not find a significant difference in the perceptions of fluency between human and robotic givers. As opposed to their study, we did not ask about the fluency of the interaction in our survey, and their platform was a mobile robot without an arm which might change the Robot Giver Human Giver perception of the receivers. Our hypothesis tests show that robot handover competence was the only metric that did not improve when comparing on-the-go handovers to stop-and-deliver handovers. This may be due to participants perceiving the robot givers as equally less competent compared to human givers, due to drawbacks shared between both robot handover behaviors such as not reactively adjusting to the receiver's preferences. However, further work should be conducted to verify this hypothesis. All other subjective metrics favoured on-the-go handovers.

Although the results regarding the subjective metrics suggest that on-the-go handovers are superior overall, this preference is less clear when participants were asked explicitly which handover they preferred, as shown in Fig. 7 . Only a slight majority of people preferred robot on-the-go handovers (OtG: 54%, S&D: 33%, No pref.: 13%). A possible explanation for this is that the preferred handover is also dependant on the specific task, as posited by Martinson et al. [21] . This would be a factor participants consider for their overall handover preference, but not when responding to Likert items regarding the specific task used in the user study. We received the following comments which support this:

• "[It] felt like each handover is good for a different purpose.

[I] preferred [the on-the-go] handover for this particular [object] ." • "If the robot was bringing me a plate of food, I would have preferred the [stop-and-deliver handover] since its safer."

Therefore, the results regarding subjective metrics may only be relevant to this specific handover task. Future work generalizing on-the-go handovers to more tasks may consider the role of object semantics for handovers. Another observation is that participants were influenced by differences in handover behaviors not directly arising from the on-the-go-stop-and-deliver handover dichotomy. Three participants commented that the position the agents stopped at to perform the stop-and-deliver handover (directly to the right of the participant) felt awkward, and would have preferred the robot to stop slightly earlier. However, this may also be an advantage of on-the-go robots, as participants can choose where to perform the handover along a continuum. Three participants commented that they would have preferred the robot on-the-go handover to reach its arm a bit further or higher. Future work may be required to remove these slight unwanted behavioral differences to ensure there are no confounding effects in the experiment. Alternatively, a more intelligent method to adaptively choose the handover location based on the user's position and/or preference could be beneficial to people's perception of the handover.

H2 was not supported by the results, showing that there was not a significant difference between people's perception of the improvement of on-the-go handovers over stop-anddeliver handovers between human and robot agents. As opposed to our original justification for H2, this may instead suggest that the benefits of a on-the-go handover are as relevant for a human agent as they are for a robot agent.

First stopping, and then delivering the object is the standard way performing handovers with mobile manipulation platforms in the existing literature. On the other hand, "onthe-go" handovers, where the robot does not come to a stop during physical handover, can be more efficient since the giver can move to its next task earlier. In this paper, we conduct a user study to investigate the subjective perception of on-the-go handovers compared to stop-and-deliver handovers, from the receivers' perspective.

Our user study results showed that receivers subjectively assessed robot-to-human on-the-go handovers as more efficient, natural, safer and predictable and to have a better timing compared to stop-and-deliver handovers. This suggests that robot-to-human handovers should be performed as on-the-go for scenarios similar to our experiment, however, further qualitative research is needed to understand when and where on-the-go handovers are suitable [3] .

A limitation of this work is the assumption that the human receiver always pays attention to the handover task. In realworld deployments, users would divide their attention between the handover and other activities they are doing, which might be as simple as checking their phones [21] . Catching the attention of the handover target is more important for onthe-go handovers compared to fixed manipulators, because if the robot cannot catch the attention of the human receiver, then the on-the-go handover would be reduced to a stopand-deliver behavior. Detecting if the user is ready for the handover would be important for real-world situations [22] . The robot can also actively seek to increase the chance of engagement by using gaze cues [23] , better approach angles [24, 25] , predictable arm motions [26, 27] , audio cues [28] , or a combination thereof [29] . If waiting for the user to complete the handover is deemed to be detrimental to the efficiency of a robot, then indirect handovers can be a way to circumvent this issue [2] , where the robot places the object on a table [30] .

Other limitations of our work include the assumption that the receiver's position is known, and that we don't detect if the user actually picked up the object from the robot's gripper. These limitations can be addressed with perception modules for detecting people [22] , predicting the object transfer point [31] , and checking grip forces to detect if the object has been picked up [32] .

Future work should consider situations where the human receiver is walking instead of sitting, or extend our previous work on human-to-robot handovers [5] into on-the-go handovers. For such handovers, the robot would need to keep the target object in-sight as much as possible [33] , and a faster gripper might be needed. It may also be useful to utilize a projection method such as augmented reality to make the robot's intent accessible to the users [34] .

V. ACKNOWLEDGEMENT This project was supported by the Australian Research Council (ARC) Discovery Project Grant DP200102858.

Hand it over or set it down: A user study of object delivery with an assistive mobile manipulator

Preemptive motion planning for human-to-robot indirect placement handovers

Learning user preferences for robot-human handovers

Reactive human-to-robot handovers of arbitrary objects

Object-independent human-to-robot handovers using real time robotic vision

Determining proper grasp configurations for handovers through observation of object movement patterns and inter-object interactions during usage

Synthesizing object receiving motions of humanoid robots with human motion database

Using spatial and temporal contrast for fluent robot-human hand-overs

Synthesizing robot motions adapted to human presence

A human-inspired object handover controller

Workspace analysis for planning humanrobot interaction tasks

Hand in hand with robots: differences between experienced and naive users in human-robot handover scenarios

Object handovers: a review for robotics

Sharing effort in planning human-robot handover tasks

Learning dynamic robot-to-human object handover from human feedback

A holistic approach to reactive mobile manipulation

Indoor navigation system using the fetch robot

Impacts of robot head gaze on robot-to-human handovers

How may i serve you? a robot companion approaching a seated person in a helping context

Comparative performance of human and mobile robotic assistants in collaborative fetch-anddeliver tasks

Towards understanding user preferences in robot-human handovers: How do we decide?

RO-MAN Workshop in Active Vision and Perception in Human-Robot Collaboration

Meet me where i'm gazing: how shared attention gaze affects human-robot handover timing

Learning how to increase the chance of human-robot engagement

How to approach humans? strategies for social robots to initiate interaction

Legibility and predictability of robot motion

An experimental validation and comparison of reaching motion models for unconstrained handovers: Towards generating humanlike motions for human-robot handovers

Review of semantic-free utterances in social human-robot interaction

A model of distributional handing interaction for a mobile robot

Learning to place objects onto flat surfaces in upright orientations

Object transfer point estimation for fluent human-robot handovers

Grip forces and load forces in handovers: implications for designing humanrobot handover controllers

Visibility maximization controller for robotic manipulation

Visualizing robot intent for object handovers with augmented reality