key: cord-0122163-93hmqroh authors: Teodorescu, Catalin Stefan; Carlson, Tom title: AssistMe: Policy iteration for the longitudinal control of a non-holonomic vehicle date: 2022-02-05 journal: nan DOI: nan sha: 75edc23e9145c69e02041f0234835c4daea64268 doc_id: 122163 cord_uid: 93hmqroh In this article we design a physically-inspired model-based assist-as-needed semi-autonomous control (ASC) algorithm to address the problem of safely driving a vehicle (a power wheelchair) in an environment with static obstacles. Once implemented online, the proposed algorithm requires limited computing power and relies on pre-computed (offline) maps (look-up tables). These are readily available by implementing policy iteration that minimizes the expected time to termination (safely stopping near an obstacle), by taking into account: (i) the vehicle dynamics; (ii) the drivers' intention modeled as three separate stochastic processes. We call them the expert driver, the naughty child and the blind driver models. A study with healthy participants confirmed that ASC outperforms a baseline rule-based control (a statistically significant result). This work is inspired by a real need for bespoke control solutions applicable to medical robotics. In the European project "ADAPT", clinicians (medical doctors) identified the needs of wheelchair users [1] and in response, roboticists proposed to make use of technology and novel scientific approaches to tackle those challenges, one of which is addressed in this article. In particular, in health and social care, about 840,000 people in the UK are regular wheelchair users [2] living with multiple impairments (physical, cognitive, visual, hearing) who might benefit from an assisted robotic wheelchair that can help them navigate safely in any indoor or outdoor environment. There are currently no such commercially available devices on the market. Consequently, they are at risk of bumping unintentionally into obstacles, hurting themselves or the people around them. Moreover, over the last half century, the UK has seen an increase in life expectancy and this trend is expected to continue [3] . To meet the needs of an ageing society, research in innovative assistive robotic technologies has the potential to empower healthy aged people towards living an independent and active life, while keeping them safe and socially connected [4, 5] ; thus increasing their self-esteem. The beneficiaries of such technology account for 58% of the people aged 60 and over in UK who suffer from mobility difficulties [6] . In 2014 there were 14.9 million people aged 60+ and their number is expected to grow to 18.5 million by 2025 [7] . In a new world reshaped by the current pandemic, novel control solutions may: (i) make life easier for these categories of people who need flexibility; (ii) help health organizations like the National Health Service (NHS) in the UK to overcome issues like limited staff resources. Assist-as-needed [8] or shared control [9] is a concept involving collaboration between a human and a machine. The human operator informs the machine about their objective (e.g. by actuating a joystick) and the machine interprets that action, then implements it in an optimal way. The algorithms studied in the literature can be divided in two categories, depending on the need to model system dynamics. On the one hand there are the black-box control solutions like the Probabilistic Shared Control [10] , optimal control in [11, 12] , and Dynamic Window Approach [13] , task-oriented control [5] . Arguably the most interesting feature of these algorithms resides in their generic value: they are most suitable in situations where models of the wheelchair dynamics or the user intention do not exist, and can easily be switched (adapted) from one vehicle type to another one. The second category of algorithms rely on white-box control solutions that require model identification; they are analyzed next. Moving vehicles obey physical laws which are well-known and understood (e.g. inertia, gravity, etc.). Consequently, it is natural to create a model integrating the capabilities of the machine. Although models in general are a mere reflection of reality, control-oriented models permit taking into account the uncertainties from the early phase of designing the control. The challenge is to be able to derive control policies based on the control-oriented model and the limited unpredictable information coming from: (i) the sensors observing the environment (e.g. low-cost time-of-flight sensors that can measure only the 1D distance to obstacles whereas their 3D shape remains unknown), and (ii) the human user (e.g. the joystick interface is a projection of what the user thinks, and their intention). In what follows, we shall make the distinction between the availability of significant amount of sensor data (e.g. measurements at high rate) and the limited information provided by each sensor (e.g. 1D distance to an obstacle is not sufficient to know the precise location of an obstacle in 2D Cartesian space; 2D joystick data is only informative about the immediate intention of the driver, not the long-term intention like getting closer to a table in order to serve their lunch). To compensate for this limited information available via online measurements, recent state-of-the-art research uses stochastic models. Taking into account the intention of the user is essential. An assistive machine should complement their intention in such a way that the user feels empowered and can make use of their existing functionality (e.g. physical, cognitive, visual, hearing, etc.) [14] . Contrary to able-bodied people who might be more in favour of autonomy (think of Tesla cars driving autonomously), the potential users of semi-autonomous assistas-needed technologies want to maximise their level of control, and thus develop new skills. They favour technology that empowers the user rather than further disabling them by taking away control authority and treating them like a precious piece of cargo. Therefore, instead of relying on generic control algorithms intended to satisfy all users, we shall put forward a method that can be tailored to a specific (individual) user. Each wheelchair user has their own style of driving and any particular assist-as-needed algorithm that works for one user might not necessarily satisfy the needs of a second user. For example, an expert wheelchair user might prefer to cut corners at maximum speed, whereas another driver might prefer a different, more cautious strategy. If we ask them to repeat the same operation over and over, their driving style can be characterized using probabilities. This observation motivates the use of multiple stochastic user models (see later on section 2.2). In view of making bespoke informed decisions, the SDP framework is generous enough as to allow taking into account stochastic uncertainties (namely driver models) from the early phase of designing the control. SDP is a control design technique that can handle stochastic models by solving a global optimization problem under constraints. The solution is not analytical, instead it is computed numerically, often on high-performance computers able to handle the burden of intensive computations and fairly large matrices, and comes in the form of optimal control policies [15] . These operations are carried out offline and the outcome, namely a lookup table containing the optimal actions, is implemented online on hardware with limited computational resources (e.g. industrial controllers or single-board computers). For this reason, SDP is particular appealing to the automotive sector [16] . One significant limitation of SDP is the well-known curse of dimensionality (exponential increase of the required computation with each new state, control variable or uncertainty) [15] . To summarize, SDP does not scale well to problems involving many variables where exact solutions become computationally intractable [17] . To mitigate its effect: (i) compact-sized problems (e.g. system dynamics with no more than two state variables) are preferred over larger ones; (ii) instead of relying on the naive implementation of Bellman's principle, faster algorithms like the policy iteration can be used. In this article we address the safe-driving problem of a semi-autonomous vehicle (a power wheelchair) in an environment with static obstacles, as depicted in Fig 1. This is still an active area of research [12] , unlike the purely autonomous vehicles case, which is out-of-scope for the purpose of this article and where robust and well-established path planning algorithms are readily available [18] , [19, §11.1.5] , [20] . In this article we propose a novel shared control algorithm based on policy iteration. To the best of our knowledge, policy iteration has not been proposed yet in the scientific literature for addressing the problem stated in section 1.6. It is our intention to make a significant contribution in this setting as follows. Inspired from the automotive sector, where a rather limited number of successful implementations of policy iteration algorithms have been published (see, e.g. [21, 22] ), this article builds upon the seminal work [23] . Our article is similar in the sense that we shall reuse the same idea of implementing policy iteration to compute global optimal policies in conjunction with stochastic models. However, our contribution differs with respect to the following points: (i) the application is different (a power wheelchair, not a passenger car); (ii) the optimization is carried out in terms of total time to complete the task (obstacle avoidance), not energy (e.g. minimize fuel consumption versus battery usage in hybrid electric vehicles [23, 24, 22, 16] ); (iii) the framework is different (control of semi-autonomous vehicles, not autonomous vehicles); (iv) the stochastic model is different (user intention, not the driving cycle); (v) environment awareness is ensured online via time-of-flight sensors able to detect obstacles, whereas other means are necessary to identify (classify) driving cycles. The model-based control solution we propose can be tailored to reach a high level of customization (personalisation) in terms of a specific driving style of any individual user. Among other candidates able to solve the problem stated in section 1.6, reinforcement learning was excluded for the following reasons. Although it has the advantages of using a theoretical framework that is more general in contract to optimal control theory [25] , and the ability to learn complex nonlinear behaviors [26] , there are other properties which seem detrimental to our problem. In particular, it relies on the exploitation versus exploration paradigm which is not suited for safe-critical applications; it requires a prohibitively large number of training iterations before getting to an effective (operational) result [27] . In this section we formulate an infinite horizon time-optimal stochastic shortest path problem [15, §7.2] . This is a finite-state problem involving a finite (but rather large) state space. The framework was largely inspired by the the inventory control problem in [15, Ex. 1.3.2, p. 28]. We start by presenting the modeling of the vehicle dynamics followed by the driver models, then set forth the control design. The aim is to compute a map (or lookup table) that is informative about the expected time to hit an obstacle for each particular stochastic driver model. We will use this information in the next section 3 to elaborate an assist-as-needed semi-autonomous control algorithm. Wheelchair models are known in the literature [28] . Here, we focus on the dynamics of the instrumented wheelchair from Fig. 2 , which are governed by the following system equations [29] : All variables and parameters are defined in Table 1 together with their units of measure; subscript k indicates the time index. The physical meaning of the constraint (2) is to avoid requesting an input velocity v d,k + v u,k that is more than the wheelchair's power module profile is set to handle. The physical meaning of the saturation in: • eq. (1a) is that we are concerned with vehicle motion towards the nearest obstacle located in the field-ofview of a sensor (e.g. an ultrasonic or time-of-flight sensor). • eq. (1b) is that each motor servo-drive (power module) includes a low-level safety functionality which bypasses (overrides) any requested control action beyond v max > 0 and below v min < 0. physically-inspired experimentally identified parameters [30] α b field of view (sector area) of sensors detecting obstacles in Fig. 1 rad Consistent with notations in [31] , superscripts "0" and "b" in Table 1 indicate that that quantity was expressed in coordinates of the inertial frame and base frame, respectively. Subscripts indicate the object they refer to, e.g. "o" stands for obstacle. In Fig. 1 , o 0 x 0 y 0 represents the fixed (inertial) frame and o b x b y b is the moving (base) frame. The latter has its origin at midpoint between the two main wheels, and the x b -axis points in the direction of the vehicle advancing forward. In this work, we define three generic stochastic user models, which we call the expert driver, the blind driver and the naughty child, depicted in Fig. 3 . The naughty child has tendency to accelerate towards the obstacle (with the intention to hit it as means to explore the world), whereas the expert driver carefully maneuvers the vehicle as to avoid getting in the situation of hitting the obstacle; finally, the blind driver is unable to acknowledge the presence of the obstacle and makes no difference in terms of driving style whether the obstacle is present or not (thus we used an uniform distribution). We end this section by mentioning that fine tuning or learning these driver models is out-of-scope and will be subject to future research. Inspired by [32] , we intend to use Gaussian processes -a powerful tool in machine learning. Fig. 1 ). Later on, in section 3 we present an extension towards the more general case of a vehicle able to detect multiple obstacles on a sector area in front of it (α b > 0 in Fig. 1 ). For each stochastic driver model, we want to know the minimum expected time to termination, defined as the time elapsed from the initial state of the moving vehicle (x, v), where notations x x k=0 and v v k=0 correspond to the initial (or starting) instant, to the moment where it stops safely near that obstacle, without hitting it. In other words, we address the following optimization problem: along trajectories defined by (1) and such that constraint (2) holds; k = 1, .., ∞ is the number of stages. In (3), E is the expected value operator expressing a sum of all possible (feasible) trajectories that are all weighted together by their probability of occurrence associated to uncertainties v d,k . The policy iteration algorithm that implements the cost (3) will compute that control policy which works best for all feasible trajectories (in the sense of a weighted sum). However, the dominant trajectories that have higher weights will account the most in the process of computing the optimal control policy. In (3), g(·, ·) is the cost per stage, to be defined next. 2.3.1. Cost per stage without penalty By choosing g(·, ·) ≡ ∆t in (3), as expected with solving time-optimal problems, the control policy comes in the form of a bang-bang control [33] . It represents an aggressive maneuver of that specific stochastic driver model advancing towards the obstacle, but still being able to stop safely near the obstacle, without bumping into it. An interesting fact of this formulation is that the total cost (3) has a physical meaning: the map in Fig. 5a shows the time measured in seconds for executing this maneuver by the 3 stochastic driver models. The interpretation of results in Fig. 5a follows. Note the descending slope as the vehicle approaches the terminal state. Comparing the 3 drivers' results, we see that, as expected, it takes more time in average for the blind driver to reach the terminal state compared to the other driver models. The naughty child outperforms the other two drivers in reaching the terminal state due to the tendency of intentionally wanting to hit the obstacle. The expert driver 's results are located in-between the other two users. In Fig. 6 we study the convergence of the policy iteration algorithm. At each iteration k, we computed the average of the map of the total cost (examples of such maps are illustrated in Fig. 5a ). It can be seen that after k = 4 iterations, the slope of the convergence curve is almost zero and the policy iteration algorithm can be stopped. The policy iteration algorithm requires a finite number of states. The saturations defined in (1) motivate the usage of bounds for the (x, v)-grid (see the axes of Fig. 5a) ; the constraint (2) suggests bounds for the (v d , v u )-grid. By carefully investigating the resulting grid, a problem arises in two situations: (i) x = 0 and v < 0; (ii) x = d max and v > 0. In both cases, what happens physically is that the vehicle would normally advance outside of the x-grid. To cope with this situation, we defined an environment similar to a toy circuit where a toy vehicle bumps into damper walls physically limiting its ability to advance any further (beyond the boundaries of the circuit) and also protecting the toy from damage. Next, we show how to remove this artificial limitation by introducing soft constraints. By choosing with β 0 a penalty factor, the cost function in (3) changes both its mean value as well as the shape: see Fig. 5b . First, we shall analyze the shape. Interestingly, we can now easily identify states where the vehicle would crash into the obstacle (the area around x = d max and v > 0) or would advance backwards beyond the range of sight of the sensors (x = 0 and v < 0): they correspond to regions where the cost function surges in Fig. 5b . From this we can conclude that the state-space domain can be conceptually sectored as depicted in Fig. 7 . Moreover, it makes sense to think of designing a physically inspired assist-as-needed semi-autonomous control (ASC) within the white sector area. Outside this area, safety measures need to be enforced (e.g. by taking authority from the user, active braking, etc.) which is a topic out-of-scope for this article. In the following, we call the white area in Fig. 7 the feasibility region where ASC may be enabled. Second, we shall analyze the z-axis values of the maps in Fig. 5b compared to their counterpart in Fig. 5a . Unfortunately, in view of designing a physically inspired ASC, we cannot make use of the values of the maps in Fig. 5b within the feasibility region: a drawback of having introduced the penalty in (4) is that the mean values of the maps in Fig. 5b went up compared to the results in Fig. 5a , to the extent of losing their physical meaning. To quantify this in an intuitive way, we shall consider the following simple calculation of a vehicle advancing at v max = 0.54 m/sec driven by a deterministic user (v d ≡ v max ), unsupervised (v u ≡ 0): it covers the distance from 0 to d max = 2.83 m in about d max /v max ≈ 5.24 sec, whereas values in Fig. 5b are at least two times higher than that. (a) without additional penalty (b) with additional penalty Figure 5 : Results of running the policy iteration algorithm with or without additional penalty on the cost per stage g(·): the total cost represents the minimum expected time to reach the terminal state (in red), for three stochastic driver models: expert driver (in green), naughty child (in blue), blind driver (in gray) We end this section by mentioning that the cost maps in Fig. 5 could have been computed using the less computationally-intensive value iteration algorithm [15, §7.2] . Instead, we chose to implement a different algorithm: although we did not exploit one of the main benefits of policy iteration, which is having access to the computed optimal control actions, this might change in future implementations as a way to quantify how different these actions are with respect to the generic driver model. In this article we propose to design the ASC as follows. We start by setting up an arbitrary threshold representing time to execute a safe stop maneuver for each stochastic driver model (see the plane in Fig. 8a ). Next, we shall make use of the associated map from Fig. 5a within the feasibility region of Fig. 7 (the white sector area). The projection onto the state space of the intersection between this plane and the map, gives a new boundary (see the line segment AC in Fig. 8b) to the region where the intention is to enable ASC (schematically depicted by the triangle ABC in Fig. 8b) . Motivated by the observation that v = v d + v u in steady-state in (1a), we are now ready to present the guiding principle for driving the vehicle assisted, by modifying the user input data (the joystick). Say, the red solid circle in Fig. 8b illustrates the current condition of the vehicle, expressed in terms of actual velocity v and distance x to an obstacle. Because it is located in the white area where we decide to enable ASC, the driver's actual demand v d will be reduced by properly adjusting v u to a value corresponding to the green solid circle. In other words, the green solid circle corresponds to v d + v u . Reiterating this operation at every time instant (sampling time), the vehicle's actual velocity v will eventually tend towards the boundary of the white area (the line segment AC in Fig. 8b) . Gradually, the vehicle converges safely towards the terminal state. Before moving to the next section we would like to mention that generating each map in Fig. 5 required significant computation power, both in terms of running time (about 24 hours of continuous operation on an i7-8750H CPU workstation laptop) and memory (we used all the 32GB RAM available). Possible ways to decrease the running time are suggested in the literature, e.g. in [25] by training a Gaussian process (GP) to learn the continuous total cost function by using a coarser state-space grid compared to what we used to compute Fig. 5 ; another GP-based approximate DP is presented in [34] ; we leave the implementation of these ideas as future work. Next, we shall generalize our results to the more general case depicted in Fig. 1 of a vehicle advancing not only in a straight line, but in 2D Cartesian space, with multiple objects in front of it. The same principles put forward in the previous section scale up to the general case of a vehicle able to sense the location of obstacles within a sector area α b > 0 in Fig. 1 as follows. A first attempt would be to extend the system dynamics (1)-(2) with additional state variables, then run offline the policy iteration algorithm to generate maps that should provide information about the minimum expected time to advance safely towards each obstacle. The problem with this approach is the curse of dimensionality mentioned in the Introduction, making it intractable both in terms of computation power (probably in the order of months of continuous CPU operation) as well as memory RAM (in the order of terabytes). For this reason we propose to use a second approach where the idea is to reuse the results of the previous section in conjunction with the following assumptions (approximations): • the time-optimal trajectory of a vehicle advancing towards an obstacle can be approximated using a polynomial (see Fig. 9 ); • in 2D Cartesian space, the vehicle dynamics on the time-optimal trajectory towards an obstacle can be approximated using the straight-line motion in 1D Cartesian space; in other words there are weak coupling effects between the longitudinal and the lateral motion dynamics of the vehicle. The power wheelchair is actuated via a standard wheelchair joystick. Its position (location) on the joystick plane is in a linear one-to-one relation to the output steady-state velocity of the vehicle. This is schematically illustrated in Fig. 10 where the position of the joystick (the red solid circle) can be mapped on the joystick velocities (v d , ω d )-plane. This motivates the analysis on the (v d , ω d )-plane hereafter. Say, the red solid circle in Fig. 10 corresponds to the actual user demand. If the current vehicle velocity v falls within the ABC triangle of Fig. 8b , then the supervisory control (SC) needs to reduce it towards a safe value as explained in the previous section. By applying the same reduction ratio on both the v d -axis as well as on the ω d -axis, will lead us to the green solid circle in Fig. 8b , which: (i) is safe from an obstacle avoidance point of view; (ii) represents (v d + v u , ω d + ω u ) sent as a request to the power module (servo-drives) that actuate the vehicle's motors. Note that all this analysis is straightforward in steady-state conditions where v d ≡ v and ω d ≡ ω; the transitory effects still need to tested in practice. Figure 10 : Applying ASC to joystick data: the unsafe user's demand (depicted by the red solid circle on the actual joystick -the left image, and on the joystick plane -the right image) is reduced linearly to a safe value (depicted by the green solid circle) before being sent as a request to the power module An interesting fact of this semi-active (dampening) control design is that in all situations it will not contradict the user's intention (e.g. cases where the driver demands to advance to the left whereas the SC would make the vehicle turn right would not occur). This is due to the linear relation between the red solid circle and the green solid circle, both located on the red dashed line in Fig. 10 . Such conflicting situations were analysed theoretically in [35] and this paper provides a practical solution to that. To end with, we provide a sample C++ ROS code of ASC that shall be used in the next section to carry out experiments. The interested reader can access it on https://github.com/UCL-Aspire-Create/whc pub ASC policy iter.git To validate the proposed ASC, we created the circuit in Fig. 11a with obstacles made of harmless cardboard boxes. We recruited four participants to this study, all able-bodied healthy people, fully trained researchers with experience in driving the wheelchair. From this point of view, their performance is closer to the expert driver from the three stochastic driver models studied in this work. Each participant was asked to execute the following operations: start driving the wheelchair from an initial position located to the left-hand side of Fig. 11a , then drive between the obstacles, next pick one sticker attached to the cardboard box located to the right-hand side of Fig. 11a , after that drive back between the obstacles to the starting point, next place the sticker in a basket and continue reiterating all these steps until all four blue stickers are placed inside the basket. Drivers are allowed to advance forward only (not backwards). This study is organized as a competition (contest) between these participants: the winner is the person who can drive in minimum time and with the least number of collisions. Say t d is the time to complete the track and c d is the number of collision, then each participant receives a score s representing a penalized task completion time measured in seconds, based on their performance: where α > 0 acts as a penalty for hitting obstacles. Inspired from circuit racing [36] , we use α = 5 sec. The aim of this study was to assess whether the ASC brings any benefit compared to: (i) the situation with no assistance (NA): this is used as the control group -terminology according to the statistics community; (c) offline 2D simulation in Matlab (view from above); note the obstacles illustrated in orange Figure 11 : The circuit used for testing the obstacle-avoidance algorithms. A video recording is available on https://youtu.be/X-dQoLpYquQ (ii) a baseline assist-as-needed semi-autonomous control: here we shall use our previous rule-based (RB) algorithm [37] . Hence, we address the research question: how does the driving condition (NA, RB, ASC) influence the user performance? We make a first hypothesis that there is a difference between all these three driving conditions (NA, RB and ASC), and second that ASC is superior to RB. In order to reduce order effects (structural bias), the order of the driving conditions (NA, RB, ASC) is assigned randomly for each participant using simple random sampling [38, §3] . An ethics approval was granted for this study by UCL Research Ethics Committee (ref. 6860/011). The interested reader can access a video recording on https://youtu.be/X-dQoLpYquQ. Table 2 illustrates the data collected: each row represents a single participant's data; each column corresponds to one of the three driving conditions under consideration (NA, RB and ASC). The experimental data in Table 2 converted into scores used for this study can be visualized in Fig. 12 . In order to compare scores of the 3 aforementioned groups, we shall use ANOVA. Before deciding which ANOVA variant to use, first we checked the assumption of parametric data. Only the group NA meets this assumption (Shapiro-Wilk p = 0.836, skewness ρ = 0.724 and no outliers), whereas the other two groups violate this assumption: RB (Shapiro-Wilk p = 0.003, skewness ρ = 2 and no outliers); ASC (Shapiro-Wilk p = 0.125, skewness ρ = 1.7 and no outliers). This motivates the use of the One Way Repeated Measures Friedman's ANOVA which is a non-parametric alternative to the more popular parametric F-test. Hence, as the data violated the parametric assumption, a Friedman's ANOVA -a within participants design [39] , was conducted to test whether the driving condition (NA, RB or ASC) affects the participants score in completing the track. The analysis revealed a significant main effect of driving condition (χ 2 (2) = 8, p = 0.018). The post-hoc analysis using pairwise comparisons (Durbin-Conover) revealed a significant difference (p < .001) between pairs of groups: (i) the NA (mean=115, σ = 11.7) and RB (mean=151, σ = 17.3); (ii) the NA and ASC (mean=137, σ = 7.37); (iii) the RB and ASC. This suggests that ASC is superior to RB although both are outperformed by NC. First, increasing or decreasing the penalty α on hitting obstacles, will have an immediate effect on the scores and thus the entire statistical analysis. For instance, inspired by and consistent with circuit racing practice [36] , another option would be α = 10 sec. Alternatively, more general and elaborate methods to adjust the weighting factor α are discussed in [40] . Second, due to the ongoing coronavirus pandemic, we had only been able to recruit a rather limited number of participants to this study. Although results are statistically significant, these should be confirmed in a follow-up study involving more participants. The statistical analysis suggested that ASC outperforms RB, which is quite a positive result because it shows a progress in our research. However, the scope of this result is limited to a particular circuit and does not account for other real-world scenarios (e.g. entering an elevator, advancing on a wide corridor with static obstacles, etc.) -see the standardized circuits in [1] . Follow-up studies are necessary to address these situations, including avoiding dynamic obstacles like pedestrians walking [41] . The rather intriguing fact that by disabling assistive control (NA group) the participants to this study obtained better scores compared to enabling assistive control (RB and ASC) can be explained by multiple factors. First, by the nature of these control solutions: both gradually take authority from the driver as the risk of collision increases. This translates into reduced effective velocities and thus the time to complete the circuit increases. Second, the participants to this study were experienced drivers (although not experts). Therefore they handled rather easily the obstacle avoidance task. Third, participants were all able-bodied whereas we expect assistive control to prove its real-world usefulness for people with known impairments (e.g. cognitive, visual, hearing, physical, etc.) [1] . To summarize, there is a trade-off between the risk of collision and the ability to finish fast the circuit: the price to pay for reducing the risk of collision when enabling assistive control is that it takes more time in average to complete the circuit. In essence, from a broad perspective, our contribution has the potential to transform the lives of many people by creating a transportation system that empowers the user by means of technology to carry out safely everyday tasks (without bumping unintentionally into static obstacles). Participants to this study appreciated the smoothness of ASC. However, the ASC might exhibit a more sudden (bumpy) behavior. This can be due to multiple factors: (i) the maps generated by the stochastic driver models used (recall Fig. 5) ; (ii) this control design is based on the steady-state assumption that v = v d + v u (see section 3), thus ignoring transitory effects. These aspects need to be investigated in future work. Our statistical analysis used a combined metric (a score), by weighting together the task completion time and the number of collisions. Instead, if we rerun the statistical analysis for each separate metric, the following results are obtained. First, the analysis using the task completion time revealed a similar (analogous) result: a significant effect of driving condition (χ 2 (2) = 8, p = 0.018). Second, the analysis using the number of collisions showed a non-significant effect of diving condition (χ 2 (2) = 3, p = 0.223), meaning we cannot conclude that ASC is superior to NA or RB: the fact that no collision occurred for ASC group in Table 2 , contrary to the other groups, might be the result of a lucky sample; we need more participants to the study in order to obtain a statistically significant result in this setting. All the sensors mounted on the research platform in Fig. 2 are responsible for building environment awareness capability for the robotic vehicle. For testing ASC we used: (i) the standard joystick to capture user's input (v d , ω d ), (ii) the wheel encoders to estimate the actual velocity v, and (iii) the front-facing ultrasonic sensors to detect the nearby obstacles (see Fig. 2b ). Although not considered here, the alternative to use time-of-flight sensors placed under the footplate (see Fig. 2a and [42] ) is subject to future research. The software architecture relies on a single-board computer (a Raspberry Pi 3b+) running Robot Operating System (ROS) on Ubuntu 16.04. The simulators in Figs. 11b and 11c were used for visualization purpose. Although not considered here, these simulators can be extended into virtual reality environments making use of digital twins [43] , which is subject to future research. In this article we developed and tested an assist-as-needed algorithm (we call it AssistMe) that provides a robotic vehicle with the intelligence (or capability) to avoid obstacles by collaborating with the user and thus ensuring a safe driving experience. We show how this algorithm can be personalized to meet the needs of each specific driver, based on their stochastic user model (we called them the expert driver, the blind driver and the naughty child); it requires low-cost hardware components, e.g. ultrasonic sensors, wheel encoders, a single-board computer, etc. The algorithm makes use of pre-computed optimal maps of the average time to hit an obstacle located ahead of the vehicle: we formulated a time-optimal stochastic shortest path problem and solved it numerically by implementing a computationally tractable algorithm relying on policy iteration. An experimental study with healthy participants, showed an improved score of this assist-as-needed algorithm over a baseline rule-based control. SWADAPT1: Assessment of an electric wheelchair-driving robotic module in standarized circuits: A prospective, controlled, and randomized pilot study Improving wheelchair services Meeting the challenges of an ageing population Power wheelchair navigation assistance using wearable vibrotactile haptics An ecosystem for heterogeneous robotic assistants in caregiving Later life in the uk factsheet 2019 Improving care for older people Assist-as-needed policy for movement therapy using teleroboticsmediated therapist supervision A topology of shared control systems -finding common ground in diversity Probabilistic vs linear blending approaches to shared control for wheelchair driving Low complex sensor-based shared control for power wheelchair navigation A semi-autonomous framework for human-aware and user intention driven wheelchair mobility assistance The dynamic window approach to collision avoidance Who's intelligent? wheelchair, driver or both? Dynamic programming and optimal control An ECMS-based powertrain control of a parallel hybrid electric forklift Learning-based model predictive control: Toward safe learning in control Creating autonomous vehicle systems MPC-based control architecture of an autonomous wheelchair for indoor environments Assessing the potential of predictive control for hybrid vehicle powertrains using stochastic dynamic programming Control strategy for an electromechanical transmission vehicle based on a double markov process A stochastic control strategy for hybrid electric vehicles Stochastic policies for online computation triggering in powertrain control Approximate dynamic programming with Gaussian processes Reinforcement learning: an introduction Learning-based nonlinear model predictive control to improve vision-based mobile robot path tracking Robust observer-based tracking control under actuator constraints for power-assisted wheelchairs A stochastic control strategy for safely driving a powered wheelchair Model-based sensor fusion and filtering for localization of a semiautonomous robotic vehicle Learning references with gaussian processes in model predictive control applied to robot assisted surgery Commande et Optimisation de Systèmes Dynamiques,Èditions de l'Ècole Polytechnique Approximate dynamic programming with Gaussian processes for optimal control of continuous-time nonlinear systems Assistive planning in complex, dynamic environments: a probabilistic approach International Automobile Federation, Formula 1 -Sporting Regulations 2021 A rule-based assistive control algorithm for safe navigation for a powered wheelchair Statistics: A Biomedical Introduction Learning statistics with jamovi: a tutorial for psychology students and other beginners How to score questionnaires From hri to cri: Crowd robot interaction -understanding the effect of robots on crowd motion A table docking and negative obstacle detection system for a smart wheelchair using tof sensors Progress in Industrial Mathematics: Success Stories We would like to thank George Walker for developing the Unity environment in Fig. 11b ; Marie Babel and her team (François Pasteau, Louise Devigne) at INSA and IRISA -Rennes for providing us with an RNET controller used to: (i) publish standard joystick data on a ROS topic, and (ii) send virtual joystick messages on the RNET bus to the power module. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. This work is funded by the project INTERREG VA FMA ADAPT -"Assistive Devices for empowering disAbled People through robotic Technologies" http://adapt-project.com/index.php. The FMA Program is a European Territorial Cooperation program which aims to finance ambitious cooperation projects in the border region between France and England. The Program is supported by the European Regional Development Fund (FEDER).