key: cord-0776051-zrhpyeax authors: Takumi, Ishikawa; Sumito, Nagasawa title: SMA micro-hand implemented in small robot for generating gestures date: 2021-04-22 journal: Intell Serv Robot DOI: 10.1007/s11370-021-00364-9 sha: 3669ba48b0ca1c7f0feebcc652957f646f2b8849 doc_id: 776051 cord_uid: zrhpyeax Research on robots that can be used for communication with humans has become popular in recent years. Communication robots should ideally be as small as an infant in order to reduce the user’s feeling of threat. In addition, non-verbal communication (such as gestures) is also important in facilitating smooth interactions between humans and robots. There are currently a few communication robots that are small sized and can generate hand gestures. In this paper, we propose a small robot hand, which is optimized for gesture communication by using a shape memory alloy (SMA). The SMA employed is a Ti–Ni alloy, which is used as an actuator. The SMA shrinks when it transforms into the austenite phase at temperatures higher than the transformation temperature. When it is in the martensitic phase at a lower temperature, it is expanded by an external force. Each finger of the robot hand is driven by an individual SMA wire. The specifications of the small robot including the hand size, operation angles in each finger joint, response times and power consumption were determined according to the human finger and existing small communication robots. These required specifications have been fulfilled by carefully designing the geometry and heating/cooling power control. A questionnaire-based survey was also conducted with a robot hand. The five-finger hand was successfully shown to generate recognizable symbolic gestures. SUPPLEMENTARY INFORMATION: The online version supplementary material available at 10.1007/s11370-021-00364-9. In recent years, research on robots for communication with humans has become very active [1] [2] [3] . The main purpose of these robots is to serve and live with them as human partners, assistants, or companions. Service robots are beginning to become part of their jobs in many areas, including shopping, education and companions. When deploying robots in pub-tion communicated verbally. Non-verbal communication is powerful in communicating the emotions and attitudes of the speaker and complementing linguistic communication [11, 12] . This might be the same in the case of communications between humans and robots. Earlier studies have reported that communication between humans and robots becomes better using gesturing robots than using the ones that do not use gestures [13] [14] [15] . There are many previous research studies on robot hands; however, the robots considered in these studies have hands almost the size of human hands, and the hands are only meant for grasping objects. There are few studies concerning small robot hands having operational fingers. One of the smallest developed robot hands is used in laparoscopic surgery [16] . However, the driving mechanism, the controller and the base of the power source for such robot hands are too large to be mounted on a small robot (Fig. 1) . Therefore, if an existing five-finger small robot hand is mounted on a robot, the large base unit will impair its mobility and appearance. The new proposal of this paper is to develop a gesture robot hand that fits in the forearm of a small robot, and to maintain mobility and give the small robot a finger drive for gestures. Another small robot hand, the SMA robot hand that is 1/3 the size of a human, has been reported [17] . This robot hand has 20 degrees of freedom for gripping small parts. The power consumption of the SMA robot hand is not evaluated in this paper. It is necessary to quantitatively evaluate the power consumption for the purpose of mounting on a small robot. Hands with five fingers can express considerable amount of information [18, 19] . For instance, symbolic motions that are substituted for specific words like the V-sign, or instructional motions that express something, such as using the index finger for pointing directions. Many researchers are trying to classify the various gestures that accompany utterances. Nehaniv et al. classified five gesture types in order to infer the intent of gesture. These five classes are irrelevant gestures, side effect of expressive behavior, symbolic gestures, interactional gestures and referential gestures [20] . Finger gestures are mainly included in symbolic gestures. According to Nehaniv et al, examples of symbolic gestures are peace signs, salutes and more. Therefore, small robot hands with five fingers are necessary for the robots to be able to communicate effectively with the users. The purpose of this study is the design, fabrication and demonstration of a small robot capable of using hand gestures. As shown in Fig. 2 , there are four requirements for the design of our robot hand. The first is that the robot hand size, comprising the driving mechanism, controller and the power source, must be small enough to be attached to a small robot. Secondly, the operational angles of the finger joints should be wide enough for a variety of gestures. Thirdly, it should evoke responses in a short time to preempt giving unnatural impression to the user. The last is that the power consumption has to be small enough to be implemented in a small robot. Our robot hand is assumed to be implemented in a small robot whose height is about H r = 400 mm. The length from the tip of the middle finger to the wrist L r is determined by referring to the human body dimension data L h and the ratio of the robot height to human height. The representative dimension of the robot hand is calculated using the following equation: [21] , then, L r is 45 mm, as calculated from Eq. (1). In this study, our robot hand is designed to satisfy the dimension L r . For gesture expression, our fingers are appropriate to operate about 90 % of the ranges of the human finger joints. Referring to the range of human finger joints, the following operational angles are required: the metacarpo-phalangeal (MP) joint angle θ MP = 80 • , the proximal inter-phalangeal (PIP) joint angle θ PIP = 90 • and the distal inter-phalangeal (DIP) joint angle θ DIP = 70 • (Fig. 3) . The system reaction time limit, generally allowed by the user, is known to be 2 s [22] [23] [24] . This is widely used in the design of various human interfaces such as the GUI (Graphical User Interface), television and computers. For this reason, the required response time of the finger of our robot hand is set to be less than 2 s, during the extending and the bending of the fingers. The required power consumption was determined considering the specifications of the small communication robot PALRO from FUJISOFT INCORPORATED [25] . When PALRO is walking, the power consumption is at its maximum of 60 W, and during conversations, the power consumption is 30 W. So, there is a 30 W margin for driving our fingers. Therefore, the required specification for the power consumption of our hand is less than 15 W for each hand. The required specifications of the drive angle, response time and power consumption are evaluated by two experiments. The drive angle and response time are measured by image analysis, by attaching markers at the tip of the robot's finger and at each joint. The power consumption is evaluated by measuring the drive current and the voltage of our robot hand. Shape memory alloy (SMA) was used in the actuators of our robot hand. Even if the SMA is deformed into an arbi-trary shape, it will recover to its original shape when heated above its own temperature. Large strain remains in SMA in the low temperature range of T ≤ 298 K, but the strain disappears when heated above the Austenite phase transformation end temperature. The shape memory effect appears in this temperature range. This shape memory effect is caused by a phase transformation from the austenite phase to the martensite phase or from the martensite phase to the matrix phase. NiTi alloys and copper-based and some iron-based alloys have been put into practical use. NiTi alloy is excellent in shape recovery characteristics and repeatability. It is effective for SMA to use both the temperature sensing function and the actuator function at the same time. The practical application of SMA has progressed in this way, and it will be applied as a sensor and actuator to home appliances, automobiles and trains. The SMA has been used for small robots since the early 1990s [26] [27] [28] . The first important feature of the SMA is miniaturization. The ratio of output power to the weight of the SMA is quite high. Therefore, the SMA is suitable for miniaturization and in the actuators of small robot hands. The second important feature is smooth movement, as the movement of the SMA is similar to that of a living thing. Therefore, it is considered suitable for building robot legs and arms actuators that move flexibly. Biometal fiber (Toki Corporation, BMF) is a wire-like SMA that contracts and stretches like a muscle. When its temperature is below the transition temperature, it is soft like a nylon thread. However, it becomes hard when a current is applied, and its temperature rises above the transformation temperature, at which it shrinks with a strong force. It softens again when the current is stopped. The operating life of the BMF is long, and the BMF100 which a type of biometal fiber, has a diameter of 100 m, can be moved a 100 million times or more under a load stress of 100 MPa. Furthermore, BMF has a better response than the other SMA actuators, and the BMF100 can react at 3 Hz in a windless room temperature (20 • C) [29] . Table 1 shows the main specifications of BMF100 [30] . In this study, BMF100 with a diameter of 0.1 mm was used as an actuator. The robot hand was designed for gestures, so one aim was for it to make gestures such as V-signs. Anatomically, a human hand has about 20 degrees of freedom. Owing to the difficulties involved in formulating a finger of the desired small size with multiple degrees of freedom, each finger was designed to only bend and extend. Each finger, excluding the thumb had four links, i.e., equivalent to the middle phalanx, the proximal phalanx, the distal phalanx and the body of the phalanges. Each joint, i.e., the MP joint, PIP joint and DIP joint has one degree of freedom of bending. In order to achieve gestures such as the V-sign or thumbsup, the bending order of the three joints is not important. Our robot finger was designed in such a way that about the three joints were bent by one BMF. To obtain enough SMA deformation for bending the robot finger sufficiently using one BMF, it is necessary for the wire length to be longer than the robot hand. Therefore, the BMF is mounted at a point near the elbow in the forearm from which it is possible to achieve the requirements of the drive angles of the finger joints. The bending/extending mechanism of the robot hand was based on the human tendon drive structure [31] . The BMF for bending was placed on the palm side of the finger, and the bias spring and the connecting wire for extending were placed on the dorsal side. As shown in Fig. 4 , each hinge of the joint was formed by a convex and a concave that were arc-shaped and joined. The hinges were held with the tension of the extending wire and the BMF passing through each phalanx. Due to this mechanism, the hinges were constructed without any shafts, and this mechanism was also helpful in reducing the size of the finger. The drive mechanism was designed with a one-way active bending mechanism [32] . A large bending angle can be obtained even with a small amount of displacement. The amount of BMF displacement depends on the positions of the holes (shown in Fig. 5 , the length of a and b), the BMF passes the holes(shown in Fig. 5 by red line). The bending angle increases as the distance between the point of action of the generated force by the SMA and the central axis of rotation of the finger becomes closer. Figure 5 shows the MP joint. The operation range of the MP joint θ max MP was designed to be up to 80 • , according to the required specifications. In order to design the operation range of the MP joint, the angle θ 2 shown in Fig. 5 is determined in Eq. (2) as follows: (2) θ 1 and θ 3 were designed as: θ 1 = 45 • and θ 3 = 55 • , so that θ 2 can change between 0 and 80 • , which corresponds to θ max MP . Therefore, when θ MP is 80, the distance c is zero and when θ MP is zero, the distance c is the BMF displacement required for the bending of the MP joint to θ max Similarly, the distances c at the MP, PIP and DIP joints were calculated as c MP , c PIP , c DIP . The BMF displacement δ is the sum of these distances as shown: The displacement of BMF δ was calculated as 10.00 mm by Eq. (4). The total length of the BMF was calculated as L = 500 mm from Eq. (5), because the strain of the BMF = 4%. Both the ends of the BMF are mounted at a point near the elbow in the forearm and turned back at the tip of the The specifications of the bias spring were calculated from Eq. (6). The spring tension has to be smaller than the generated force of the BMF shown in Table 1 For designing the spring tension, a significant consideration is needed, because when the tension is too weak, the response of the finger during extending becomes slow, and when its tension is too strong the response of the finger during bending becomes slow. Both responses, extending and bending, have to take less than 2 s. Both the ends of the BMF are mounted at a point near the elbow in the forearm and turned back at the tip of the finger. Therefore, the force of the BMF is twice the generated force of the BMF in Table 1 . The force to bend the finger by BMF is calculated as 1.372 N. P i = 0.265 N, k = 0.034 N mm, L i = 43 mm, L 0 = 21 mm, then, T spr is 1.353 N, as calculated from Eq. (6) . T spr is the spring tension, P i the initial tension, k the spring constant, L 0 the initial length L i the installation length, and δ the displacement of BMF. A finger prototype of the robot hand was fabricated. Figure 6 shows the prototype. The fingers were made by cutting acrylic resin with a CO 2 laser cutter. The operating angles and responses of the joints of this finger were measured by video analysis. A five-fingered robot hand was then fabricated and its power consumption was evaluated. Figure 7 shows the five-fingered robot hand. The dimensions of the robot hand from the tip of the finger to the wrist were L r = 44 mm, and the overall dimensions were 161 mm × 46 mm × 40 mm. The thumb was made from two joints and its driving direction was perpendicular to those of the other fingers. The video analysis using the ImageJ software was used to measure the response time and operation angles of the robot finger. The basic form of a measurement of angles and response time is described in Fig. 8 , and this experiment was performed to check whether the target requirements were satisfied. This experimental setup comprised a robot finger, a control circuit, an MCU, a power supply, a video camera and a processing PC. Markers were attached to the tips and each joint of the robot finger, and the relationship between time and the current joint angle was measured by image analysis using the video. The power feed of the SMA was changed by pulse width modulation (PWM). The BMF was energized for 15 s with 14 V, and the fingers were bent; then, the BMF was air-cooled for each extending joint, and a video was taken at 30 fps. By observing the LED changes from turning on to off in the video analysis, we can determine the switch timing from SMA heating to cooling. The LED is on when the SMA is heating and off when it is cooling. The LED flicker is not a problem because the LED blink frequency is 100 Hz and the video frame rate is 30fps. The ratio of the turn-on duration to the period used in the PWM control was defined as the duty ratio D [%] . The SMA actuators are often driven by a pulse width modulation (PWM) driver [33] . The input of the SMA wire is a duty ratio. The SMA wire is in the martensite state when the actuator is not driven with the duty ratio of zero. Assume that a temperature reaches at the steady state for a duty ratio of one. If is much higher than, overheating occurs and causes serious damage to the memorized shape. Moreover, higher values of increase the heating rate, but do not increase cooling rate. In this measurement, the SMA heating with D = 100% lasts for the first 3 s; then, the heating with a minimum duty ratio D = 45%, which can keep the robot finger bending, lasts for the subsequent 12 s. The positions of the markers O(x o , y o )A(x a , y a )B(x b , y b ) and C(x c , y c ) in Fig. 8 were measured using the image analysis software (ImageJ). The angles θ MP , θ PIP and θ DIP of each joint were calculated from the marker positions by Eqs. (7) (8) (9) . Regarding the response time, it was difficult to determine the finishing time of the finger with complete bend- ing/extending; this is because there is a slight fluctuation in the angle data, and the response becomes slow, when the BMF temperate is close to the room temperature (20 • C) . Therefore, the rising and the falling times were used as the evaluation indexes for the robot finger bending and extending. The time, from the start of the heating/cooling to the time when all three joints were bent/extended to 90% of the required operation angle, was defined as the bending time/the extending time. These were compared with the required specification of 2 s. The basic form of an evaluation of the power consumption is described in Fig. 9 . The objective is to evaluate whether the power consumption satisfies the requirement of 15 W. The experimental setup comprised a robot hand, a shunt resis- Fig. 9 Evaluation of the power consumption tance of 1 , the data logger Analog discovery 2 from the Digilent, Inc., and a stabilized DC power supply. The power consumption is the maximum at the moment when the five fingers are simultaneously activated. The time variation of the circuit voltage and current was measured at the time the five fingers of the robot hand were driven. The measurement was also performed by heating, with D = 100%, during the first 3 s and with the minimum duty ratio D = 40% that could sustain the bending of the robot finger for subsequent 15 s. However, the commencement of the bending of the thumb was delayed by 0.5 s to avoid interference with the other fingers. The power consumption was calculated from the measured voltage and current. A questionnaire survey using a robot hand was also conducted. The objective is to evaluate whether the generated gesture is recognized and the smoothness of the gesture. Four questions were developed and distributed to 15 students in the laboratory. Considering the 3 essentials of prevention in the COVID-19 pandemic, respondents answered the ques- In the first video, the thumb, ring finger and little finger are bent to generate a peace sign (Video 1). The peace sign was selected because it is a typical gesture that is classified as Symbolic Gestures or emblems. In the second video, the thumb and index finger are bent and their fingertips touch each other to generate an OK sign (Video 2). The OK sign was chosen because it seems more difficult to generate and perceive than the peace sign. In the third video, the index finger, middle finger, ring finger and little finger repeatedly bend and extend to generate a "Come on" gesture (Video 3). It was selected because it is a dynamic gesture. After watching the three videos, the respondents chose from four options: "I understood," "I could understand somewhat," "I didn't understand much" and "I didn't understand at all." This item is related to the validity of the driving angle of the robot hand when generating a symbolic gesture. Regarding the drive of gestures, Respondents also chose from four options: "I felt unnatural," "I felt a little unnatural," "I didn't feel unnatural," and "I didn't feel unnatural at all." This item is related to the validity of the responsiveness of the robot hand. If the participants do not feel unnatural in driving, the robot hand can provide a more human-like gesture. In order to know future issues and the feelings of the respondents, a free text box was provided at the end of the questionnaire. The results of the questionnaire survey were aggregated by simple tabulation. Figure 10 shows the relationship between the current joint angle and time, with the horizontal axis representing time and the vertical axis representing the angle. The heating duration is displayed in red, and at the end of the heating duration the supply of power is stopped, and then the cooling duration starts and is displayed in blue. The response of each joint is shown in the MP, PIP and DIP series, respectively. As a result, the bending angle of each joint reached 91% of the required specification. The remaining 9% is probably due to an error in the position of the hole, through which the BMF passed, or due to the clearance between the BMF and the hole. The response time was 1.87 s for bending and 1.63 s for extending. Even during the use of duty ratio D = 45%, each joint maintained its bending state without starting extension. The bending state was maintained, because the amount of Joule heating and the amount of heat cooling are balanced. At that time, the extending time was 1.63 s, which was within the required specification. The result was that the bending/extending time was within the required specification. If the Joule heating of BMF was stronger than the cooling, the SMA temperature became too high for extension in the required time. Figure 11a shows the relationship between the SMA voltage/current and time, with the horizontal axis representing time and the vertical axis representing the voltage/current. Figure 11b shows the relationship between the power consumption and time, with the horizontal axis representing time and the vertical axis representing power. The BMF voltage dropped from 14 V to 12.7 V at the start of bending, and it was 13.4 V while maintaining the bend. The voltage dropped due to the internal resistance of the power supply. The maximum current used was 910 mA and it was 480 mA while maintaining the bend. As a result, the maximum power consumption was 11.7 W, which is below the required specification of 15 W. Moreover, it was only 6.4 W while holding the bend. Four questions were developed and distributed to 15 students in the laboratory, with a response rate of 93%. Participants consist of 14 students from laboratory. The sample consists of 3 women (20%) and 12 men (80%). In the first video meaning "peace sign," 14 of them (100%) indicate that they understood the meaning of gestures clearly (Fig. 12) . In the second video meaning "OK sign," 12 of them (86%) indicate that they understood the meaning of gestures clearly, 2 of them (14%) indicate that they could understand somewhat. The driving angle of the robot's hand was sufficient to generate static symbolic gestures. Some respondents were difficult to recognize because the ring made by the thumb and index finger was distorted. In the third video meaning "Come on," 7 of them (50%) indicate that they understood the meaning of gestures clearly, 5 of them (36%) indicate that they could understand somewhat and 2 of them (14%) indicate that they didn't understand much. The third video assumed a "Come on" gesture, but many respondents recognized it as a thumbs-up, according Regarding the unnaturalness of gestures, 2 of them (14%) indicate that they felt unnatural, 8 of them (57%) indicate that they felt a little unnatural and 4 of them (29%) indicate that they didn't feel unnatural (Fig. 13 ). Many respondents find it unnatural when multiple fingers are out of sync. It seems that the reason why the finger drive is not synchronized is that the tensions of the five BMFs are not the same and that the training process of the BMFs is also insufficient [34] . Figure 14 shows some typical gestures such as a thumbsup, a peace sign and an OK sign. Since the three joints are driven by the one actuator, each joint cannot bend to an arbitrary angle, but by adjusting the duty ratio, it was possible to maintain a lightly bent state, like that of the index finger with an OK sign. The communication between humans and robots becomes smoother by using gestures. Moreover, to alleviate the feeling Fig. 13 Results of the gesture unnaturalness questionnaire of threat, a small robot, the size of an infant, is suitable. Therefore, we proposed the development of an SMA robot hand for a small interface robot that communicates with humans using gestures. Many studies using SMA for robot hand actuators are aimed at gripping and are the same size as human hands. On the other hand, the small robot hand with five fingers in the previous research has a large base unit and cannot be attached to a small robot. In this paper, a robot hand with a hand part of 45 mm and an overall length of 161 mm × 46 mm × 40 mm has been developed and can be mounted on an existing small robot with a height of 40-50 cm. Four specifications were required for our robot hand. The distance from the wrist to the middle fingertip was set to 45 mm, for mounting on a small robot. In order to express gestures, the robot hand requires a driving angle that bends the MP joint to θ MP = 80 • , the PIP joint to θ PIP = 90 • , and the DIP joint to θ DIP = 70 • . In order to avoid giving the user an unnatural impression, the required response time of the fingers of the robot hand is set to be less than 2 s, for each case of extension and bending. Considering the power margin of commercial- We designed and fabricated an SMA robot finger first followed by a five-fingered SMA robot hand, and evaluated whether the required specifications were achieved by two experiments. The joint angle and response time were measured using a single finger. As a result, by adjusting the duty ratio by PWM, the bending and extending times were 1.87 s and 1.63 s, respectively, and the joint angle reached 91% of the required specification. The power consumption was measured, while bending five fingers simultaneously using the five-fingered robot hand. It was observed that the maximum power consumption was 11.7 W. In conclusion, our small robot hand could satisfy all four requirements. A questionnaire-based survey was also conducted with a robot hand. The typical gestures that are classified as symbolic gestures such as a peace sign and an OK sign were perceived well by participants. Dynamic gesture repeating the bending and extending such as "Come on" was difficult to perceive by participants. Many participants find it unnatural when multiple finger are out of sync. Not only response of individual finger, but driving of multiple fingers also needs to be evaluated. In the future, we also need to evaluate the characteristics of the gestures and the visibility from the user's point of view. We further recommend the modeling of the torque generated around each joint by BMF and the bias spring in the future. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecomm ons.org/licenses/by/4.0/. A communication robot interacting with people daily in an office Effects of nonverbal communication on efficiency and robustness in human-robot teamwork Effective application of PALRO: a humanoid type robot for people with dementia The effects of robot voice and gesture types on the perceived robot personalities How to build robots that make friends and influence people Development and evaluation of an interactive interactive humanoid robot Analysis of Impression of Robot Bodily Expression Are bigger robots scary? The relationship between robot size and psychological threat A friendly gesture: investigating the effect of multimodal robot behaviour in humanrobot Interaction Conversational gestures in human-robot interaction Nonverbal communication in humans Nonverbal communication in socially assistive human-robot interaction Adaptive emotional expression in robot-child interaction Outside appearance of a healing robot by using sensitivity analysis Development of a master-slave five finger robot hand for laparoscopic surgery to improve operators sense of immersion Miniature five-fingered robot hand driven by shape memory alloy actuators Louder than words. In: An introduction to nonverbal communication Aspects of nonverbal communication. Swets & Zeitlinger lisse, Olanda A methodological approach relating the classification of gesture to identification of human intent in the context of human-robot interaction Human body dimension database 1991-1992 Response time in man-computer conversational transactions The challenge of Wearable computing Designing the user interface List of functions/specifications Biomimetic inchworm robot using SMA coil actuator and smart composite microstructures (SCM) SMA actuator and pulse-type hardware neural networks IC for fast walking motion of insect-type MEMS microrobot Shape memory alloy actuated robot prostheses: initial experiments Metal artificial muscle BioMetal fiber A shape memory alloy-based tendon-driven actuation system for biomimetic artificial fingers, part I: design and evaluation One directional active bending mechanism using shape memory alloy wire for minimally invasive medicine and feedback control by changing in electrical resistance of wire Modeling of shape memory alloy actuator and tracking control system with the model Cyclic deformation of TiNi shape memory alloy helical springs (effect of training and shape memory processing temperature)