key: cord-0076545-yl3gpmdw
authors: Kim, SangUn; Gu, SeongMo; Kim, Jooyong
title: Variable Shape and Stiffness Feedback System for VR Gloves Using SMA Textile Actuator
date: 2022-04-01
journal: Fibers Polym
DOI: 10.1007/s12221-022-3349-3
sha: 8c0170a3c9b70c09d77b245772ca5f0f5478159b
doc_id: 76545
cord_uid: yl3gpmdw

Currently, when direct human contact is limited owing to the coronavirus era, virtual reality systems are a technology that enables interaction without spatial restrictions. In this study, a system capable of feedbacking various shapes and stiffness when touching an object in virtual reality was conceived using the SMA textile actuator, which is a shape memory alloy (SMA) that remembers by training at 500 °C for 15 min with training tools that fixed shapes of SMA origin, and it returns to the shape at 40.13 °C or higher by Joule heating. SMA textile actuators were fabricated by zigzag stitching with 0.5 mm intervals. Various shapes were feedback because the SMA textile actuators on gloves interfere with the movement of the finger joints and the feedback force levels of stiffness are divided based on the number of actuators operating, and feedback is possible in three levels. We verified shape feedback that matches sensing points that are placed on the virtual finger joints created by Unity 3D, which is a software for making virtual reality to feedback points on VR gloves with three events using Unity 3D logs. We measured the forces of each feedback force level with SMA diameters 0.4 of 0.6 mm. The results showed 2.3 N with 0.6 mm; however, owing to its rigidity, 0.5 mm SMA textile actuator is appropriate. Finally, by implementing one feedback point of the VR gloves, we verified the operation of the system from the occurrence of an event by touching an object in virtual reality to control the feedback force level through the MCU and relay.

Virtual reality (VR) is a technology that removes spatial constraints and allows for various experiences. Recently, it has attracted attention as a technology that can prevent direct contact between people caused by the coronavirus pandemic. There are various types of devices such as sticks, pads, guns, and VR gloves using hands, the subject of human actions, making the wearer feel more realistic. To improve this sense of reality, a feedback system that can effectively deliver events occurring in virtual reality to the hand is needed. Bergamasco [1] provided a guideline for a force feedback system to the wearer by analyzing the contact force based on the movement of the hand. Subsequently, in 1995, an interface system required in a virtual environment was studied as a far-end force feedback system. Uchiyama et al. [2] studied feedback gloves that inform the direction and space of wheelchair users owing to visual dysfunction by attaching nine vibration motors. Kim et al. [3] developed gloves that can apply to multimedia such as movies and computer games by increasing the number of vibration motors to 20 similarly. Through many studies, a feedback system using a vibration motor could express a 3D space in a direction. In [4] , a feedback system in virtual reality was implemented using a vibration motor. This is a glove-type feedback system that signals the glove's control board through wireless communication with mobile phones, computers, game consoles, etc., and informs it through vibrations from the fingers and palms [5] , developed gloves that can operate through data in virtual reality by producing an exoskeleton that can assist hands using actuators with vibration feedback. The feedback system using the vibration motor studied earlier can express the direction and space in virtual reality; however, it is difficult to express information about the stiffness and shape when holding an object in virtual reality. Therefore [6] , developed a feedback system in which the position of the handle held by the operator by pneumatic pressure varies depending on the stiffness or shape of an object in virtual reality. In this study, a feedback motion was designed using a shape memory alloy (SMA) textile actuator that returns the gloves to their original shape at a certain temperature or higher, which can provide feedback on the stiffness or shape of an object in virtual reality. The principle of shape feedback is interference when the SMA textile actuator operates each feedback point, which returns the original linear shape such that the wearer can feel the feedback, and the stiffness feedback is the number of operating SMA textile actuators that produce feedback force levels.

The shape memory effect (SME), which is the effect of returning to its original shape at a certain temperature or higher, of shape memory alloys (SMA), and research has been conducted to develop an actuator using this effect as a *Corresponding author: jykim@ssu.ac.kr driving force. Consequently, based on [7] , it was confirmed that the SMA actuator can lift over 100 times the working density and weight of the actuator, 25 times that of conventional electric motors. In addition, SMA does not require an auxiliary device because the driving force is the change in the crystal structure in the martensite and austenite states, each having a temperature range and not an electrical force. In smart wearables, which are effective in areas requiring soft actuators with limited size and weight, and wearable smart devices, SMA is fabricated to develop textile actuators [8] . Trained SMA textile actuators with SMA training tools of a protective actuator can create an air gap between the skin and clothes through a knitting process using a linear SMA and applied to firefighting suits and [9] SMA actuator studies. An experiment was conducted to load a certain weight on the manufactured textile actuator and to measure its restoring force. Consequently, SMA textile actuators were developed as wearable actuators, and [10] memorized the bent shape and applied it to develop a haptic device. In this study, we referenced the methods of operating SMA with Joule heating and training SMA with a linear shape to a feedback system that can provide feedback on variable shape and stiffness by applying SMA textile actuators to virtual reality.

To provide feedback on shape and stiffness in virtual reality, sensing and feedback must be divided. In this study, by referring to the study of [1] , the central part of the finger joint in virtual reality can identify the event of contact and action with an object. It was divided into VR sensing points and real feedback points that provide feedback on the shape and stiffness by interfering the motion of the joint with the SMA textile actuator at the finger joint. The principle of feedback is a system in which when grasping an object, the finger is bent by the joint; when the SMA textile actuator is operated, the stiffness increases and the actuator tries to return it to a linear shape using the SME. Generally, the shape and stiffness feedback is a system that detects the part of the virtual reality gripping the object at the center of the finger joint and feeds it back by interfering with the movement of the finger joint.

Shape Stiffness Feedback Information on occurring event was provided to the microcontroller unit (MCU) in the VR gloves using wireless communication such as Bluetooth or Wi-Fi with Unity 3D, which is a software that can make VR content when an event occurs by touching an object or holding an object in virtual reality. The SMA textile actuator was delivered and operated by Joule heating. The feedback on the shape is indicated by the point that interferes with the movement of the finger joint, and the feedback of the stiffness operates several SMA textile actuators progressively to vary the interference.

Fabrication of SMA Textile Actuator SMA textile actuator, SMA Co., Ltd. In Korea, a wiretype SMA made with a 1:1 ratio of nickel and titanium was used, and it was found that the temperature at which the SME of the SMA wire occurs is 40.13°C through DSC measurement, as shown in Figure 3 . To memorize the original linear shape that the SMA textile actuators are returned when they are operating, a training tool was created, as shown in Figure 3 , and the SMA textile actuator was fixed to the original shape by a training tool that was memorized at 500°C for 15 min. The SMA textile actuator can be fabricated in various ways, such as knitting, stitching, and inserting by a layer; however, in this study, zigzag set-in internal programming using a sewing machine (Brother Company) to provide partial operation through stitching is shown in Figure 4 . The stitching was fabricated with SMA textile actuators at 0.5 mm intervals. A cotton/Spandex blended fabric was used as the base fabric to maximize the feedback effect by being in close contact with the finger when actuating the gloves.

Shape To provide feedback on the shape of an object in virtual reality, when the user touches the object, the object's touch box is touched by virtual hands to determine which sensing points of the event occur; when force feedback is performed, the feedback force can give the wearer the feeling of touching the object. Therefore, by creating a virtual reality hand and putting 15 sensing points on the finger joints and palms, a system that notifies the occurrence of events to the sensing points that touch the object to determine the number of touch points when touching objects such as cubes, spheres, and cups for measuring, as shown in Figure 5 .

To feedback the stiffness of objects in virtual reality, divide the object's touch box based on the stiffness level, as shown in Figure 6 , and divide it by the number of SMA textile actuators. With three levels, when a user touches an object, it is possible to know how the user touches it strongly or the hardness of the object in the virtual reality is touched based on each level. As a measurement device, Dacell's DN-KGA 5 K was used to measure the force (SME) of the SMA textile actuator under loading, as in the study of [9] with a resolution of 0.1 N, as shown in Figure 7 .

The shape and stiffness feedback system is made up of VR gloves that have one feedback point that can be checked by the entire system, and it uses Unity 3D, a software that can produce virtual reality. As shown in Figure 8 , a log is generated on the occurrence of an event between the object and the virtual reality hand. The MCU determines the actuating point of the shape feedback as the touched point for the occurrence of this event and determines the stiffness force feedback level through the touch box of the object. Therefore, VR rendering actuators check the communication between the MCU and VR through the logs of Unity 3D and check the actuation through the on/off relay connected to the MCU to control the SMA textile actuator.

Using Unity 3D, the shape of the virtual reality hand and the sensing point were checked as a log using three methods, as shown in Figure 5 , and the point to give feedback by checking the sensing point where the virtual reality hand touched the object was found. Figure 9 (a), which results from the shape of VR hands, is feedback on the motion of touching the object with one finger, and the sensing point is P4, and a touch log occurred. In the case of (b), as a motion of placing a ball-shaped object on the hand, the sensing points were p1, p3, p6, p9, p12, p15, and a touch log was generated. In the case of (c), which is a more complex motion, the touch log of all sensing points except p2 and the sensing point of the thumb tip from the fist, was generated by the motion of holding the handle of the cup-shaped object.

Based on the measurement method in stiffness, the feedback force of SMA textile actuators made with SMA diameters of 0.4, 0.5, and 0.6 mm was compared. In the case of the maximum force, the force balanced for 10 s was defined as the maximum force of the actuator. In addition, direct resistance heating passed a constant current through the power supply and was set as the amount of current that can be heated to approximately 45-50°C, assuming the operating temperature of the SMA textile actuator at 40.13°C.

As shown in Figure 10 , the maximum forces of 2.3 N, 0.8 N, and 0.4 N were measured in the order of 0.6 to 0.4 with large diameters in a graph, based on the feedback force level 3 of the SMA textile actuator. When comparing the actuator force, the force of the SMA textile actuator of 0.6 mm was large; however, the 0.5 mm SMA wire was judged to be suitable owing to the rigidity of the alloy based on the diameter in the fabrication. In graphs b, c, and d, the force based on the force feedback level was measured in the SMA textile actuator of the same diameter, and it was confirmed that the force increased as the level increased. Figure 8 shows an experimental method to confirm the system delivered to the SMA textile actuator manufactured in this study. The event signal generated by the object was transmitted to the MCU, and the operation LED of the relay channel controlling the actuator operation was seen.

The VR gloves in Figure 11 were manufactured by realizing a point to check the operation of the feedback system, and a circuit was made using conductive textile to the SMA textile actuator, and a protective layer was fabricated by the thermal bonding of a protective film. The operation of the overall feedback system confirmed that a relay controlling the SMA textile actuator was turned on because of an event that occurred while touching an object in Unity 3D.

In this study, a system capable of producing SMA textile actuators in the form of fabric using the characteristics of a shape memory alloy that memorizes the shape at a certain temperature or higher and returns to that shape, and provides feedback on the shape and stiffness of the object in virtual reality was envisioned.

In the case of the shape, the object is touched in three different motions in virtual reality, and through the sensing points of the virtual reality hand, it can be confirmed as shown in Figure 9 . If these points are feedback, the wearer can feel the shape of the object.

In the case of stiffness, to determine the force of each feedback force level, the force from level 1 to level 3 as per the diameter of 0.4 (SMA of 0.6 mm) is shown in Figure 10 . The feedback force was the greatest from 0.6 mm in diameter to 2.3 N; however, in consideration of the strength during non-operation owing to its rigidity, we concluded that 0.5 mm is suitable considering the wearer's fit.

The confirmation of the communication system between virtual reality and reality is shown in Figure 11 . The event of touching the object, sensing points, and operation of the SMA textile actuator can be confirmed through serial communication with the MCU. This was confirmed through the relay operation based on the object touch implemented using Unity 3D.

The shape and stiffness feedback system using the SMA textile actuator presented in this study can provide specific information that cannot be fed back to the existing vibration element of the actual wearer, and if various force feedback levels are generated, spatially, it will be used in various fields, such as industry and education, where there are restrictions.

Design of Hand Force Feedback Systems for Glove-like Advanced Interfaces

Vibrotactile Glove Guidance for Semi-autonomous Wheelchair Operations

Vibrotactile Feedback for VR Systems

Haptic Guidance of Light-Exoskeleton for Arm-rehabilitation Tasks

A Variable Shape and Variable Stiffness Controller for Haptic Virtual Interactions

Variable Stiffness Fabrics with Embedded Shape Memory Materials for Wearable Applications

Shape Memory Alloy Wire Actuators for Soft, Wearable Haptic Devices