A wearable, ultralight interface for bimanual teleoperation of a compliant, whole-body-controlled humanoid robot

Dexterous bimanual manipulation still represents a challenging problem in autonomous robotics. An alternative solution in difficult situations is the teleoperation of a robotic slave. This video presents an unobtrusive bimanual teleoperation setup with very low weight (about 180 grams on each arm), consisting of two Vive virtual reality visual motion trackers and two Myo surface electromyography bracelets. Incremental Ridge Regression with Random Fourier Features is used to robustly enforce multiple hand configurations through simultaneous and proportional control, including power grasp and finger pointing, irrespective of body postural changes and rotation of the wrist; the model can be updated in real time. The video demonstrates complex, dexterous teleoperated bimanual daily-living tasks performed by the torque-controlled humanoid robot TORO. The tasks include unscrewing a lid from a bottle, pouring fluid from the bottle into a pot, grasping, carrying and releasing objects of various sizes, weights and shapes, holding a cordless phone while dialing a number on it, and balancing a ball between the two hands. All tasks require a timely transmission of positions, which are translated into suitable robot torques to let the operator achieve bimanual coordination and high repeatability in the pose of the robotic hands in order to enable fine manipulation. Category: regular video. I. ADDITIONAL DETAILS We have previously developed a setup for commercial hand prostheses mounted on splints and controlled via interactive learning and surface electromyography, allowing human subjects to perform dexterous manipulation in controlled conditions [1], [2]. Still, the subjects needed to use a wired interface, implying that no effectiveness of the approach while moving around could be proved. In this video we show that the approach can be effectively extended to teleoperated bimanual manipulation in a daily-living, homelike environment, grasping, carrying and using standard (i.e., non-instrumented nor prepared) objects. The human-machine interface we use is totally unobtrusive: it consists of two HTC Vive motion trackers and two Myo gesture control armbands. The interface is completely wireless, leaving the subject free to move around and operate, and its overall weight per arm is about 180 gr. The teleoperated platform is the torque-controlled robot TORO, with 25 degrees of freedom based on the KUKA LWR joints, 2 position controlled joints in the neck, and two iLimb prosthetic hands with 6 degrees of freedom per hand [3], [4]. The robot displays a compliant whole-body balancing controller that takes care of All authors are with the Institute of Robotics and Mechatronics, German Aerospace Center (DLR), 82234 Wessling, Germany. Email: .@dlr.de This work was partially funded by the DFG (Deutsche Forschungsgemeinschaft), Project No. 272314643, TACT-HAND. Fig. 1. Experimental Setup. The wearable interface is used as master device, while the torque-controlled humanoid robot TORO works as slave device. keeping the robot balance even in the presence of unknown disturbances [5]. The video shows a collection of daily living tasks performed in real time, with the human operator standing beside the robot. For the experiment, the master system provides a stream of combined data at 50 Hz, which is processed and then transmitted via wi-fi to the robot controller (which operates at 1 kHz). The implemented demonstrator is a realistic setting for testing prosthesis control for amputees, as they lack proprioception and rely mostly on visual feedback to perform the tasks. The technique provides natural and reliable control of dexterous hands.