Gionata Salvietti

Mapping Synergies From Human to Robotic Hands With Dissimilar Kinematics: An Approach in the Object Domain

One of the major limitations to the use of advanced robotic hands in industries is the complexity of the control system design due to the large number of motors needed to actuate their degrees of freedom. It is our belief that the development of a unified control framework for robotic hands will allow us to extend the use of these devices in many areas. Borrowing the terminology from software engineering, there is a need for middleware solutions to control the robotic hands independently from their specific kinematics and focus only on the manipulation tasks. To simplify and generalize the control of robotic hands, we take inspiration from studies in neuroscience concerning the sensorimotor organization of the human hand. These studies demonstrated that, notwithstanding the complexity of the hand, a few variables are able to account for most of the variance in the patterns of configurations and movements. The reduced set of parameters that humans effectively use to control their hands, which are known in the literature as synergies, can represent the set of words for the unified control language of robotic hands, provided that we solve the problem of mapping human hand synergies to actions of the robotic hands. In this study, we propose a mapping designed in the manipulated object domain in order to ensure a high level of generality with respect to the many dissimilar kinematics of robotic hands. The role of the object is played by a virtual sphere, whose radius and center position change dynamically, and the role of the human hand is played by a hand model referred to as “paradigmatic hand,” which is able to capture the idea of synergies in human hands.

SynGrasp: A MATLAB toolbox for grasp analysis of human and robotic hands

SynGrasp is a MATLAB toolbox developed for the analysis of grasping, suitable both for robotic and human hands. It includes functions for the definition of hand kinematic structure and of the contact points with a grasped object. The coupling between joints induced by an underactuated control can be modeled. The hand modeling allows to define compliance at the contact, joint and actuator levels. The provided analysis functions can be used to investigate the main grasp properties: controllable forces and object displacement, manipulability analysis, grasp quality measures. Functions for the graphical representation of the hand, the object and the main analysis results are provided.

The flying hand: A formation of UAVs for cooperative aerial tele-manipulation

The flying hand is a robotic hand consisting of a swarm of UAVs able to grasp an object where each UAV contributes to the grasping task with a single contact point at the tooltip. The swarm of robots is teleoperated by a human hand whose fingertip motions are tracked, e.g., using an RGB-D camera. We solve the kinematic dissimilarity of this unique master-slave system using a multi-layered approach that includes: a hand interpreter that translates the fingertip motion in a desired motion for the object to be manipulated; a mapping algorithm that transforms the desired object motions into a suitable set of virtual points deviating from the planned contact points; a compliant force control for the case of quadrotor UAVs that allows to use them as indirect 3D force effectors. Visual feedback is also used as sensory substitution technique to provide a hint on the internal forces exerted on the object. We validate the approach with several human-in-the-loop simulations including the full physical model of the object, contact points and UAVs.

The Sixth-Finger: A modular extra-finger to enhance human hand capabilities

Robotic prosthesis are usually intended as artificial device extensions replacing a missing part of a human body. A new approach regarding robotic limbs is presented here. A modular robot is used not only for replacing a missing part of the body but also as an extra-limb in order to enhance manipulation dexterity and enlarge the workspace of human beings. In this work, the model and control of an additional finger, the Sixth-Finger, is presented as a case study of this type of robotic limbs. The robotic finger has been placed on the wrist opposite to the hand palm. This solution allows to enlarge the hand workspace, increasing the grasp capability of the user. An object-based mapping algorithm is proposed to control the robotic extra-finger by interpreting the whole hand motion in grasping action. A four DoFs modular prototype is presented along with numerical simulations and real experiments. The proposed Sixth-Finger can lead to a wide range of applications in the direction of augmenting human capabilities through wearable robotics.

The Soft-SixthFinger: a Wearable EMG Controlled Robotic Extra-Finger for Grasp Compensation in Chronic Stroke Patients

This letter presents the Soft-SixthFinger, a wearable robotic extra-finger designed to be used by chronic stroke patients to compensate for the missing hand function of their paretic limb. The extra-finger is an underactuated modular structure worn on the paretic forearm by means of an elastic band. The device and the paretic hand/arm act like the two parts of a gripper working together to hold an object. The patient can control the flexion/extension of the robotic finger through the eCap, an electromyography-based (EMG) interface embedded in a cap. The user can control the device by contracting the frontalis muscle. Such contraction can be achieved simply moving his or her eyebrows upwards. The Soft-SixthFinger has been designed as tool that can be used by chronic stroke patients to compensate for grasping in many activities of daily living (ADL). It can be wrapped around the wrist and worn as a bracelet when not used. The light weight and the complete wireless connection with the EMG interface guarantee a high portability and wearability. We tested the device with qualitative experiments involving six chronic stroke patients. Results show that the proposed system significantly improves the performances of the patients in the proposed tests and, more in general, their autonomy in ADL.

SynGrasp: A MATLAB Toolbox for Underactuated and Compliant Hands

SynGrasp is a MATLAB toolbox for grasp analysis of fully or underactuated robotic hands with compliance. Compliance can be modeled at contact points in the joints or in the actuation system, including transmission. It is possible to use a graphical user interface (GUI) or directly assemble and modify the available functions to exploit all of the toolbox features. Grasps can be described either using the provided grasp planner or by directly defining contact points on the hand with the respective contact normal directions. Several analysis functions have been developed to investigate the main grasp properties: 1) controllable forces and object displacement, 2) manipulability analysis, and 3) grasp stiffness and quality measures. The functions for the graphical representation of the hand and object as well as the main analysis results are provided. The toolbox is freely available at http://syngrasp.dii.unisi.it.

An Object-Based Approach to Map Human Hand Synergies onto Robotic Hands with Dissimilar Kinematics.

This chapter introduces the main equations necessary to study hands controlled by synergies. Most of the results presented here are related to grasp and manipulation analysis of underactuated structures that present compliance at joint and contact level. The introduction of the synergies concept in robotics can be seen, in fact, as a possible reduction of the hand DoF space. Compliance is needed to solve static indeterminacy. 2.1 Quasi-static manipulation model Consider a generic hand as a collection of arbitrary numbers of robot “fingers” (i.e., simple chains of links connected through revolute or prismatic joints) attached to a common base “palm”, and an object, which is in contact with all or some of the links as sketched in Fig. 2.1. The hand and the object have nc contact points. The position of the contact point i in {N} is defined by the vector pi ∈R3. At contact point i, we define a frame {C}i, with axes {n̂i, t̂i, ôi}. The unit vector n̂i contains pi is normal to the contact tangent plane and is directed toward the object. The other two unit vectors are orthogonal and lie in the tangent plane of the contact. 12 2. Mathematics of hand synergies Table 2.1: Notation for Grasp Analysis Notation Definition u ∈R6 position and orientation of the object w ∈R6 external wrench applied to the grasped object nd system dimension nc number of contact points Co i reference system at the i-th contact point on the object ci ∈R3 position of the contact point i ĉi ∈Rnd position and orientation of reference frame Co i Ch i reference system at the i-th contact point on the hand ĉi ∈Rnd position and orientation of reference frame Ch i λi vector of forces (and moments) at the contact i λ ∈Rnl vector of contact forces (and moments) nl dimension of the contact force vector nq number of joints q ∈Rnq actual joint variables qre f ∈Rnq reference joint variables τ vector of joint forces and torques nz number of postural synergies z ∈Rnz synergy variables σ ∈Rnz generalized forces along synergies G ∈Rnd×nl grasp matrix J ∈Rnl×nq hand jacobian matrix S ∈Rnq×nz synergy matrix Let the joints be numbered from 1 to nq. Denote by q = [q1...qnq ] T ∈ Rnq the vector of joint displacement. Also, let τ = [τ1...τnq ] T ∈ Rnq represent joint loads (forces in prismatic joints and torques in revolute joints). Let u ∈ Rnu denote the vector describing the position and orientation of {B} relative to {N}. For planar systems nu = 3. For spatial systems, nu is three plus the number of parameter used to represent orientation, typically three for Euler angle and four for quaternion. Denote by ν = [vT ωT ] ∈ Rnν the twist of object described in {N}. It is composed of the translational velocity ν ∈ R3 of the point o and the angular velocity ω ∈R3 of the object, both expressed in {N}. A twist of a rigid body can be referred to any convenient frame fixed to the body. The components of the referred twist represent the velocity of the origin of the new frame and the angular velocity of the body, both expressed in the new frame. 2.1. Quasi-static manipulation model 13

Using the robotic sixth finger and vibrotactile feedback for grasp compensation in chronic stroke patients

This paper presents a wearable robotic extra finger used by chronic stroke patients to compensate for the missing hand functions of the paretic limb. The extra finger is worn on the paretic forearm by means of an elastic band, and it is coupled with a vibrotactile ring interface worn on the healthy hand. The robotic finger and the paretic hand act like the two parts of a gripper working together to hold an object. The human user is able to control the flexion/extension of the robotic finger through a switch placed on the ring, while being provided with vibrotactile feedback about the forces exerted by the robotic finger on the environment. To understand how to control the vibrotactile interface to evoke the most effective cutaneous sensations, we carried out perceptual experiments to evaluate its absolute and differential thresholds. Finally, we performed a qualitative experiment, the Franchay Arm Test, with a chronic post-stroke patient presenting a partial loss of sensitivity on the paretic limb. Results show that the proposed system significantly improves the performance of the considered test.

On the use of homogeneous transformations to map human hand movements onto robotic hands

Replicating the human hand capabilities is a great challenge in telemanipulation as well as in autonomous grasping and manipulation. One of the main issues is the difference between human and robotic hands in terms of kinematic structure, which does not allow a direct correlation of the joints. We recently proposed an object-based mapping algorithm able to replicate on several robotic hand models the human hand synergies. In such approach the virtual object shapes were a-priori defined (e.g. a sphere or an ellipsoid) and the transformation was represented as the composition of a rigid body motion and a scale variation. In this work, we introduce a generalization of the object-based mapping that overcomes the definition of a shape for the virtual object. We consider only a set of reference points on the hands. We estimate a homogeneous transformation matrix that represents how the human hand motion changes its reference point positions. The same transformation is then imposed to the reference points on the robotic hand and the joints values obtained through a kinematic inversion technique. The mapping approach is suitable also for telemanipulation scenarios where the hand joint motions are combined with a wrist displacement.

An object-based mapping algorithm to control wearable robotic extra-fingers

One of the new targets of wearable robots is not to enhance the lift strength far above human capability by wearing a bulky robot, but to support human capability within its range by wearing lightweight and compact robots. A new approach regarding robotic extra-fingers is presented here. In particular, an object-based mapping algorithm is proposed to control the robotic extra-fingers by interpreting the whole or a part of the hand motion in grasping and manipulation tasks. As a case study, the model and control of an additional robotic finger is presented. The robotic finger has been placed on the wrist opposite to the hand palm. This solution enlarges the hand workspace, increasing the grasp capability of the user. The proposed mapping algorithm do not require the human operator to activate explicit commands. Rather, the motion of the extra-fingers is connected to the human hand so that the user can perceive the robotic fingers as an extension of his body.