Guido Gioioso

Mapping Synergies From Human to Robotic Hands With Dissimilar Kinematics: An Approach in the Object Domain

One of the major limitations to the use of advanced robotic hands in industries is the complexity of the control system design due to the large number of motors needed to actuate their degrees of freedom. It is our belief that the development of a unified control framework for robotic hands will allow us to extend the use of these devices in many areas. Borrowing the terminology from software engineering, there is a need for middleware solutions to control the robotic hands independently from their specific kinematics and focus only on the manipulation tasks. To simplify and generalize the control of robotic hands, we take inspiration from studies in neuroscience concerning the sensorimotor organization of the human hand. These studies demonstrated that, notwithstanding the complexity of the hand, a few variables are able to account for most of the variance in the patterns of configurations and movements. The reduced set of parameters that humans effectively use to control their hands, which are known in the literature as synergies, can represent the set of words for the unified control language of robotic hands, provided that we solve the problem of mapping human hand synergies to actions of the robotic hands. In this study, we propose a mapping designed in the manipulated object domain in order to ensure a high level of generality with respect to the many dissimilar kinematics of robotic hands. The role of the object is played by a virtual sphere, whose radius and center position change dynamically, and the role of the human hand is played by a hand model referred to as “paradigmatic hand,” which is able to capture the idea of synergies in human hands.

SynGrasp: A MATLAB toolbox for grasp analysis of human and robotic hands

SynGrasp is a MATLAB toolbox developed for the analysis of grasping, suitable both for robotic and human hands. It includes functions for the definition of hand kinematic structure and of the contact points with a grasped object. The coupling between joints induced by an underactuated control can be modeled. The hand modeling allows to define compliance at the contact, joint and actuator levels. The provided analysis functions can be used to investigate the main grasp properties: controllable forces and object displacement, manipulability analysis, grasp quality measures. Functions for the graphical representation of the hand, the object and the main analysis results are provided.

Turning a near-hovering controlled quadrotor into a 3D force effector

In this paper the problem of a quadrotor that physically interacts with the surrounding environment through a rigid tool is considered. We present a theoretical design that allows to exert an arbitrary 3D force by using a standard near-hovering controller that was originally developed for contact-free flight control. This is achieved by analytically solving the nonlinear system that relates the quadrotor state, the force exerted by the rigid tool on the environment, and the near-hovering controller action at the equilibrium points, during any generic contact. Stability of the equilibria for the most relevant actions (pushing, releasing, lifting, dropping, and left-right shifting) are proven by means of numerical analysis using the indirect Lyapunov method. An experimental platform, including a suitable tool design, has been developed and used to validate the theory with preliminary experiments.

The flying hand: A formation of UAVs for cooperative aerial tele-manipulation

The flying hand is a robotic hand consisting of a swarm of UAVs able to grasp an object where each UAV contributes to the grasping task with a single contact point at the tooltip. The swarm of robots is teleoperated by a human hand whose fingertip motions are tracked, e.g., using an RGB-D camera. We solve the kinematic dissimilarity of this unique master-slave system using a multi-layered approach that includes: a hand interpreter that translates the fingertip motion in a desired motion for the object to be manipulated; a mapping algorithm that transforms the desired object motions into a suitable set of virtual points deviating from the planned contact points; a compliant force control for the case of quadrotor UAVs that allows to use them as indirect 3D force effectors. Visual feedback is also used as sensory substitution technique to provide a hint on the internal forces exerted on the object. We validate the approach with several human-in-the-loop simulations including the full physical model of the object, contact points and UAVs.

SynGrasp: A MATLAB Toolbox for Underactuated and Compliant Hands

SynGrasp is a MATLAB toolbox for grasp analysis of fully or underactuated robotic hands with compliance. Compliance can be modeled at contact points in the joints or in the actuation system, including transmission. It is possible to use a graphical user interface (GUI) or directly assemble and modify the available functions to exploit all of the toolbox features. Grasps can be described either using the provided grasp planner or by directly defining contact points on the hand with the respective contact normal directions. Several analysis functions have been developed to investigate the main grasp properties: 1) controllable forces and object displacement, 2) manipulability analysis, and 3) grasp stiffness and quality measures. The functions for the graphical representation of the hand and object as well as the main analysis results are provided. The toolbox is freely available at http://syngrasp.dii.unisi.it.

An Object-Based Approach to Map Human Hand Synergies onto Robotic Hands with Dissimilar Kinematics.

This chapter introduces the main equations necessary to study hands controlled by synergies. Most of the results presented here are related to grasp and manipulation analysis of underactuated structures that present compliance at joint and contact level. The introduction of the synergies concept in robotics can be seen, in fact, as a possible reduction of the hand DoF space. Compliance is needed to solve static indeterminacy. 2.1 Quasi-static manipulation model Consider a generic hand as a collection of arbitrary numbers of robot “fingers” (i.e., simple chains of links connected through revolute or prismatic joints) attached to a common base “palm”, and an object, which is in contact with all or some of the links as sketched in Fig. 2.1. The hand and the object have nc contact points. The position of the contact point i in {N} is defined by the vector pi ∈R3. At contact point i, we define a frame {C}i, with axes {n̂i, t̂i, ôi}. The unit vector n̂i contains pi is normal to the contact tangent plane and is directed toward the object. The other two unit vectors are orthogonal and lie in the tangent plane of the contact. 12 2. Mathematics of hand synergies Table 2.1: Notation for Grasp Analysis Notation Definition u ∈R6 position and orientation of the object w ∈R6 external wrench applied to the grasped object nd system dimension nc number of contact points Co i reference system at the i-th contact point on the object ci ∈R3 position of the contact point i ĉi ∈Rnd position and orientation of reference frame Co i Ch i reference system at the i-th contact point on the hand ĉi ∈Rnd position and orientation of reference frame Ch i λi vector of forces (and moments) at the contact i λ ∈Rnl vector of contact forces (and moments) nl dimension of the contact force vector nq number of joints q ∈Rnq actual joint variables qre f ∈Rnq reference joint variables τ vector of joint forces and torques nz number of postural synergies z ∈Rnz synergy variables σ ∈Rnz generalized forces along synergies G ∈Rnd×nl grasp matrix J ∈Rnl×nq hand jacobian matrix S ∈Rnq×nz synergy matrix Let the joints be numbered from 1 to nq. Denote by q = [q1...qnq ] T ∈ Rnq the vector of joint displacement. Also, let τ = [τ1...τnq ] T ∈ Rnq represent joint loads (forces in prismatic joints and torques in revolute joints). Let u ∈ Rnu denote the vector describing the position and orientation of {B} relative to {N}. For planar systems nu = 3. For spatial systems, nu is three plus the number of parameter used to represent orientation, typically three for Euler angle and four for quaternion. Denote by ν = [vT ωT ] ∈ Rnν the twist of object described in {N}. It is composed of the translational velocity ν ∈ R3 of the point o and the angular velocity ω ∈R3 of the object, both expressed in {N}. A twist of a rigid body can be referred to any convenient frame fixed to the body. The components of the referred twist represent the velocity of the origin of the new frame and the angular velocity of the body, both expressed in the new frame. 2.1. Quasi-static manipulation model 13

On the use of homogeneous transformations to map human hand movements onto robotic hands

Replicating the human hand capabilities is a great challenge in telemanipulation as well as in autonomous grasping and manipulation. One of the main issues is the difference between human and robotic hands in terms of kinematic structure, which does not allow a direct correlation of the joints. We recently proposed an object-based mapping algorithm able to replicate on several robotic hand models the human hand synergies. In such approach the virtual object shapes were a-priori defined (e.g. a sphere or an ellipsoid) and the transformation was represented as the composition of a rigid body motion and a scale variation. In this work, we introduce a generalization of the object-based mapping that overcomes the definition of a shape for the virtual object. We consider only a set of reference points on the hands. We estimate a homogeneous transformation matrix that represents how the human hand motion changes its reference point positions. The same transformation is then imposed to the reference points on the robotic hand and the joints values obtained through a kinematic inversion technique. The mapping approach is suitable also for telemanipulation scenarios where the hand joint motions are combined with a wrist displacement.

Object-based bilateral telemanipulation between dissimilar kinematic structures

This paper presents a bilateral telemanipulation framework where the master and slave sub-systems have different kinematic structures. A virtual object is defined on the master and slave sides and used to capture the human hand motion and to compute the related force feedback. The force feedback is determined imposing that the same wrench acts on the master and slave virtual objects. An abstraction from the sub-system structures is obtained focusing on the effects produced on the manipulated object. The proposed approach has been tested with an experimental setup consisting of two haptic interfaces able to capture index and thumb motions on the master side and a DLR-HIT Hand II as slave sub-system.

HANDS.DVI: A DeVice-Independent Programming and Control Framework for Robotic HANDS

The scientific goal of HANDS.DVI consists of developing a common framework to programming robotic hands independently from their kinematics, mechanical construction, and sensor equipment complexity. Recent results on the organization of the human hand in grasping and manipulation are the inspiration for this experiment. The reduced set of parameters that we effectively use to control our hands is known in the literature as the set of synergies. The synergistic organization of the human hand is the theoretical foundation of the innovative approach to design a unified framework for robotic hands control. Theoretical tools have been studied to design a suitable mapping function of the control action (decomposed in its elemental action) from a human hand model domain onto the articulated robotic hand co-domain. The developed control framework has been applied on an experimental set up consisting of two robotic hands with dissimilar kinematics grasping an object instrumented with force sensors.

A force-based bilateral teleoperation framework for aerial robots in contact with the environment

In this paper a novel teleoperation framework for aerial robots that physically interact with the environment is presented. This framework allows to teleoperate the robot both in contact-free flight and in physical contact with the environment in order, e.g., to apply desired forces on objects of the environment. The framework is build upon an impedance-like indirect interaction force controller that allows to use standard underactuated aerial robots as force effectors. Haptic feedback from the master side enables the user to feel the contact forces exerted by the robot. An automatic potential field-based slowing-down policy is used by the robot to ensure a smooth transition between the contact-free motion phase and the force interaction phase. The effectiveness of the approach has been shown in extensive human-in-the-loop simulations including remote pressing of buttons on a surface and pushing a cart until it touches a wall.