Anna Lisa Ciancio

Restoring Natural Sensory Feedback in Real-Time Bidirectional Hand Prostheses

A multigrasp, bidirectional hand prosthesis delivers dynamic sensory feedback, allowing a user with a hand amputation to achieve fine grasping force control and realistic object sensing. An Artificial Hand’s Sense of Touch To feel the hard curvature of a baseball or the soft cylinder that is a soda can—these sensations we often take for granted. But amputees with a prosthetic arm know only that they are holding an object, the shape and stiffness discernible only by eye or from experience. Toward a more sophisticated prosthetic that can “feel” an object, Raspopovic and colleagues incorporated a feedback system connected to the amputee’s arm nerves, which delivers sensory information in real time. The authors connected electrodes in the arm nerves to sensors in two fingers of the prosthetic hand. To “feel” an object, the electrodes delivered electrical stimuli to the nerves that were proportional to the finger sensor readouts. To grasp an object and perform other motor commands, muscle signals were decoded. This bidirectional hand prosthetic was tested in a single amputee who was blindfolded and acoustically shielded to assure that sound and vision were not being used to manipulate objects. In more than 700 trials, the subject showed that he could modulate force and grasp and identify physical characteristics of different types of objects, such as cotton balls, an orange, and a piece of wood. Such sensory feedback with precise control over a hand prosthetic would allow amputees to more freely and naturally explore their environments. Hand loss is a highly disabling event that markedly affects the quality of life. To achieve a close to natural replacement for the lost hand, the user should be provided with the rich sensations that we naturally perceive when grasping or manipulating an object. Ideal bidirectional hand prostheses should involve both a reliable decoding of the user’s intentions and the delivery of nearly “natural” sensory feedback through remnant afferent pathways, simultaneously and in real time. However, current hand prostheses fail to achieve these requirements, particularly because they lack any sensory feedback. We show that by stimulating the median and ulnar nerve fascicles using transversal multichannel intrafascicular electrodes, according to the information provided by the artificial sensors from a hand prosthesis, physiologically appropriate (near-natural) sensory information can be provided to an amputee during the real-time decoding of different grasping tasks to control a dexterous hand prosthesis. This feedback enabled the participant to effectively modulate the grasping force of the prosthesis with no visual or auditory feedback. Three different force levels were distinguished and consistently used by the subject. The results also demonstrate that a high complexity of perception can be obtained, allowing the subject to identify the stiffness and shape of three different objects by exploiting different characteristics of the elicited sensations. This approach could improve the efficacy and “life-like” quality of hand prostheses, resulting in a keystone strategy for the near-natural replacement of missing hands.

Literature Review on Needs of Upper Limb Prosthesis Users.

The loss of one hand can significantly affect the level of autonomy and the capability of performing daily living, working and social activities. The current prosthetic solutions contribute in a poor way to overcome these problems due to limitations in the interfaces adopted for controlling the prosthesis and to the lack of force or tactile feedback, thus limiting hand grasp capabilities. This paper presents a literature review on needs analysis of upper limb prosthesis users, and points out the main critical aspects of the current prosthetic solutions, in terms of users satisfaction and activities of daily living they would like to perform with the prosthetic device. The ultimate goal is to provide design inputs in the prosthetic field and, contemporary, increase user satisfaction rates and reduce device abandonment. A list of requirements for upper limb prostheses is proposed, grounded on the performed analysis on user needs. It wants to (i) provide guidelines for improving the level of acceptability and usefulness of the prosthesis, by accounting for hand functional and technical aspects; (ii) propose a control architecture of PNS-based prosthetic systems able to satisfy the analyzed user wishes; (iii) provide hints for improving the quality of the methods (e.g., questionnaires) adopted for understanding the user satisfaction with their prostheses.

Control of Prosthetic Hands via the Peripheral Nervous System.

This paper intends to provide a critical review of the literature on the technological issues on control and sensorization of hand prostheses interfacing with the Peripheral Nervous System (i.e., PNS), and their experimental validation on amputees. The study opens with an in-depth analysis of control solutions and sensorization features of research and commercially available prosthetic hands. Pros and cons of adopted technologies, signal processing techniques and motion control solutions are investigated. Special emphasis is then dedicated to the recent studies on the restoration of tactile perception in amputees through neural interfaces. The paper finally proposes a number of suggestions for designing the prosthetic system able to re-establish a bidirectional communication with the PNS and foster the prosthesis natural control.

Differences between kinematic synergies and muscle synergies during two-digit grasping.

The large number of mechanical degrees of freedom of the hand is not fully exploited during actual movements such as grasping. Usually, angular movements in various joints tend to be coupled, and EMG activities in different hand muscles tend to be correlated. The occurrence of covariation in the former was termed kinematic synergies, in the latter muscle synergies. This study addresses two questions: (i) Whether kinematic and muscle synergies can simultaneously accommodate for kinematic and kinetic constraints. (ii) If so, whether there is an interrelation between kinematic and muscle synergies. We used a reach-grasp-and-pull paradigm and recorded the hand kinematics as well as eight surface EMGs. Subjects had to either perform a precision grip or side grip and had to modify their grip force in order to displace an object against a low or high load. The analysis was subdivided into three epochs: reach, grasp-and-pull, and static hold. Principal component analysis (PCA, temporal or static) was performed separately for all three epochs, in the kinematic and in the EMG domain. PCA revealed that (i) Kinematic- and muscle-synergies can simultaneously accommodate kinematic (grip type) and kinetic task constraints (load condition). (ii) Upcoming grip and load conditions of the grasp are represented in kinematic- and muscle-synergies already during reach. Phase plane plots of the principal muscle-synergy against the principal kinematic synergy revealed (iii) that the muscle-synergy is linked (correlated, and in phase advance) to the kinematic synergy during reach and during grasp-and-pull. Furthermore (iv), pair-wise correlations of EMGs during hold suggest that muscle-synergies are (in part) implemented by coactivation of muscles through common input. Together, these results suggest that kinematic synergies have (at least in part) their origin not just in muscular activation, but in synergistic muscle activation. In short: kinematic synergies may result from muscle synergies.

Hierarchical reinforcement learning and central pattern generators for modeling the development of rhythmic manipulation skills

The development of manipulation skills is a fundamental process for young primates as it leads them to acquire the capacity to modify the world to their advantage. As other motor skills, manipulation develops on the basis of motor babbling processes which are initially heavily based on the production of rhythmic movements. We propose a computational bio-inspired model to investigate the development of functional rhythmic hand skills from initially unstructured movements. The model is based on a hierarchical reinforcement-learning actor-critic model that searches the parameters of a set of central pattern generators (CPGs) having different degrees of sophistication. The model is tested with a simulated robotic hand engaged in rotating bottle cap-like objects having different shape and size. The results show that the model is capable of developing skills based on different combinations of CPGs so as to suitably manipulate the different objects. Overall, the model shows to be a valuable tool for the study of the development of rhythmic manipulation skills in primates.

The Role of Learning and Kinematic Features in Dexterous Manipulation: a Comparative Study with Two Robotic Hands

Dexterous movements performed by the human hand are by far more sophisticated than those achieved by current humanoid robotic hands and systems used to control them. This work aims at providing a contribution in order to overcome this gap by proposing a bio-inspired control architecture that captures two key elements underlying human dexterity. The first is the progressive development of skilful control, often starting from – or involving – cyclic movements, based on trial-and-error learning processes and central pattern generators. The second element is the exploitation of a particular kinematic features of the human hand, i.e. the thumb opposition. The architecture is tested with two simulated robotic hands having different kinematic features and engaged in rotating spheres, cylinders, and cubes of different sizes. The results support the feasibility of the proposed approach and show the potential of the model to allow a better understanding of the control mechanisms and kinematic principles underlying human dexterity and make them transferable to anthropomorphic robotic hands.

Learning by Demonstration for Motion Planning of Upper-Limb Exoskeletons

The reference joint position of upper-limb exoskeletons is typically obtained by means of Cartesian motion planners and inverse kinematics algorithms with the inverse Jacobian; this approach allows exploiting the available Degrees of Freedom (i.e. DoFs) of the robot kinematic chain to achieve the desired end-effector pose; however, if used to operate non-redundant exoskeletons, it does not ensure that anthropomorphic criteria are satisfied in the whole human-robot workspace. This paper proposes a motion planning system, based on Learning by Demonstration, for upper-limb exoskeletons that allow successfully assisting patients during Activities of Daily Living (ADLs) in unstructured environment, while ensuring that anthropomorphic criteria are satisfied in the whole human-robot workspace. The motion planning system combines Learning by Demonstration with the computation of Dynamic Motion Primitives and machine learning techniques to construct task- and patient-specific joint trajectories based on the learnt trajectories. System validation was carried out in simulation and in a real setting with a 4-DoF upper-limb exoskeleton, a 5-DoF wrist-hand exoskeleton and four patients with Limb Girdle Muscular Dystrophy. Validation was addressed to (i) compare the performance of the proposed motion planning with traditional methods; (ii) assess the generalization capabilities of the proposed method with respect to the environment variability. Three ADLs were chosen to validate the system: drinking, pouring and lifting a light sphere. The achieved results showed a 100% success rate in the task fulfillment, with a high level of generalization with respect to the environment variability. Moreover, an anthropomorphic configuration of the exoskeleton is always ensured.

Interplay of Rhythmic and Discrete Manipulation Movements During Development: A Policy-Search Reinforcement-Learning Robot Model

The flexibility of human motor behavior strongly relies on rhythmic and discrete movements. Developmental psychology has shown how these movements closely interplay during development, but the dynamics of that are largely unknown and we currently lack computational models suitable to investigate such interaction. This work initially presents an analysis of the problem from a computational and empirical perspective and then proposes a novel computational model to start to investigate it. The model is based on a movement primitive capable of producing both rhythmic and end-point discrete movements, and on a policy search reinforcement learning algorithm capable of mimicking trial-and-error learning processes underlying development and efficient enough to work on real robots. The model is tested with hand manipulation tasks (“touching,” “tapping,” and “rotating” an object). The results show how the system progressively shapes the initial rhythmic exploration into refined rhythmic or discrete movements depending on the task demand. The tests on the real robot also show how the system exploits the specific hand-object physical properties, some possibly shared with developing infants, to find effective solutions to the tasks. The results show that the model represents a useful tool to investigate the interplay of rhythmic and discrete movements during development.

Multilevel control of an anthropomorphic prosthetic hand for grasp and slip prevention

The success of grasping and manipulation tasks of commercial prosthetic hands is mainly related to amputee visual feedback since they are not provided either with tactile sensors or with sophisticated control. As a consequence, slippage and object falls often occur. This article wants to address the specific issue of enhancing grasping and manipulation capabilities of existing prosthetic hands, by changing the control strategy. For this purpose, it proposes a multilevel control based on two distinct levels consisting of (1) a policy search learning algorithm combined with central pattern generators in the higher level and (2) a parallel force/position control managing slippage events in the lower level. The control has been tested on an anthropomorphic robotic hand with prosthetic features (the IH2 hand) equipped with force sensors. Bi-digital and tri-digital grasping tasks with and without slip information have been carried out. The KUKA-LWR has been employed to perturb the grasp stability inducing controlled slip events. The acquired data demonstrate that the proposed control has the potential to adapt to changes in the environment and guarantees grasp stability, by avoiding object fall thanks to prompt slippage event detection.

Tactile piezoresistive sensors for robotic application: Design and metrological characterization

Microfabricated tactile sensors gain importance for their application in bio-robotics. They are useful for the measurement of contact properties, in particular force and pressure, in three main fields, i.e., prosthetics and artificial skin, minimal access surgery and collaborative robotics. Among the different technological solutions, piezoresistive materials proved to be suitable for such an application. These materials show a change of electrical resistivity as a function of the applied strain. This work describes the design of a 2×2 array of piezoresistive elements and the experimental setup arranged for the array characterization, intended to be embedded within an artificial fingertip. The size of the bare array is 1.5×1.5×0.65 mm3. The finger has been designed to bio-mimic the behaviour of a human finger tip, thanks to the external layer of dragon skin. The static calibration of the sensors has been carried out by applying quasistatic normal loads on the mesa of each sensor of the array in two configurations (i.e., bare array and the array embedded in a fingertip). The sensors showed a linear response; the sensitivity ranges from 34 mV/N to 65 mV/N for the bare array, and from 16 mV/N to 39 mV/N for the array in the fingertip. No significant cross-talk (∼2%) has been observed during the test on the bare array. Further tests will be designed to characterize the response to tangential loads and assess the dynamic response of the sensors, as well as additional features which can be crucial for bio-robotic applications.