Lorenzo Peppoloni

A novel 7 degrees of freedom model for upper limb kinematic reconstruction based on wearable sensors

Wearable motion tracking systems have gained large popularity in the last decades because of their effectiveness in many fields, from performance assessment to human-robot interaction. Among all the approaches, those based on inertial sensors have been widely explored. Since inertial sensors are affected by measurements drift, they need to be aided by other sensors, thus requiring sensor measurements to be fused. The most used sensor fusion techniques are based on Kalman filter. In particular, the Extended Kalman Filter (EKF) and the Unscented Kalman Filter (UKF) are used because of the non linearity characterizing most of the models. They often aim at reconstructing human motion by estimating limbs orientation, involving humans kinematics to constrain relative motion of the limbs. These models often neglect part of the degrees of freedom (DoFs) that characterize human upper limbs, especially when modeling humerus motion with respect to the chest. In this paper we present a novel 7 DoFs model which represents a trade-off between modeling accuracy and complexity for the human upper limb. In particular, we model the human shoulder girdle taking into account also the humerus heads elevation and the retraction due to the scapulas and the clavicles motions. The model exploits inertial sensors measurements by means of an Unscented Kalman filter to reconstruct human movements. The system performance is validated firstly against a reconstruction based on an optical tracking system. Secondly, the 5 DoFs model extracted form the 7 DoFs one was checked to have state of the art performance and used to estimate the improvement of position estimation that are obtained by extending the model to 7 DoFs.

Immersive ROS-integrated framework for robot teleoperation

The development of natural interfaces for human-robot interaction provides the user an intuitive way to control and guide robots. In this paper, we propose a novel ROS (Robot Operating System)-integrated interface for remote control that allows the user to teleoperate the robot using his hands motion. The user can adjust online the autonomy of the robot between two levels: direct control and waypoint following. The hand tracking and gestures recognition capabilities of the Leap Motion device are exploited to generate the control commands. The user receives a real-time 3D augmented visual feedback using a Kinect sensor and a HMD. To assess the practicability of the system experimental results are presented using as a benchmark the remote control of a Kuka Youbot.

Sensor fusion for complex articulated body tracking applied in rowing

This paper presents a sensor fusion model for integrating wearable inertial measures with sensors in the environment. This approach is designed and tested to support body motion tracking of rowing in indoor and outdoor environments. This paper presents the approach based on a complex kinematic model and Unscented Kalman filtering. The approach is validated in an indoor setup based on the SPRINT rowing system by comparison against results obtained from a commercial motion capture system, thus providing future directions for the assessment of rowers’ performance on an instrumented boat.

A novel approach to motion tracking with wearable sensors based on Probabilistic Graphical Models

Wearable motion tracking systems represent a breakthrough in ecological motion tracking. Their effectiveness has been proved in many fields, from performance assessment to human-robot interaction. Most of the approaches are based on the exploitation of optimal probabilistic filtering of inertial motion units (IMUs) signals, ranging from linear Kalman Filters (KF) to Particle filters (PF). Since most of the models are highly nonlinear, filters such as Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF) are typically used. These approaches cause all the variables of the models to be correlated each other. Probabilistic Graphical Models (PGM) are a framework for probabilistic reasoning that allows to explicitly declare the actual dependencies among variables. In this paper we propose a novel algorithm for motion tracking with IMUs based on PGM. The model is compared to the state of the art UKF algorithm in tracking the human upper limb. The results show that the proposed approach perform a slightly better compared to the UKF.

Assessment of task ergonomics with an upper limb wearable device

Upper Limb Work-related Musculo Skeletal Disorders (ULWMSD) are constantly increasing every year in developed countries. It is estimated that in Italy, in 2007 ULWMSD were the 41,6% of all the work-related pathologies. In this context, the importance to correctly diagnose and treat this kind of pathology is growing. Traditionally the assessment is done using pen-and-paper observational techniques in which movements are manually classified, labelled and compared integrating the results with subjective questionnaires given to the monitored subjects. The main problem with those traditional methods is the lack of objective assessment regarding the motion and the forces exerted, which are inferred by subjects inquiry and the body posture, manually extracted from video tapes. In this context we propose a novel wired system for assessing the muscular effort and posture of the human upper limb for ULWMSDs diagnosis in ecologic environment. The system is composed of inertial units to reconstruct the upper limb posture and EMG sensors to assess the muscle effort. The upper limb is considered as a kinematic chain comprising three degrees of freedom (DoFs) for the shoulder, two DoFs for the elbow and two DoFs for the wrist, while forearm flexor muscles are monitored through EMG. We propose a preliminary validation of the system testing it for assessing posture and muscle effort of a check-out operator during everyday real-life operations.

Stacked generalization for scene analysis and object recognition

The problem of object recognition and detection has been largely addressed by the robotics community, since its importance both in mapping and manipulation problems. One possible approach for the recognition task is to assume a specific a-priori knowledge of the objects possibly present in a scene. In this framework, this paper presents a novel technique for object detection and recognition based on Stacked Generalization (SG) method developed by Wolpert in 1992. The innovation of the proposed technique is the introduction of SG classification method to perform a multi-layer object recognition fusing heterogeneous spatial and color data acquired with an RGB-D camera. To improve the accuracy and the robustness of the system to environmental variability, we introduce a second layer classifier. Its goal is to evaluate and weights the results of the first layer classifiers, thus combining and improving the overall classification performance. This technique has a low computational cost and is suitable for on-line applications, such as robotic manipulation or automated logistic systems. To validate the presented approach experimental tests have been carried out and results are reported.

Augmented reality-aided tele-presence system for robot manipulation in industrial manufacturing

This work investigates the use of a highly immersive telepresence system for industrial robotics. A Robot Operating System integrated framework is presented where a remote robot is controlled through operators movements and muscle contractions captured with a wearable device. An augmented 3D visual feedback is sent to the user providing the remote environment scenario from the robots point of view and additional information pertaining to the task execution. The system proposed, using robot mounted RGB-D camera, identifies known objects and relates their pose to robot arm pose and to targets relevant to the task execution. The system is preliminary validated during a pick-and-place task using a Baxter robot. The experiment shows the practicability and the effectiveness of the proposed approach.

Characterization of the disruption of neural control strategies for dynamic fingertip forces from attractor reconstruction

The Strength-Dexterity (SD) test measures the ability of the pulps of the thumb and index finger to compress a compliant and slender spring prone to buckling at low forces (<3N). We know that factors such as aging and neurodegenerative conditions bring deteriorating physiological changes (e.g., at the level of motor cortex, cerebellum, and basal ganglia), which lead to an overall loss of dexterous ability. However, little is known about how these changes reflect upon the dynamics of the underlying biological system. The spring-hand system exhibits nonlinear dynamical behavior and here we characterize the dynamical behavior of the phase portraits using attractor reconstruction. Thirty participants performed the SD test: 10 young adults, 10 older adults, and 10 older adults with Parkinson’s disease (PD). We used delayed embedding of the applied force to reconstruct its attractor. We characterized the distribution of points of the phase portraits by their density (number of distant points and interquartile range) and geometric features (trajectory length and size). We find phase portraits from older adults exhibit more distant points (p = 0.028) than young adults and participants with PD have larger interquartile ranges (p = 0.001), trajectory lengths (p = 0.005), and size (p = 0.003) than their healthy counterparts. The increased size of the phase portraits with healthy aging suggests a change in the dynamical properties of the system, which may represent a weakening of the neural control strategy. In contrast, the distortion of the attractor in PD suggests a fundamental change in the underlying biological system, and disruption of the neural control strategy. This ability to detect differences in the biological mechanisms of dexterity in healthy and pathological aging provides a simple means to assess their disruption in neurodegenerative conditions and justifies further studies to understand the link with the physiological changes.

A ROS-integrated architecture to learn manipulation tasks from a single demonstration

In the robot programming by demonstration (PbD) framework, the high-level representation of a skill in a series of action units gives an intuitive method to program and instruct robot behaviors. In this context we present a ROS (Robot Operating System) integrated architecture for learning households manipulation tasks by one observation. The user is observed during the execution of everyday tasks, every action is analyzed and its effect is translated into changes in the environment state. During the observation a strip-like map of the task is built and stored as a sequence of actions. From the map obtained the task can be performed. A planner robustly adapts the execution both to different environment initial conditions and to possible faults, occurring during the operations. We test the capability of the chosen approach to autonomously learn and robustly perform complex tasks, such as setting up and clearing a table in a real kitchen-like environment.

Effects of Augmented Reality on the Performance of Teleoperated Industrial Assembly Tasks in a Robotic Embodiment

Teleoperation in robotic embodiments allows operators to perform and program manipulation tasks with better accuracy, dexterity, and visualization than what is possible with traditional human–robot interaction paradigms. However, the perception of cues (e.g., egocentric distances) relevant to task execution, is known to be distorted in virtual environments due to many factors, which can be grouped into technical, human, and methodological categories. This phenomenon becomes more pronounced in a low-cost/encumbrance setup, where the dynamic environment is captured with color and depth (RGB-D) cameras and presented in a virtual environment. In this paper, the effects of augmented reality (AR) are evaluated as a tool to deliver additional information, which helps in overcoming the differences in perception between telepresence and actual presence. The AR feedback is used to improve the embodiment illusion and to guide the operator during task execution. The AR setup, comprising an RGB-D camera and a head-mounted display, is integrated with the Baxter robot and evaluated by involving 22 participants in an experiment while they execute a pick-and-place task, taking into account their expertise in AR/virtual reality (VR) and gaming. The use of AR results in enhancing the accuracy and efficiency of the task performance, besides significantly reducing the effect of the differences in skillfulness between the participants. Furthermore, it is found that the sense of presence and embodiment for the participant is positively affected by different types of AR.