Henk G. Kortier

Assessment of hand kinematics using inertial and magnetic sensors

BackgroundAssessment of hand kinematics is important when evaluating hand functioning. Major drawbacks of current sensing glove systems are lack of rotational observability in particular directions, labour intensive calibration methods which are sensitive to wear and lack of an absolute hand orientation estimate.MethodsWe propose an ambulatory system using inertial sensors that can be placed on the hand, fingers and thumb. It allows a full 3D reconstruction of all finger and thumb joints as well as the absolute orientation of the hand. The system was experimentally evaluated for the static accuracy, dynamic range and repeatability.ResultsThe RMS position norm difference of the fingertip compared to an optical system was 5±0.5 mm (mean ± standard deviation) for flexion-extension and 12.4±3.0 mm for combined flexion-extension abduction-adduction movements of the index finger. The difference between index and thumb tips during a pinching movement was 6.5±2.1 mm. The dynamic range of the sensing system and filter was adequate to reconstruct full 80 degrees movements of the index finger performed at 116 times per minute, which was limited by the range of the gyroscope. Finally, the reliability study showed a mean range difference over five subjects of 1.1±0.4 degrees for a flat hand test and 1.8±0.6 degrees for a plastic mold clenching test, which is smaller than other reported data gloves.ConclusionCompared to existing data gloves, this research showed that inertial and magnetic sensors are of interest for ambulatory analysis of the human hand and finger kinematics in terms of static accuracy, dynamic range and repeatability. It allows for estimation of multi-degree of freedom joint movements using low-cost sensors.

Hand Pose Estimation by Fusion of Inertial and Magnetic Sensing Aided by a Permanent Magnet

Tracking human body motions using inertial sensors has become a well-accepted method in ambulatory applications since the subject is not confined to a lab-bounded volume. However, a major drawback is the inability to estimate relative body positions over time because inertial sensor information only allows position tracking through strapdown integration, but does not provide any information about relative positions. In addition, strapdown integration inherently results in drift of the estimated position over time. We propose a novel method in which a permanent magnet combined with 3-D magnetometers and 3-D inertial sensors are used to estimate the global trunk orientation and relative pose of the hand with respect to the trunk. An Extended Kalman Filter is presented to fuse estimates obtained from inertial sensors with magnetic updates such that the position and orientation between the human hand and trunk as well as the global trunk orientation can be estimated robustly. This has been demonstrated in multiple experiments in which various hand tasks were performed. The most complex task in which simultaneous movements of both trunk and hand were performed resulted in an average rms position difference with an optical reference system of 19.7±2.2 mm whereas the relative trunk-hand and global trunk orientation error was 2.3±0.9 and 8.6±8.7 deg respectively.

Sensor Fusion for Robotic Workspace State Estimation

We consider the problem of tool position and orientation state estimation for robot manipulators in workspace by sensor fusion of the internal robot joint measurements with inertial measurement unit data. A prerequisite for this to be successful is accurate calibration of the sensors used. Therefore, we discuss a method for calibration of the sensor with respect to the robot end-effector, which is straightforward to apply on an arbitrary industrial manipulator. We also consider two different workspace state-estimation algorithms requiring a minimum of robot modeling; the first is based on the extended Kalman filter and the second is based on the Rao-Blackwellized particle filter. The calibration procedure and the state-estimation algorithms were evaluated and compared in extensive experiments. Both state-estimation algorithms exhibited an accuracy improvement compared to estimates provided by the forward kinematics of the robot. Moreover, both algorithms were shown to satisfy the constraints of real-time execution at 4-ms sampling period. To further evaluate and compare the robustness of the methods, the algorithms were investigated with respect to the sensitivity of the filter parameters and the noise modeling.

Measuring 3D hand and finger kinematics - a comparison between inertial sensing and an opto-electronic marker system

Objective analysis of hand and finger kinematics is important to increase understanding of hand function and to quantify motor symptoms for clinical diagnosis. The aim of this paper is to compare a new 3D measurement system containing multiple miniature inertial sensors (PowerGlove) with an opto-electronic marker system during specific finger tasks in three healthy subjects. Various finger movements tasks were performed: flexion, fast flexion, tapping, hand open/closing, ab/adduction and circular pointing. 3D joint angles of the index finger joints and position of the thumb and index were compared between systems. Median root mean square differences of the main joint angles of interest ranged between 3.3 and 8.4deg. Largest differences were found in fast and circular pointing tasks, mainly in range of motion. Smallest differences for all 3D joint angles were observed in the flexion tasks. For fast finger tapping, the thumb/index amplitude showed a median difference of 15.8mm. Differences could be explained by skin movement artifacts caused by relative marker movements of the marker system, particularly during fast tasks; large movement accelerations and angular velocities which exceeded the range of the inertial sensors; and by differences in segment calibrations between systems. The PowerGlove is a system that can be of value to measure 3D hand and finger kinematics and positions in an ambulatory setting. The reported differences need to be taken into account when applying the system in studies understanding the hand function and quantifying hand motor symptoms in clinical practice.

On-body inertial and magnetic sensing for assessment of hand and finger kinematics

A novel instrumentation is proposed to estimate hand and finger movements. Current dataglove systems often measure a reduced set of joint angles and lack a position and orientation measure of the hand with respect to the trunk. Our proposed system, based on inertial and magnetic sensing, is fully ambulatory, light weighted, has a low energy consumption and is therefore suitable to assess kinematics of the hand and fingers in daily life. Results showed an rms difference of the hand pose with an optical reference system of 16.9±4.0 mm for the position and 1.5±0.4 deg for the orientation. Index finger tip positions could be estimated with 5.0±0.5 mm for flexion extensions movements and 12.4±3.0 mm for more complex movements in which both flexion-extension and abduction-adduction movements were involved.

Load identification during object handling

In this paper a new concept to identify environmental loads during the interaction with the human body by sensing interface forces and movement is proposed. Mass and spring loads were moved by hand over a fixed height difference. Kinematic and kinetic quantities were measured between the hand and the load using an instrumented handle. Force was measured using a force transducer module, movement was measured using an accelerometer and rate gyroscope. Under the condition that the human body was actively generating force at the load, while the load was passive, the dynamic characteristics of the load could be estimated. The estimated parameter values were compared to their specified values and appeared to be accurate within 4% for both mass and spring loads.

Identification of Object Dynamics Using Hand Worn Motion and Force Sensors

Emerging microelectromechanical system (MEMS)-based sensors become much more applicable for on-body measurement purposes lately. Especially, the development of a finger tip-sized tri-axial force sensor gives the opportunity to measure interaction forces between the human hand and environmental objects. We have developed a new prototype device that allows simultaneous 3D force and movement measurements at the finger and thumb tips. The combination of interaction forces and movements makes it possible to identify the dynamical characteristics of the object being handled by the hand. With this device attached to the hand, a subject manipulated mass and spring objects under varying conditions. We were able to identify and estimate the weight of two physical mass objects (0.44 kg: 29.3%±18.9% and 0.28 kg: 19.7%±10.6%) and the spring constant of a physical spring object (16.3%±12.6%). The system is a first attempt to quantify the interactions of the hand with the environment and has many potential applications in rehabilitation, ergonomics and sports.