Jan Frederik Steffen

Experience-based and tactile-driven dynamic grasp control

Algorithms for dextrous robot grasping always have to cope with the challenge of achieving high object specialisation for a wide range of grasping contexts. In this paper, we present a tactile-driven approach that dynamically uses the robots grasping experience to address this issue. During the grasp movement, the current contact information is used to dynamically adapt the grasping control by targeting the best matching posture from the experience base. Thus, the robot recalls and actuates a grasp it already successfully performed in a similar tactile context. To efficiently represent the experience, we introduce the grasp manifold assuming that grasp postures form a smooth manifold in hand posture space. We present a simple way of providing approximations of grasp manifolds using self-organising maps (SOMs). The algorithm is evaluated on three different geometry primitives - box, cylinder and sphere - in a physics-based computer simulation.

Bio-inspired motion strategies for a bimanual manipulation task

We consider the complex task of coordinating two five-fingered anthropomorphic robot hands for taking a jar passed from a human user and unscrewing its cap. Using a pair of 7-DOF redundant arms for operating the hands, we study how the incorporation of human movement strategies at the finger and arm levels can aid in the solution of the overall bimanual task. At the finger level, we employ a finger control manifold for the unscrewing motion that has been synthesized with a kernel approach applied to human motion data captured with a data glove. At the arm level, we use a heuristic motivated from the observation of human arm movements to enhance the space of pass-over configurations that the system can successfully handle. In addition, we provide a brief description of the architecture of the overall system that comprises 54 motor degrees of freedom and integrates camera vision, arm and finger control as well as a speech output component for interaction with the human user.

Robust tracking of human hand postures for robot teaching

To enable the creation of manual interaction databases, aiding the replication of dexterous capabilities with anthropomorphic robot hands by utilizing information about how humans perform complex manipulation tasks, requires the capability to record and analyze large amounts of manual interaction sequences. For this goal we have studied and compared three mappings from captured human hand motion data to a simulated model, which allow for robust and accurate real-time hand posture tracking. We evaluate the effectiveness of these mappings and discuss their pros and cons in various real-world scenarios. The first method is based on data glove data and aims for direct gaging of hand joints. The other two methods utilize a VICON motion tracking system which monitors markers placed on all finger segments. Here we compare two approaches: a direct computation of hand postures from angles between adjacent markers and an iterative inverse kinematics approach to optimally reproduce fingertip positions. For a quantitative evaluation, we employ a “calibration objects” technique to obtain a reliable ground truth of task-relevant hand posture data.

Robust dataglove mapping for recording human hand postures

We present a novel dataglove mapping technique based on parameterisable models that handle both the cross coupled sensors of the fingers and thumb, and the under-specified abduction sensors for the fingers. Our focus is on realistically reproducing the posture of the hand as a whole, rather than on accurate fingertip positions. The method proposed in this paper is a vision-free, object free, data glove mapping and calibration method that has been successfully used in robot manipulation tasks.

Towards dextrous manipulation using manipulation manifolds

In dextrous manipulation, the implementation of manipulation movements still is a complex and intricate undertaking. Often, a lot of object physics and modelling effort has to be incorporated into a controller working only for a very restricted task specification and performing quite artificially looking movements. In this paper, we present the first steps towards a representation of manipulation movements recorded from human demonstration which facilitates later application and promotes natural motion. We use manifolds of hand postures embedded in the finger joint angle space which are constructed such that manipulation parameters including the advance in time are represented by distinct manifold dimensions. This allows for purposive navigation within such manifolds. We present the manifold construction using the Unsupervised Kernel Regression (UKR) and the way of applying it for manipulation in the example of turning a bottle cap in a physics-based simulation.

Using Structured UKR manifolds for motion classification and segmentation

Task learning from observations of non-expert human users will be a core feature of future cognitive robots. However, the problem of task segmentation has only received minor attention. In this paper, we present a new approach to classifying and segmenting series of observations into a set of candidate motions. As basis for these candidates, we use Structured UKR manifolds, a modified version of Unsupervised Kernel Regression which has been introduced in order to easily reproduce and synthesise represented dextrous manipulation tasks. Together with the presented mechanism, it then realises a system that is able both to reproduce and recognise the represented motions.

Towards Semi-supervised Manifold Learning: UKR with Structural Hints

We explore generic mechanisms to introduce structural hints into the method of Unsupervised Kernel Regression (UKR) in order to learn representations of data sequences in a semi-supervised way. These new extensions are targeted at representing a dextrous manipulation task. We thus evaluate the effectiveness of the proposed mechanisms on appropriate toy data that mimic the characteristics of the aimed manipulation task and thereby provide means for a systematic evaluation.

Structured unsupervised kernel regression for closed-loop motion control

Transferring human skills to dextrous robots in an easy, fast and robust way is one of the key challenges that still have to be tackled in order to bring robots to our every-day life. However, many problems remain unsolved. In particular, researchers are seeking new paradigms along with efficient and robust task representations that facilitate adaptation to new contexts and provide a means to appropriately react to unforeseen situations. In this paper, we present a new method for robot behaviour synthesis, where intrinsic characteristics of ‘Structured UKR manifolds’ [13] are used to derive a closed-loop controller based on motion data obtained by the ‘Robot Skill Synthesis via Human Learning’ paradigm [10]. We apply the method to the task of swapping Chinese health balls with a real 16 DOF robotic hand. Our results indicate that the marriage of ‘Structured UKR manifolds’ with the ‘Robot Skill Synthesis via Human Learning’ paradigm yields an efficient way of realising a dexterous manipulation capability on real robots.

A manifold representation as common basis for action production and recognition

In this paper, we first review our previous work in the domain of dextrous manipulation, where we introduced Manipulation Manifolds - a highly structured manifold representation of hand postures which lends itself to simple and robust manipulation control schemes. Coming from this scenario, we then present our idea of how this generative system can be naturally extended to the recognition and segmentation of the represented movements providing the core representation for a combined system for action production and recognition.

Integrating feature maps and competitive layer architectures for motion segmentation

We present a generic approach to integrate feature maps with a competitive layer architecture to enable segmentation by a competitive neural dynamics specified in terms of the latent space mappings constructed by the feature maps. We demonstrate the underlying ideas for the case of motion segmentation, using a system that employs Unsupervised Kernel Regression (UKR) for the creation of the feature maps, and the Competitive Layer Model (CLM) for the competitive layer architecture. The UKR feature maps hold learned representations of a set of candidate motions and the CLM dynamics, working on features defined in the UKR domain, implements the segmentation of observed trajectory data according to the competing candidates. We also demonstrate how the introduction of an additional layer can provide the system with a parametrizable rejection mechanism for previously unknown observations. The evaluation on trajectories describing four different letters yields improved classification results compared to our previous, pure manifold approach.