Yoshihiko Nakamura

Statistical mutual conversion between whole body motion primitives and linguistic sentences for human motions

This paper describes a novel approach to linguistic mutual inference, which enables robots not only to linguistically interpret the motion patterns in the form of sentences but also to generate the motions from the sentences. The inference can be established based on two modules, the motion language model and the natural language model. The motion language model stochastically represents an association structure between symbols of motion patterns and the words in sentences assigned to the motion. This is a statistical model with a three layered structure of motion symbols, latent states and words. The natural language model statistically represents a structure of sentences based on word bigrams. The motion language model and the natural language model correspond to semantics and syntax respectively. An approach to the integration of motion language model with the natural language model allows the linguistic mutual inference for the robots. The two kinds of inference can be made by solving search problems, search for a sequence of words corresponding to a motion and search for a symbol of motion pattern corresponding to a sentence. The proposed approach to interpretation of motion patterns as sentences and generation of motion patterns from the sentences through the integration of motion language model with the natural language model is validated by an experiment on the human behavioral data.

Leveraging Cone Double Description for Multi-contact Stability of Humanoids with Applications to Statics and Dynamics

We build on previous works advocating the use of the Gravito-Inertial Wrench Cone (GIWC) as a general contact stability criterion (a “ZMP for non-coplanar contacts”). We show how to compute this wrench cone from the friction cones of contact forces by using an intermediate representation, the surface contact wrench cone, which is the minimal representation of contact stability for each surface contact. The observation that the GIWC needs to be computed only once per stance leads to particularly efficient algorithms, as we illustrate in two important problems for humanoids: “testing robust static equilibrium” and “time-optimal path parameterization”. We show, through theoretical analysis and in physics simulations, that our method is more general and/or outperforms existing ones.

Stability of surface contacts for humanoid robots: Closed-form formulae of the Contact Wrench Cone for rectangular support areas

Humanoids locomote by making and breaking contacts with their environment. Thus, a crucial question for them is to anticipate whether a contact will hold or break under effort. For rigid surface contacts, existing methods usually consider several point-contact forces, which has some drawbacks due to the underlying redundancy. We derive a criterion, the Contact Wrench Cone (CWC), which is equivalent to any number of applied forces on the contact surface, and for which we provide a closed-form formula. It turns out that the CWC can be decomposed into three conditions: (i) Coulomb friction on the resultant force, (ii) CoP inside the support area, and (iii) upper and lower bounds on the yaw torque. While the first two are well-known, the third one is novel. It can, for instance, be used to prevent the undesired foot yaws observed in biped locomotion. We show that our formula yields simpler and faster computations than existing approaches for humanoid motions in single support, and assess its validity in the OpenHRP simulator.

Symbolically structured database for human whole body motions based on association between motion symbols and motion words

Motion capture systems have been commonly used to enable humanoid robots or CG characters to perform human-like motions. However, prerecorded motion capture data cannot be reused efficiently because picking a specific motion from a large database and modifying the motion data to fit the desired motion patterns are difficult tasks. We have developed an imitative learning framework based on the symbolization of motion patterns using Hidden Markov Models (HMMs), where each HMM (hereafter referred to as motion symbol) abstracts the dynamics of a motion pattern and allows motion recognition and generation. This paper describes a symbolically structured motion database that consists of original motion data, motion symbols, and motion words. Each motion data is labeled with motion symbols and motion words. Moreover, a network is formed between two layers of motion symbols and motion words based on their probability association. This network makes it possible to associate motion symbols with motion words and to search for motion datasets using motion symbols. The motion symbols can also generate motion data. Therefore, the developed framework can provide the desired motion data when only the motion words are input into the database. This paper proposes a database of human motions and their descriptive words.The stochastic model constructs the mapping between the motions and words.The database can be applied to the retrieval of the motion data from the word.

Modeling and Identification of a Realistic Spiking Neural Network and Musculoskeletal Model of the Human Arm, and an Application to the Stretch Reflex

This study develops a multi-level neuromuscular model consisting of topological pools of spiking motor, sensory and interneurons controlling a bi-muscular model of the human arm. The spiking output of motor neuron pools were used to drive muscle actions and skeletal movement via neuromuscular junctions. Feedback information from muscle spindles were relayed via monosynaptic excitatory and disynaptic inhibitory connections, to simulate spinal afferent pathways. Subject-specific model parameters were identified from human experiments by using inverse dynamics computations and optimization methods. The identified neuromuscular model was used to simulate the biceps stretch reflex and the results were compared to an independent dataset. The proposed model was able to track the recorded data and produce dynamically consistent neural spiking patterns, muscle forces and movement kinematics under varying conditions of external forces and co-contraction levels. This additional layer of detail in neuromuscular models has important relevance to the research communities of rehabilitation and clinical movement analysis by providing a mathematical approach to studying neuromuscular pathology.

Low-friction tendon-driven robot hand with carpal tunnel mechanism in the palm by optimal 3D allocation of pulleys

Underactuated hands usually have high adaptability in power grasping but they are limited in pinching task with fingertip. In this paper, we propose the design of a tendon-driven underactuated hand that is capable of fingertip pinching by utilizing our proposed coupling mechanism. To reduce the friction resulting from tendon routing, we introduce the carpal tunnel mechanism that replace all sliding-contact tendon routing with the pulley system allocated in 3-dimensional space. The prototype of 11-DOF anthropomorphic robot hand is fabricated using rapid prototyping. Experiments are done to prove the effectiveness of our proposed coupling mechanism and low-friction tendon-driven system for underactuated robot hand.

Construction of a space of motion labels from their mapping to full-body motion symbols

Language is an indispensable for humanoid robot to be integrated into daily life. This paper proposes a novel approach to construct a space of motion labels from their mapping to human whole body motions. The motions are abstracted by Hidden Markov Models, which are referred to as motion symbols. The human motions are automatically partitioned into motion segments, and recognized as sequences of the motion symbols. Sequences of motion labels are also assigned to these motions. The referential relationship between the motion symbols and the motion labels is extracted by stochastic translation model, and distances among the labels are calculated from the association probability of the motion symbols being generated by the labels. The labels are located in a multidimensional space so that the distances are satisfied, and it results in a label space. The label space encapsulates relations among the motion labels such as their similarities. The label space also allows motion recognition. The validity of the constructed label space is demonstrated on a motion capture data-set. Graphical Abstract

Online deformation of optimal trajectories for constrained nonprehensile manipulation

This paper discusses an online dynamic motion generation scheme for nonprehensile object manipulation by using a set of predefined motions and a trajectory deformation algorithm capable of incorporating positional and velocity boundary constraints. By creating optimal trajectories offline and deforming them online, computational complexity during execution is reduced considerably. As tight convex hulls of the deformed trajectories can be found, possible obstacles or workspace boundaries can be circumnavigated precisely without collision. The approach is verified through experiments on an inclined planar air-table for volleyball scenario using two 3-DoF robots.

Gesture recognition using hybrid generative-discriminative approach with Fisher Vector

Gesture recognition is used for many practical applications such as human-robot interaction, medical rehabilitation and sign language. In this paper, we apply a hybrid generative-discriminative approach by using the Fisher Vector to improve the recognition performance. The strategy is to merge the generative approach of Hidden Markov Model dealing with spatio-temporal motion data with the discriminative approach of Support Vector Machine focusing on the classification task. The motion segments are encoded into HMMs, and each segment is converted to FV, whose elements can be obtained as the derivative of the probability of the segment being generated by the HMMs with respect to their parameters. SVM is subsequently trained by the FVs. An input gesture can be classified to corresponding gesture category by SVM. In the experiments, we test our approach by comparing three HMM chain models and four categorization methods on dataset provided by the ChaLearn Looking at People Challenge 2014 (LAP 2014). The results show that similar gesture patterns are clustered closely in several categories. Our approach based left-to-right HMMs outperforms other gesture recognition methods. More specifically, the hybrid generative-discriminative approach overcomes the standard HMM approach and the generative kernel approach overcomes the generative embedding approach. For these results, our approach is effective to improve the recognition performance.

A New Trajectory Deformation Algorithm Based on Affine Transformations

We propose a new approach to deform robot trajectories based on affine transformations. At the heart of our approach is the concept of affine invariance: Trajectories are deformed in order to avoid unexpected obstacles or to achieve new objectives but, at the same time, certain definite features of the original motions are preserved. Such features include, for instance, trajectory smoothness, periodicity, affine velocity, or more generally, all affine-invariant features, which are of particular importance in human-centered applications. Furthermore, this approach enables one to “convert” the constraints and optimization objectives regarding the deformed trajectory into constraints and optimization objectives regarding the matrix of the deformation in a natural way, making constraints satisfaction and optimization substantially easier and faster in many cases. As illustration, we present an application to the transfer of human movements to humanoid robots while preserving equiaffine velocity, a well-established invariant of human hand movements. Building on the presented affine deformation framework, we finally revisit the concept of trajectory redundancy from the viewpoint of group theory.