Taku Komura

A Virtual Reality Dance Training System Using Motion Capture Technology

In this paper, a new dance training system based on the motion capture and virtual reality (VR) technologies is proposed. Our system is inspired by the traditional way to learn new movements-imitating the teachers movements and listening to the teachers feedback. A prototype of our proposed system is implemented, in which a student can imitate the motion demonstrated by a virtual teacher projected on the wall screen. Meanwhile, the students motions will be captured and analyzed by the system based on which feedback is given back to them. The result of user studies showed that our system can successfully guide students to improve their skills. The subjects agreed that the system is interesting and can motivate them to learn.

Computing inverse kinematics with linear programming

Inverse Kinematics (IK) is a popular technique for synthesizing motions of virtual characters. In this paper, we propose a Linear Programming based IK solver (LPIK) for interactive control of arbitrary multibody structures. There are several advantages of using LPIK. First, inequality constraints can be handled, and therefore the ranges of the DOFs and collisions of the body with other obstacles can be handled easily. Second, the performance of LPIK is comparable or sometimes better than the IK method based on Lagrange multipliers, which is known as the best IK solver today. The computation time by LPIK increases only linearly proportional to the number of constraints or DOFs. Hence, LPIK is a suitable approach for controlling articulated systems with large DOFs and constraints for real-time applications.

Spatial relationship preserving character motion adaptation

This paper presents a new method for editing and retargeting motions that involve close interactions between body parts of single or multiple articulated characters, such as dancing, wrestling, and sword fighting, or between characters and a restricted environment, such as getting into a car. In such motions, the implicit spatial relationships between body parts/objects are important for capturing the scene semantics. We introduce a simple structure called an interaction mesh to represent such spatial relationships. By minimizing the local deformation of the interaction meshes of animation frames, such relationships are preserved during motion editing while reducing the number of inappropriate interpenetrations. The interaction mesh representation is general and applicable to various kinds of close interactions. It also works well for interactions involving contacts and tangles as well as those without any contacts. The method is computationally efficient, allowing real-time character control. We demonstrate its effectiveness and versatility in synthesizing a wide variety of motions with close interactions.

A Feedback Controller for Biped Humanoids that Can Counteract Large Perturbations During Gait

In this paper, we propose a new method for biped humanoids to compensate for large amounts of angular momentum induced by strong external perturbations applied to the body during gait motion. Such angular momentum can easily cause the humanoid to fall down onto the ground. We use an Angular Momentum inducing inverted Pendulum Model (AMPM), which is an enhanced version of the 3D linear inverted pendulum model to model the robot dynamics. Because the AMPM allows us to explicitly calculate the angular momentum generated by the ground reaction force, it is possible to calculate a counteracting motion that compensates for the angular momentum generated by external perturbations in real-time.

Relationship descriptors for interactive motion adaptation

This paper presents an interactive motion adaptation scheme for close interactions between skeletal characters and mesh structures, such as moving through restricted environments, and manipulating objects. This is achieved through a new spatial relationship-based representation, which describes the kinematics of the body parts by the weighted sum of translation vectors relative to points selectively sampled over the surfaces of the mesh structures. In contrast to previous discrete representations that either only handle static spatial relationships, or require offline, costly optimization processes, our continuous framework smoothly adapts the motion of a character to large updates of the mesh structures and character morphologies on-the-fly, while preserving the original context of the scene. The experimental results show that our method can be used for a wide range of applications, including motion retargeting, interactive character control and deformation transfer for scenes that involve close interactions. Our framework is useful for artists who need to design animated scenes interactively, and modern computer games that allow users to design their own characters, objects and environments.

Creating and Retargetting Motion by the Musculoskeletal Human Body Model

Recently, optimization has been used in various ways to interpolate or retarget human body motions obtained by motion-capturing systems. However, in such cases, the inner structure of a human body has rarely been taken into account, and hence there have been difficulties in simulating physiological effects such as fatigue or injuries. In this paper, we propose a method to create/retarget human body motions using a musculoskeletal human body model. Using our method, it is possible to create dynamically and physiologically feasible motions. Since a muscle model based on Hills model is included in our system, it is also possible to retarget the original motion by changing muscular parameters. For example, using the muscle fatigue model, a motion where a human body gradually gets tired can be simulated. By increasing the maximal force exertable by the muscles, or decreasing it to zero, training or displacement effects of muscles can also be simulated. Our method can be used for biomechanically correct inverse kinematics, interpolation of motions, and physiological retargetting of the human body motion.

Simulating pathological gait using the enhanced linear inverted pendulum model

In this paper, we propose a new method to simulate human gait motion when muscles are weakened. The method is based on the enhanced version of three-dimensional linear inverted pendulum model that is used for generation of gait in robotics. After the normal gait motion is generated by setting the initial posture and the parameters that decide the trajectories of the center of mass and angular momentum, the muscle to be weakened is specified. By minimizing an objective function based on the force exerted by the specified muscle during the motion, the set of parameters that represent the pathological gait was calculated. Since the number of parameters to describe the motion is small in our method, the optimization process converges much more quickly than in previous methods. The effects of weakening the gluteus medialis, the gluteus maximus, and vastus were analyzed. Important similarities were noted when comparing the predicted pendulum motion with data obtained from an actual patient.

A deep learning framework for character motion synthesis and editing

We present a framework to synthesize character movements based on high level parameters, such that the produced movements respect the manifold of human motion, trained on a large motion capture dataset. The learned motion manifold, which is represented by the hidden units of a convolutional autoencoder, represents motion data in sparse components which can be combined to produce a wide range of complex movements. To map from high level parameters to the motion manifold, we stack a deep feedforward neural network on top of the trained autoencoder. This network is trained to produce realistic motion sequences from parameters such as a curve over the terrain that the character should follow, or a target location for punching and kicking. The feedforward control network and the motion manifold are trained independently, allowing the user to easily switch between feedforward networks according to the desired interface, without re-training the motion manifold. Once motion is generated it can be edited by performing optimization in the space of the motion manifold. This allows for imposing kinematic constraints, or transforming the style of the motion, while ensuring the edited motion remains natural. As a result, the system can produce smooth, high quality motion sequences without any manual pre-processing of the training data.

The dynamic postural adjustment with the quadratic programming method

The postural balance system is one of the most fundamental functions for humanoid robot control. In this paper, we propose a new feedback balance control system for the human body. This system can manipulate large perturbations. It finds the optimal motion for maintaining balance in the 3D space without receiving any feed-forward input beforehand. Two different strategies are adopted for the optimization: the quadratic programming method and the PD control. Simulation results are compared with real human motion; many common features such as rotating arms are observed.

Interaction patches for multi-character animation

We propose a data-driven approach to automatically generate a scene where tens to hundreds of characters densely interact with each other. During off-line processing, the close interactions between characters are precomputed by expanding a game tree, and these are stored as data structures called interaction patches. Then, during run-time, the system spatio-temporally concatenates the interaction patches to create scenes where a large number of characters closely interact with one another. Using our method, it is possible to automatically or interactively produce animations of crowds interacting with each other in a stylized way. The method can be used for a variety of applications including TV programs, advertisements and movies.