Weidong Geng

Motion retrieval based on movement notation language

With the increased availability of motion capture data, the volume of motion library grows so large that it is difficult for animators to manually browse the dataset to search desired motions for reuse. To address this issue, we implement a framework, which allows the user to retrieve motions via Labanotation. For each motion clip in the library, we generate a corresponding Labanotation sequence as additional motion property. A similarity metric for Labanotation sequences is proposed and used to search the motions that have similar Laban descriptions. Our search algorithm is able to retrieve motion segments that only match part of the query Laban sequence. Then based on dynamic programming, these segments are stitched together to form a smooth output motion that is in an optimal sense of matching query Laban sequence. Experimental results demonstrate our method could effectively improve the utilization of motion data. Copyright

Mocap data editing via movement notations

In most motion editing approaches, the users often make changes on mocap data by specifying numerical parameters. However, it is not intuitive for a novice to edit motion sequences in motion planning tasks. In this paper, we present a method of notation-based motion editing, in which Labanotation, a well-developed notation language for human movements, is employed as the editing interface. It allows the user to specify the editing requirements via notation score, and the system will semi-automatically generate the desired motion data. The algorithmic steps of its pipeline and the core issues of converting Labanotation into motion sequences are discussed in detail. We show the results of our systems using martial arts motion as its testing data.

Motion-Based Perceptual User Interface

Perceptual user interface takes advantage of human perceptual capabilities in order to present semantical information in native and natural ways. In this paper, we present a novel approach to provide users with an accelerometer-based interface for interactively controlling not only functions or devices in digital environments but virtual characters in game-like scenarios. A general approach suitable for both PC and mobile platforms is proposed. Its core techniques include automatic generation and preprocessing of training samples and proper setup of machine learning models. Three sample applications are given: gesture-controlled lantern slide system with Wii Remote, gesture recognition system to make phone calls on Nokia N95, and performance-driven motion choreographing system with Xsens MTx. Experimental results show that the recognition rate is over 95% which is quite acceptable for gesture interaction systems.

Animating cartoon faces by multi-view drawings

Realistic character animation requires elaborate rigging built on top of high quality 3D models. Sophisticated anatomically based rigs are often the choice of visual effect studios where life-like animation of CG characters is the primary objective. However, rigging a character with a muscular-skeletal system is very involving and time-consuming process, even for professionals. Although, there have been recent research efforts to automate either all or some parts of the rigging process, the complexity of anatomically based rigging nonetheless opens up new research challenges. We propose a new method to automate anatomically based rigging that transfers an existing rig of one character to another. The method is based on a data interpolation in the surface and volume domain, where various rigging elements can be transferred between different models. As it only requires a small number of corresponding input feature points, users can produce highly detailed rigs for a variety of desired character with ease. Copyright

Implementation of a notation-based motion choreography system

Motion choreography is a design process of creating, structuring and forming body movements. We proposed and implemented a notation-based motion choreography system, LabanChoreographer version 0.1. Given the user specified movement notations, it will firstly perform Laban-based motion retrieval to find the most similar motions from a motion capture database notated with Labanotation, and snap together these retrieved motion clips into a continuous rough motion sequence that match the input Laban sequence as much as possible. Secondly, Laban-based motion editing algorithm is employed to semi-automatically revise and refine this candidate motion sequence such that the resulting motion data can best approximate the desired motion in the user’s mind. The choreographer is able to move flexibly back and forth between Labanotations and its rapid-prototyped motion data. It well supports the iterative and interactive development of movement ideas at a conceptual level.

Performance-driven animation of hand-drawn cartoon faces

We present a novel performance‐driven approach to animating cartoon faces starting from pure 2D drawings. A 3D approximate facial model automatically built from front and side view master frames of character drawings is introduced to enable the animated cartoon faces to be viewed from angles different from that in the input video. The expressive mappings are built by artificial neural network (ANN) trained from the examples of the real face in the video and the cartoon facial drawings in the facial expression graph for a specific character. The learned mapping model makes the resultant facial animation to properly get the desired expressiveness, instead of a mere reproduction of the facial actions in the input video sequence. Furthermore, the lit sphere, capturing the lighting in the painting artwork of faces, is utilized to color the cartoon faces in terms of the 3D approximate facial model, reinforcing the hand‐drawn appearance of the resulting facial animation. We made a series of comparative experiments to test the effectiveness of our method by recreating the facial expression in the commercial animation. The comparison results clearly demonstrate the superiority of our method not only in generating high quality cartoon‐style facial expressions, but also in speeding up the animation production of cartoon faces. Copyright

Motion retrieval based on movement notation language: Motion Capture and Retrieval

Not everything is perceived as it is provided by the environment. Depending on focus and attention perception can vary and therefore also the knowledge about the world. Virtual humans are sensing the virtual world, storing knowledge and using it to perform tasks. This paper describes our approach to model perceiving, storing and forgetting knowledge as the main regulation of tasks. We use different forms and levels of knowledge which can be independently adapted to different personalities and situations by combining computer graphics methods with psychological models. Copyright

A perceptual user interface for motion retrieval

In this paper, we present a novel approach that can intuitively and efficiently retrieve motions from large motion databases based on perceptual user interface. It aims at providing the easier motion retrieval interface for both the amateurs and skilled users by adopting motion-sensing techniques. A series of experiments have been set up to evaluate the usability of this system. The experimental results show that our system has significant advantages over other motion retrieval systems in terms of usability, especially characters of ease-of-use and user experience.