Jacky C. P. Chan

A Virtual Reality Dance Training System Using Motion Capture Technology

In this paper, a new dance training system based on the motion capture and virtual reality (VR) technologies is proposed. Our system is inspired by the traditional way to learn new movements-imitating the teachers movements and listening to the teachers feedback. A prototype of our proposed system is implemented, in which a student can imitate the motion demonstrated by a virtual teacher projected on the wall screen. Meanwhile, the students motions will be captured and analyzed by the system based on which feedback is given back to them. The result of user studies showed that our system can successfully guide students to improve their skills. The subjects agreed that the system is interesting and can motivate them to learn.

Interactive partner control in close interactions for real-time applications

This article presents a new framework for synthesizing motion of a virtual character in response to the actions performed by a user-controlled character in real time. In particular, the proposed method can handle scenes in which the characters are closely interacting with each other such as those in partner dancing and fighting. In such interactions, coordinating the virtual characters with the human player automatically is extremely difficult because the system has to predict the intention of the player character. In addition, the style variations from different users affect the accuracy in recognizing the movements of the player character when determining the responses of the virtual character. To solve these problems, our framework makes use of the spatial relationship-based representation of the body parts called interaction mesh, which has been proven effective for motion adaptation. The method is computationally efficient, enabling real-time character control for interactive applications. We demonstrate its effectiveness and versatility in synthesizing a wide variety of motions with close interactions.

Interactive dancing game with real-time recognition of continuous dance moves from 3D human motion capture

We have implemented an interactive dancing game using optical 3D motion capture technology. We propose a Progressive Block Matching algorithm to recognize the dance moves performed by the player in real-time. This makes a virtual partner be able to recognize and respond to the players movement without a noticeable delay. The completion progress of a move is tracked progressively and the virtual partners move is rendered in synchronization with the players current action. Our interactive dancing game contains moves with various difficulty levels that suits for both novices and skillful players. Through animating the virtual partner in response to the players movements, the player gets immersed into the virtual environment. A user test is performed to have a subjective evaluation of our game and the feedbacks from the subjects are positive.

Correlation Among Joint Motions Allows Classification of Parkinsonian Versus Normal 3-D Reaching

In this paper, an objective assessment for determining whether a person has Parkinson disease is proposed. This is achieved by analyzing the correlation between joint movements, since Parkinsonian patients often have trouble coordinating different joints in a movement. Thus, the auto-correlation coefficient of single joint movements and the cross-correlation between movements in a pair of joints (hand, wrist, elbow, and shoulder) were studied. These features were used to train and provide classification of subjects as having or not having Parkinsons disease using the least square support vector machine (LS-SVM). Experimental results showed that using either auto-correlation or cross-correlation features for classification provided over 91% correct classification. Using both features together provided better performance (96.0%) than using either feature alone. In addition, the performance of LS-SVM is better than that of self-organizing map (SOM) and k-nearest neighbor (KNN) in this case.

Interaction Retrieval by Spacetime Proximity Graphs

In this paper, we propose a new method to index and retrieve animation scenes in which multiple characters closely interact with one another. Such a technique can be an important tool for animators when they want to automatically extract the desired scene from a large database of animation sequence. Existing methods for single character movements do not scale well for multiple characters as they do not take into account the interaction of different body parts. In this paper, we propose a new distance function that computes the similarity of two‐character interations using the spatial relationship of the body parts. For each interaction, we produce a time‐varying graph structure based on the proximity of different joints, and compute the similarity of interactions by comparing the topology and Laplacian coordinates of the time‐varying graph. Experimental results show that the proposed method outperforms previous methods which are based on the kinematics of individual characters. The top retrieved samples are found similar in high level semantics while containing style variations.

Synthesizing Two-character Interactions by Merging Captured Interaction Samples with their Spacetime Relationships

Existing synthesis methods for closely interacting virtual characters relied on user-specified constraints such as the reaching positions and the distance between body parts. In this paper, we present a novel method for synthesizing new interacting motion by composing two existing interacting motion samples without the need to specify the constraints manually. Our method automatically detects the type of interactions contained in the inputs and determines a suitable timing for the interaction composition by analyzing the spacetime relationships of the input characters. To preserve the features of the inputs in the synthesized interaction, the two inputs will be aligned and normalized according to the relative distance and orientation of the characters from the inputs. With a linear optimization method, the output is the optimal solution to preserve the close interaction of two characters and the local details of individual character behavior. The output animations demonstrated that our method is able to create interactions of new styles that combine the characteristics of the original inputs.