Howard Leung

A Virtual Reality Dance Training System Using Motion Capture Technology

In this paper, a new dance training system based on the motion capture and virtual reality (VR) technologies is proposed. Our system is inspired by the traditional way to learn new movements-imitating the teachers movements and listening to the teachers feedback. A prototype of our proposed system is implemented, in which a student can imitate the motion demonstrated by a virtual teacher projected on the wall screen. Meanwhile, the students motions will be captured and analyzed by the system based on which feedback is given back to them. The result of user studies showed that our system can successfully guide students to improve their skills. The subjects agreed that the system is interesting and can motivate them to learn.

A Feedback Controller for Biped Humanoids that Can Counteract Large Perturbations During Gait

In this paper, we propose a new method for biped humanoids to compensate for large amounts of angular momentum induced by strong external perturbations applied to the body during gait motion. Such angular momentum can easily cause the humanoid to fall down onto the ground. We use an Angular Momentum inducing inverted Pendulum Model (AMPM), which is an enhanced version of the 3D linear inverted pendulum model to model the robot dynamics. Because the AMPM allows us to explicitly calculate the angular momentum generated by the ground reaction force, it is possible to calculate a counteracting motion that compensates for the angular momentum generated by external perturbations in real-time.

Simulating pathological gait using the enhanced linear inverted pendulum model

In this paper, we propose a new method to simulate human gait motion when muscles are weakened. The method is based on the enhanced version of three-dimensional linear inverted pendulum model that is used for generation of gait in robotics. After the normal gait motion is generated by setting the initial posture and the parameters that decide the trajectories of the center of mass and angular momentum, the muscle to be weakened is specified. By minimizing an objective function based on the force exerted by the specified muscle during the motion, the set of parameters that represent the pathological gait was calculated. Since the number of parameters to describe the motion is small in our method, the optimization process converges much more quickly than in previous methods. The effects of weakening the gluteus medialis, the gluteus maximus, and vastus were analyzed. Important similarities were noted when comparing the predicted pendulum motion with data obtained from an actual patient.

Automatic panel extraction of color comic images

In this paper, an automatic approach for detecting and extracting panels in a color comic image is proposed. Panel extraction is challenging because the background color, the background pixel locations, the panel shapes and the panel layout are not known in advance. In our approach, uniform color stripes are first identified and used as separators to segment the color comic page image into sub-regions in a recursive manner. Panels are recognized as the sub-regions that cannot be further segmented. The structure of the panels is thus obtained in the extraction process and it contains the layout of the panels as well as the reading order. Panel extraction is useful because: 1) the extracted panels can be better fitted into a handheld device for viewing; and 2) the panels can then be further analyzed to extract features used for content based indexing and retrieval.

Animating reactive motions for biped locomotion

In this paper, we propose a new method for simulating reactive motions for running or walking human figures. The goal is to generate realistic animations of how humans compensate for large external forces and maintain balance while running or walking. We simulate the reactive motions of adjusting the body configuration and altering footfall locations in response to sudden external disturbance forces on the body. With our proposed method, the user first imports captured motion data of a run or walk cycle to use as the primary motion. While executing the primary motion, an external force is applied to the body. The system automatically calculates a reactive motion for the center of mass and angular momentum around the center of mass using an enhanced version of the linear inverted pendulum model. Finally, the trajectories of the generalized coordinates that realize the precalculated trajectories of the center of mass, zero moment point, and angular momentum are obtained using constrained inverse kinematics. The advantage of our method is that it is possible to calculate reactive motions for bipeds that preserve dynamic balance during locomotion, which was difficult using previous techniques. We demonstrate our results on an application that allows a user to interactively apply external perturbations to a running or walking virtual human model. We expect this technique to be useful for human animations in interactive 3D systems such as games, virtual reality, and potentially even the control of actual biped robots.

e-learning martial arts

Traditionally, when people want to learn martial arts, they have to go to training clubs and learn under the coach together with the other students. To reduce the tuition fees, there are usually a lot of students under a single coach and hence, it is difficult for the students to get enough suggestions in the class. It would be far easier if the students could practice themselves at home and ask for suggestions from a virtual coach in the computer. Occasionally, in case they find difficulties going to the next step, they may then approach the real coach for suggestions and training. In this paper, we propose such a training system based on the motion capture system. The system automatically analyzes the motions of the player and gives suggestions. The students can also view the martial art techniques stored in the system or their own techniques captured by the motion capture system from various points of view in order to gain objective ideas of the techniques.

Model-based analysis of Chinese calligraphy images

Chinese fonts with smooth outlines and solid colouring have been produced for computer displays and printings for a long time. However, the aesthetic properties of the characters produced by calligraphers could not be simulated with these methods. Ip and Wong proposed a parameterised brush model that enables efficient generation of Chinese calligraphic writings such that the rendering is scalable in resolution and it allows high quality publishing. While this graphical model facilitates the synthesis of calligraphy writing given a set of writing parameters, for the inverse problem, a lot of user stroke manipulation is required to regenerate the model parameters given a calligraphic image. Consequently, an intelligent method is required to automatically determine the model parameters from images of Chinese calligraphy. This paper describes a methodology for automatically estimating the set of 3D geometric and dynamic writing parameters along a stroke trajectory from images of calligraphic writings.

Chinese handwriting performance of primary school children with dyslexia

The aim of this study was to investigate the Chinese handwriting performance of typical children and children with dyslexia, and to examine whether speed and accuracy of handwriting could reliably discriminate these two groups of children. One hundred and thirty-seven children with dyslexia and 756 typical children were recruited from main stream primary schools for the study. They were requested to copy 90 Chinese characters using the Chinese Handwriting Assessment Tool (CHAT) jointly developed by a project team from two universities in Hong Kong. The process of handwriting was recorded and the stroke errors in writing were analyzed using the CHAT system. Results indicated that children with dyslexia wrote significantly slower, with greater average character size and variation in size (p<.05) than the typical children of same age group. They also wrote with significantly lower accuracy (p<.05). Commonly observed writing errors among the Dyslexic group were missing strokes and concatenated strokes. From the discriminant analysis, it was found that writing speed and accuracy were satisfactory discriminators that could discriminate students into the two groups, with reasonably good classification accuracy of over 70% for every grade. The results were discussed with theoretical implications in relation to fine motor skills, kinesthetic abilities, visual perceptual skills, and the demand of written tasks in school.

Interactive partner control in close interactions for real-time applications

This article presents a new framework for synthesizing motion of a virtual character in response to the actions performed by a user-controlled character in real time. In particular, the proposed method can handle scenes in which the characters are closely interacting with each other such as those in partner dancing and fighting. In such interactions, coordinating the virtual characters with the human player automatically is extremely difficult because the system has to predict the intention of the player character. In addition, the style variations from different users affect the accuracy in recognizing the movements of the player character when determining the responses of the virtual character. To solve these problems, our framework makes use of the spatial relationship-based representation of the body parts called interaction mesh, which has been proven effective for motion adaptation. The method is computationally efficient, enabling real-time character control for interactive applications. We demonstrate its effectiveness and versatility in synthesizing a wide variety of motions with close interactions.

A web-based chinese handwriting education system with automatic feedback and analysis

We propose a web-based Chinese handwriting system that allows students to learn Chinese handwriting at anytime and anywhere. The system combines technologies such as electronic ink, database, Flash animations, and pattern recognition technique to provide students with an intelligent and effective learning environment. In particular, our system contains an Automatic Feedback and Analysis (AFA) tool that can check multiple kinds of handwriting mistakes and provide useful feedback to the user. Survey results show that most users agreed that our proposed system can help them to learn Chinese handwriting more efficiently as well as reducing teachers’ workload. The user evaluation results suggest that it takes less time for users to learn Chinese handwriting with our system than with traditional teaching method.