Xianjie Qiu

A novel framework for athlete training based on interactive motion editing and silhouette analysis

There are mainly two Hi-Tech methods for athlete training. One method is based on virtual reality, where the athlete can learn and improve performance mainly through using virtual equipments to interact with the virtual environment. Another method is based on video analysis, where improvements can be made by comparing the videos of the trainees with those of excellent trainers. In this paper, we present a novel framework for athlete training, which can circumvent difficulties the current methods faced in practical applications. For retargeting the example motion to personalized virtual athlete, the coach interactively sets motion constraints with his experience based on motion warping and motion verification techniques. The display of the simulated motion is adjusted semi-automatically to create the reference virtual video with the same viewpoint as the real one. The moment invariants of both virtual and real athletes silhouette are computed, and motion analysis result is presented subsequently. This method is more suitable for gymnastic athlete training because of without virtual equipment and more instructive having the same viewpoint in video analysis. Finally, an application of the proposed techniques to trampoline training is implemented.

Learning local models for 2D human motion tracking

We present a novel approach to tracking 2D human motion in uncalibrated monocular videos. Human motion usually exhibits time-varying patterns, and we propose to use locally learnt prior models to capture this characteristics. For each input image, our method automatically learns a local probability density model and a local dynamical model from a set of training examples that are close matches to the input. We evaluate the image likelihood by matching a deformable 2D human body model to the input images. The local models and the image likelihood are integrated to optimize the pose for the current input. Experiments on both synthetic and real videos demonstrate the effectiveness of our method.

Inferring 3d body pose from uncalibrated video

Recovery of 3D body pose is a fundamental problem for human motion analysis in many applications such as motion capture, vision interface, visual surveillance, and gesture recognition. In this paper, we present a new image-based approach to infer 3D human structure parameters from uncalibrated video. The estimation is example based. First, we acquire a special motion database through an off-line motion capture process. Second, given uncalibrated motion video, we abstract the extrinsic parameters and then silhouettes database associated with 3D poses is built by projecting each data of the 3D motion database into 2D plane with the extrinsic parameters. Next, with the image silhouettes abstracted from video, the unknown structure parameters are inferred by performs a similarity search in the database of silhouettes using approach based on shape matching. That is, the 3D structure parameters whose 2D projective silhouette is the most similar to the 2D image silhouette are took as the 3D reconstruction structure. We use trampoline sport motion, an example of complex human motion, to demonstrate the effectiveness of our approach.

Shape matching in pose reconstruction using shape context

Recovery of 3D body pose is a fundamental problem for human motion analysis in many applications. This paper introduces a new silhouette-based framework for inferring human pose from monocular uncalibrated camera and pays more attention on how to retrieving 3D body pose by matching 2D silhouette based on shape context. Finally we retrieve a group of 2D silhouettes from one human motion which has two different viewpoints. By many experiments, the rate of 2D silhouette matching we got is been proved more accurate than that generated by using other methods we have known in this field. To accelerate the matching process of calculating shape context, we use PCA (principal components analysis) to reduce the computation of complexity. In the paper we use trampoline and box sport as examples of complex human motion, to demonstrate the effectiveness of our approach and compare the results with those obtained with Hu moments methods

Calibration and segmentation free 3D modelling from images based on GPU

This paper presents an automatic image-based modelling method based on shape from silhouettes that does not need any user interactions of camera calibration or image segmentation. Under circular motion constraints, using an iterative optimisation of graph cuts and conjugate direction minimisation, we can label an objects visual hull and minimise silhouette coherence, which is the difference between projected visual hull and the background-subtracted silhouettes. This process can converge to accurate camera parameters and smooth visual hull. Using Graphics Processing Unit and its parallel computation ability, our approach is not only automatic but also efficient, and can produce realistic 3D models.

Hardware-based camera calibration and 3D modelling under circular motion

In this paper, we present a combined camera calibration and image based modeling method using an iterative optimization of shape from silhouette under circular motion. By minimizing the difference between the projections of reconstructed visual hull and the silhouette images using graphics hardware, the optimization can finally converge to accurate camera parameters and realistic visual hull efficiently and robustly. Using this method, we can automatically create photorealistic 3D models directly from images.

A video-driven approach to continuous human motion synthesis

We propose a framework to reconstruct human motion based on monocular camera video and motion database. In this framework, we use silhouettes for rough motion estimation based on a set of discriminative features and search motion database to find out the exact motion clips that meet with the video content. We model motion as a first-order Markov process. The transition probabilities between motion clips are preprocessed with consideration of the continuousness and smoothness of human motion. To eliminate the discontinuities between motion clips, we also adopt a seamless motion stitch method using multiresolution analysis technique. We verify the effectiveness of our method by reconstructing trampoline sports video as an example. The reconstruction results are visually comparable to those motions obtained by a commercial motion capture system in the premise that similar motions are included in the motion database.

Human silhouette matching based on moment invariants

This paper aims to apply the method of silhouette matching based on moment invariants to infer the human motion parameters from video sequences of single monocular uncalibrated camera. Currently, there are two ways of tracking human motion: Marker and Markerless. While a hybrid framework is introduced in this paper to recover the input video contents. A standard 3D motion database is built up by marker technique in advance. Given a video sequences, human silhouettes are extracted as well as the viewpoint information of the camera which would be utilized to project the standard 3D motion database onto the 2D one. Therefore, the video recovery problem is formulated as a matching issue of finding the most similar body pose in standard 2D library with the one in video image. The framework is applied to the special trampoline sport where we can obtain the complicated human motion parameters in the single camera video sequences, and a lot of experiments are demonstrated that this approach is feasible in the field of monocular video-based 3D motion reconstruction.

Efficient reconstruction from architectural drawings

This paper presents an efficient method for reconstructing 3D building models from Electronic Architectural Drawings (EADs). EADs are matched via recognising axes and elevation. After recognising Candidate Contours (CCs) of Architectural Components (ACs) in one floor plan, the method incorporates the results into recognising the neighbour floors, which is achieved by a mapping algorithm. CC classification is proposed to reduce the interaction for 3D reconstruction. A parameterised 3D model database is used to improve details of reconstructed results. Our method requires some interaction as guidance. The experimental results show that ordinary users could create a 3D building model that consists of over 100 components in several minutes.

Reconstruct 3D human motion from monocular video using motion library

In this paper, we present a new approach to reconstruct 3D human motion from video clips with the assistance of a precaputred motion library. Given a monocular video clip recording of one person performing some kind of locomotion and a motion library consisting of similar motions, we can infer the 3D motion from the video clip. We segment the video clip into segments with fixed length, and by using a shape matching method we can find out from the motion library several candidate motion sequences for every video segment, then from these sequences a coarse motion clip is generated by performing a continuity test on the boundaries of these candidate sequences. We propose a pose deformation algorithm to refine the coarse motion. To guarantee the naturalness of the recovered motion, we apply a motion splicing algorithm to the motion clip. We tested the approach using synthetic and real sports videos. The experimental results show the effectiveness of this approach.