Jen- Yu

Ball tracking and 3D trajectory approximation with applications to tactics analysis from single-camera volleyball sequences

Providing computer-assisted tactics analysis in sports is a growing trend. This paper presents an automatic system for ball tracking and 3D trajectory approximation from single-camera volleyball sequences as well as demonstrates several applications to tactics analysis. Ball tracking in volleyball video has great complexity due to the high density of players on the court and the complicated overlapping of ball-player. The 2D-to-3D inference is intrinsically challenging due to the loss of 3D information in projection to 2D frames. To overcome these challenges, we propose a two-phase ball tracking algorithm in which we first detect ball candidates for each frame, and then use them to compute the ball trajectories. With the aid of camera calibration, we involve physical characteristics of ball motion to approximate the 3D ball trajectory from the 2D trajectory. The visualization of 3D trajectory and the applications to trajectory-based tactics analysis not only assist the coaches and players in game study but also make game watching a whole new experience. The experiments on international volleyball games show encouraging results. We believe that the proposed framework can be extended and applied to various kinds of sports games.

Pitch-by-Pitch Extraction from Single View Baseball Video Sequences

This paper presents a novel method for reducing a baseball video segment from one batter to next batter into a more compact pitch-by-pitch video by pitching ball trajectory detection. The pitch-by-pitch video shows the complete pitching and batting process and largely reduces the source video data, making pitching analysis of broadcast baseball sequences an easier task. The proposed method has been tested for several long sequences, and promising results are reported.

Visual language model for face clustering in consumer photos

For consumer photos, this work clusters faces with large variations in lighting, pose, and expression. After matching face images by local feature points, we transform matching situations into a novel representation called visual sentences. Then, visual language models are constructed to describe the dependency of image patches on faces. With the probabilistic framework, we develop a clustering algorithm to group the same individuals face images into the same cluster. An interesting observation about evaluating face clustering performance is proposed, and we demonstrate the superiority of the proposed visual language model approach.

Human action recognition based on layered-HMM

We address the problem of human action understanding of the upper human body from video sequences. Time-sequential images expressing human actions are transformed to sequences of feature vectors containing the configuration of the human body. A human is modeled as a collection of body parts, linked in a kinematic structure. The relation of the joints is used to estimate the human pose. A proposed layered HMM framework decomposes the human action recognition problem into two layers. The first layer models the actions of two arms individually from low-level features. The second layer models the interrelationship of two arms as an action. Experiments with a set of six types of human actions demonstrate the effectiveness of our proposed scheme, and the comparisons with other HMM systems show the robustness.

Extraction and representation of human body for pitching style recognition in broadcast baseball video

In baseball games, different release points of pitchers form several kinds of pitching styles. Different pitching styles possess individual advantages. This paper presents a novel pitching style recognition approach for automatic generation of game information and video annotation. First, an effective object segmentation algorithm is designed to compute the body contour and extract the pitchers body. Then, star skeleton is used as the representative descriptor of the pitcher posture for pitching style recognition. The proposed approach has been tested on broadcast baseball video and the promising experimental results validate the robustness and practicability.

Computer-assisted self-training system for sports exercise using kinects

Self-training plays an important role in sports exercise. However, if not under the instruction of a coach, improper training postures can cause serious harm to muscles and ligaments of the body. Hence, the development of computer-assisted self-training systems for sports exercise is a recently emerging research topic. In this paper, we propose a Yoga self-training system, entitled YogaST, which aims at instructing the user/practitioner to perform the asana (Yoga posture) correctly and preventing injury caused by improper postures. Involving professional Yoga training knowledge, YogaST analyzes the practitioners posture from both front and side views using two Kinects with perpendicular viewing directions and assists him/her in rectifying bad postures. The contour, skeleton, and feature axes of the human body are extracted as posture representation. Then, YogaST analyzes the practitioners posture and presents visualized instruction for posture rectification so that the practitioner can easily understand how to adjust his/her posture.

Using context information and local feature points in face clustering for consumer photos

We introduce local feature points to achieve face clustering for consumer photos. After combining eigenfaces with context information like clothes, we further investigate the usage of local feature points to match face images. The relationships between face images are constructed by feature matching and then described as a graph. Outliers in the results of preliminary clustering are detected and are re-clustered according to matching characteristics. We report complete performance comparison for different datasets and show that the proposed method has superior performance than conventional approaches.

Feature classification for representative photo selection

This paper points out that different local feature points provide different impacts to near-duplicate detection and related applications. Aiming to automatic representative photo selection, we develop three feature classification methods, i.e., point-based, region-based, and pLSA-based classification, to differentiate local feature points described by SIFT descriptors. We investigate the performance of these classification methods, and discuss how they influence near-duplicate detection and extended applications. Experiments show that, with effective feature classification, more accurate representative selection results can be achieved.

A Tempo Analysis System for Automatic Music Accompaniment

This paper proposes a music accompaniment system capable of catching the tempo of music. The original signal is first reduced to a detection function revealing the pulses of music. To induce the tempo and locate beats, a Fourier analysis-based algorithm with high practicality and generality is designed that it is robust to beat strength and not restricted to specific music genres. The regularity and periodicity of music beats are extracted and even the missing beats are recovered. The performance is validated using a comprehensive testing data set, and results in both formal objective experiments and subjective listening evaluations show convincible performance. Moreover, an interactive interface is designed that users are allowed to select an instrument to accompany the music based on the obtained tempo.

A packet loss estimation model and its application to reliable mesh-based P2P video streaming

This paper proposes a model to estimate the packet loss probability in a mesh-based P2P network. Because of the irregular mesh structure, packet loss estimation for a mesh-based P2P network is more complicated than that in a tree-based network. The proposed model takes into account the channel packet drop rate, peer dynamics, and FEC protection to capture the heterogeneous packet loss behavior of individual video substreams transmitted over the irregular transmission paths of a mesh network. The simulation results show that the proposed packet loss model can accurately estimate the packet loss in a mesh-based P2P network. Based on the proposed model, we also propose a peer selection mechanism which can effectively mitigate packet loss propagation by selecting at a parent-peer the candidate child-peers that can achieve the minimal packet loss probability compared to others, to transmit the FEC redundant substream.