Ming-Ho Hsiao

A Trajectory-Based Ball Tracking Framework with Visual Enrichment for Broadcast Baseball Videos *

Pitching contents play the key role in the resultant victory or defeat in a baseball game. Utilizing the physical characteristic of ball motion, this paper presents a trajectory-based framework for automatic ball tracking and pitching evaluation in broadcast baseball videos. The task of ball detection and tracking in broadcast baseball videos is very challenging because in video frames, the noises may cause many ball-like objects, the ball size is small, and the ball may deform due to its high speed movement. To overcome these challenges, we first define a set of filters to prune most non-ball objects but retain the ball, even if it is deformed. In ball position prediction and trajectory extraction, we analyze the 2D distribution of ball candidates and exploit the characteristic that the ball trajectory presents in a near parabolic curve in video frames. Most of the non-qualified trajectories are pruned, which greatly improves the computational efficiency. The missed balls can also be recovered in the trajectory by applying the position prediction. The experiments of ball tracking on the testing sequences of JPB, MLB and CPBL captured from different TV channels show promising results. The ball tracking framework is able to extract the ball trajectory, superimposed on the video, and in near real-time provide visual enrichment before the next pitch coming up without specific cameras or equipments set up in the stadiums. It can also be utilized in strategy analysis and intelligence statistics for player training.

Shot Classification of Basketball Videos and its Application in Shooting Position Extraction

In this paper, we propose a system that can automatically segment a basketball video into several clips on the basis of a GOP-based scene change detection method. The length of each clip and the number of dominant color pixels of each frame are used to classify shots into close-up view, medium view, and full court view. Full court view shots are chosen to do advanced analyses such as ball tracking and parameter extracting for the transformation from a 3D real-world court to a 2D image. After that, we map points in the 2D image to the corresponding coordinates in a real-world court by some physical properties of the 3D shooting trajectory, and compute the statistics of all shooting positions. Eventually we can obtain the information about the most possible shooting positions of a professional basketball team, which is useful for opponents to adopt appropriate defense tactics.

A baseball exploration system using spatial pattern recognition

Despite a lot of research efforts in baseball video processing, little work has been done in analyzing the detailed process and ball movement of the batting content. This paper proposes a novel system to automatically summarize the progress of each batting in baseball videos. Utilizing the strictly-defined specifications of the baseball field, the system recognizes the spatial patterns in each frame and identifies what region of the baseball field is currently focused. Finally, an annotation string which abstracts the batting content is generated. With the annotation strings, the system is able to make descriptions and provide exploration for baseball videos, so that users can be given a further insight into the game quickly. The experiments on broadcast baseball videos of MLB and JPB show promising results.

Content-Aware Fast Motion Estimation Algorithm

In this paper, we propose the Content-Aware Fast Motion Estimation Algorithm (CAFME) that can reduce computation complexity of motion estimation (ME) in H.264/AVC while maintaining almost the same coding efficiency. Motion estimation can be divided into two phases: searching phase and matching phase. In searching phase, we propose the Simple Dynamic Search Range Algorithm (SDSR) based on video characteristics to reduce the number of search points (SP). In matching phase, we integrate the Successive Elimination Algorithm (SEA) and the integral frame to develop a new SEA for H.264/AVC video compression standard, called Successive Elimination Algorithm with Integral Frame (SEAIF). Besides, we also propose the Early Termination Algorithm (ETA) to early terminate the motion estimation of current block. We implement the proposed algorithm in the reference software JM9.4 of H.264/AVC and the experimental results show that our proposed algorithm can reduce the number of search points about 93.1%, encoding time about 42%, while maintaining almost the same bitrate and PSNR.

Automatic closed caption detection and filtering in MPEG videos for video structuring

Video structuring is the process of extracting temporal structural information of video sequences and is a crucial step in video content analysis especially for sports videos. It involves detecting temporal boundaries, identifying meaningful segments of a video and then building a compact representation of video content. Therefore, in this paper, we propose a novel mechanism to automatically parse sports videos in compressed domain and then to construct a concise table of video content employing the superimposed closed captions and the semantic classes of video shots. First of all, shot boundaries are efficiently examined using the approach of GOP-based video segmentation. Color-based shot identification is then exploited to automatically identify meaningful shots. The efficient approach of closed caption localization is proposed to first detect caption frames in meaningful shots. Then caption frames instead of every frame are selected as targets for detecting closed captions based on long-term consistency without size constraint. Besides, in order to support discriminate captions of interest automatically, a novel tool-font size detector is proposed to recognize the font size of closed captions using compressed data in MPEG videos. Experimental results show the effectiveness and the feasibility of the proposed mechanism.

A Tempo Analysis System for Automatic Music Accompaniment

This paper proposes a music accompaniment system capable of catching the tempo of music. The original signal is first reduced to a detection function revealing the pulses of music. To induce the tempo and locate beats, a Fourier analysis-based algorithm with high practicality and generality is designed that it is robust to beat strength and not restricted to specific music genres. The regularity and periodicity of music beats are extracted and even the missing beats are recovered. The performance is validated using a comprehensive testing data set, and results in both formal objective experiments and subjective listening evaluations show convincible performance. Moreover, an interactive interface is designed that users are allowed to select an instrument to accompany the music based on the obtained tempo.

Object-based video streaming technique with application to intelligent transportation systems

With rapid development of the Internet, traffic surveillance systems using Internet video streaming techniques are becoming mature. This paper presents a vision-based intelligent transportation system that can perform real-time traffic monitoring remotely. A background-registration method is used to dynamically adapt the background and separate vehicles from the background with an adaptive threshold. To utilize the bandwidth efficiently, important vehicles can, in real time, be segmented, encoded, and transmitted with higher quality and higher frame rate than those of the background. According to the traffic flow and mean speed, traffic events such as traffic jams and speeding can be detected. Automatic monitoring of traffic events would be useful in reducing the workload of human operators and providing road information. The experimental results show that the real time traffic surveillance system is indeed effective and efficient.

Content-aware video adaptation under low-bitrate constraint

With the development of wireless network and the improvement of mobile device capability, video streaming is more and more widespread in such an environment. Under the condition of limited resource and inherent constraints, appropriate video adaptations have become one of the most important and challenging issues in wireless multimedia applications. In this paper, we propose a novel content-aware video adaptation in order to effectively utilize resource and improve visual perceptual quality. First, the attention model is derived from analyzing the characteristics of brightness, location, motion vector, and energy features in compressed domain to reduce computation complexity. Then, through the integration of attention model, capability of client device and correlational statistic model, attractive regions of video scenes are derived. The information object- (IOB-) weighted rate distortion model is used for adjusting the bit allocation. Finally, the video adaptation scheme dynamically adjusts video bitstream in frame level and object level. Experimental results validate that the proposed scheme achieves better visual quality effectively and efficiently.

Searching the video: an efficient indexing method for video retrieval in peer to peer network

More and more applications require peer-to-peer (P2P) systems to support complex queries over multi-dimensional data. The retrieval facilities of most P2P systems are limited to queries based on a unique identifier or a small set of keywords. The techniques used for this purpose are hardly applicable for content-based video retrieval in a P2P network (CBP2PVR). In this paper, we present the design of a distributed P2P video sharing system that supports content-based video retrieval. First we will propose the compact signature generation of video shot which can be distributed in a P2P network and used as the basis for a source selection. Second, a Global Indexing structure based on proposed novel PVR-tree index schema allows communicating only with a small fraction of all peers during query processing without deteriorating the result quality significantly. We will also present experimental results confirming our approach.

Automatic Closed Caption Detection and Font Size Differentiation in MPEG Video

In this paper, a novel approach of automatic closed caption detection and font size differentiation among localized text regions in I-frames of MPEG videos is proposed. The approach consists of five modules: video segmentation, shot selection, caption frame detection, caption localization and font size differentiation. Rather than directly examines scene cut frame by frame, the module of video segmentation first verifies video streams GOP by GOP and then finds out the actual scene boundaries in the frame level. Tennis videos are selected as the case study and the module of shot selection is designed to automatically select specific type of shot for further closed caption detection. The noise of potential captions is filtered out based on its long-term consistency over consecutive frames. While the general closed captions are localized, we select the specific caption that is discriminated utilizing the module of font size differentiation. The detected closed captions can support video structuring, video browsing, high-level video indexing and video content description in MPEG-7. Experimental results show the effectiveness and the feasibility of the proposed scheme.