Joan Llach

Tracking of generic objects for video object generation

A tracking technique for creating generic video objects is presented. It assumes that the object in the initial image has been previously defined by an object partition. The object tracking relies on the concept of partition projection. The projection of a partition accommodates the previous partition information into the current image. The object partition is re-segmented so a texture partition is created. The texture partition guarantees the spatial homogeneity of each region and it is projected on the following image using motion information. Projected regions are used as markers and, to validate the position of the markers, they are fit into an intra fine partition of the current image. The texture partition of the current image is obtained by a modified watershed algorithm based on the fit markers. Finally, the current object partition is obtained by a labeling process.

Players and ball detection in soccer videos based on color segmentation and shape analysis

This paper proposes a scheme to detect and locate the players and the ball on the grass playfield in soccer videos. We put forward a shape analysis-based approach to identify the players and the ball from the roughly extracted foreground, which is obtained by a trained, color histogram-based playfield detector and connected component analysis. We employ Euclidean distance transform to extract skeletons for every foreground blob, and then perform shape analysis to remove false alarms (non-player and non-ball blobs) and cut-off the artifacts (mostly due to playfield lines) based on skeleton pruning and reverse Euclidean distance transform. Results are given to demonstrate the proposed algorithm works well in soccer video clips.

Adaptive spatio-temporal filtering for video denoising

Video denoising is an important feature of many modem video encoding architectures since it can considerably enhance coding efficiency while at the same time improving objective and subjective quality. In this paper, a new scheme for video denoising based on spatio-temporal filtering is presented, which can be used as a preprocessing stage within a video encoder. Spatial filtering is performing through the adaptive selection and combination of a wavelet based and a 2D Wiener filter, while for temporal filtering we employ bidirectional block based motion compensation using the Enhanced Predictive Zonal Search (EPZS) algorithm. Experimental results are presented which show a significant improvement in video quality when the denoised video sequence is encoded with an H.264 encoder.

Improving the quality of depth image based rendering for 3D Video systems

In 3D Video (3DV) applications, a reduced number of views plus depth maps are transmitted or stored. When there is a need to render virtual views in between the actual views, the technique of depth image based rendering (DIBR) can be used to generate the intermediate views. To address the problem of noisy depth information in 3DV systems, we propose novel methods that can be easily incorporated into DIBR to improve synthesized image quality. These include: (1) a heuristic scheme with adaptive spatting that blends multiple warped reference pixels based on their depth, warped pixel positions and camera parameters; (2) an approximation of the first scheme with up-sampling for fast processing; (3) boundary only splatting; and (4) view weighting based on hole distribution. Experiment results show that the proposed methods can improve synthesis quality significantly.

Multiscale Probabilistic Dithering for Suppressing Contour Artifacts in Digital Images

A method is proposed for reducing the visibility of ldquocontour artifacts,rdquo i.e., false contours resulting from color quantization in digital images. The method performs a multiscale analysis on the neighborhood of each pixel, determines the presence and scale of contour artifacts, and probabilistically dithers (perturbs) the color of the pixel. The overall effect is to ldquobreak downrdquo the false contours, making them less visible. The proposed method may be used to reduce contour artifacts at the same bit depth as the input image or at higher bit depths. The contour artifact detection mechanism ensures that artifact-free regions remain unaffected during the process.

A method of small object detection and tracking based on particle filters

In this paper an efficient method of small object localization is proposed that integrates detection and tracking. The system is initialized using a strong detector and then it locates the object over time using a weak detector and a temporal tracker. Both of strong and weak detectors are based on foreground-background segmentation. The strong detector is created from shape analysis of foreground blobs and used to trigger the object tracker. The weak detector is built with outputs from the foreground detection likelihood and integrated into the trackerpsilas observation likelihood. In the particle filter-based object tracker, motion estimation is embedded to generate a better proposal distribution and a mixture model is tailored to handle the ambiguity of template matching due to cluttered background. As a case study, the proposed scheme is applied to ball detection and tracking in soccer game videos. Promising results are presented to illustrate the proposed method effectively handles heavy clutter, occlusion and motion blur.

H.264 encoder with low complexity noise pre-filtering

In this paper, an H.264 encoder is proposed which incorporates a noise pre-filter with little additional complexity. The motion estimation process in the H.264 encoder, applied to multiple reference pictures, is re-used for temporal noise filtering. Significant objective and subjective improvement is observed for the proposed system versus a standalone H.264 encoder, with greater improvement at higher bitrates. For sequences with artificially generated noise, average PSNR improvements of 0.46 to 1.96 dB were obtained. The optimum number and type of pictures to use in the temporal noise filter was also studied. The proposed single stage encoder with noise filter system provides only a slight reduction in performance as compared to a more computationally complex two-stage system with separate noise filter and encoder.

Interactive tone mapping for High Dynamic Range video

Despite considerable progress in HDR image tone mapping for the past decade, little work has been done for HDR video. For applications such as film post-production, the capability of local tone manipulation is highly regarded by the content creators. This paper presents an interactive tone mapping scheme for HDR video sequences. It provides a simple scribble/ stroke based interface for local tone manipulation and is capable of propagating user input information throughout a video sequence by using Gaussian mixture model (GMM) and edge preserving filtering. The experimental results demonstrated its effectiveness for HDR video tone mapping as well as its flexibility for users to easily and intuitively manipulate the appearance of the video while maintaining temporal consistency.

Tracking the small object through clutter with adaptive particle filter

Cluttered background and occlusion cause large ambiguity in the tracking of video objects. When the object is small (like a soccer ball in broadcast game video signals), the ambiguity gets even more severe. In this paper, we propose an adaptive particle filter with effective proposal distribution to handle these situations. In the proposed tracking approach, motion estimation is embedded into the state transition to tackle abrupt motion changes and generate good proposal distributions. We also propose a mixture model to account for multiple hypotheses in the template correlation surface when estimating the appearance likelihood. In addition, motion continuity and trajectory smoothness are combined with template correlation in the observation likelihood to further filter out visual distracters. As an example of small object tracking, promising results of the ball tracking (as small as 30 pixels) in soccer game videos are presented to illustrate that the proposed scheme handles the cluttered background and occlusion effectively.

Non-uniform backlighting computation for high dynamic range displays

In recent years, non-uniform LED based backlighting techniques have been used to build display prototypes and products with extremely high dynamic range, thus making possible to show video content with very high fidelity. These displays have two modulation channels: a low resolution LED layer that provides non-uniform backlighting, and a high resolution LCD front panel for color. Unfortunately, displaying an HDR image on such screens is not straightforward because the lower resolution of the LED layer and the cross-talk between LEDs make not possible to individually control the output of each pixel. Using the wrong backlighting results in low image quality and may even lead to visual artifacts such as false contouring and visible LED patterns. In this paper, we characterize the LED/LCD display using a general model and formulate the backlighting computation as an optimization problem. We further propose an iterative algorithm to solve the optimization problem. Compared to existing methods, our model is more precise and provides better image quality on the same display device.