Regis J. Crinon

Hierarchical video summarization

We address the problem of key-frame summarization of vide in the absence of any a priori information about its content. This is a common problem that is encountered in home videos. We propose a hierarchical key-frame summarization algorithm where a coarse-to-fine key-frame summary is generated. A hierarchical key-frame summary facilitates multi-level browsing where the user can quickly discover the content of the video by accessing its coarsest but most compact summary and then view a desired segment of the video with increasingly more detail. At the finest level, the summary is generated on the basis of color features of video frames, using an extension of a recently proposed key-frame extraction algorithm. The finest level key-frames are recursively clustered using a novel pairwise K-means clustering approach with temporal consecutiveness constraint. We also address summarization of MPEG-2 compressed video without fully decoding the bitstream. We also propose efficient mechanisms that facilitate decoding the video when the hierarchical summary is utilized in browsing and playback of video segments starting at selected key-frames.

Adaptive model-based motion estimation

A general discrete-time, adaptive, multidimensional framework is introduced for estimating the motion of one or several object features from their successive nonlinear projections on an image plane. The motion model consists of a set of linear difference equations with parameters estimated recursively from a nonlinear observation equation. The model dimensionality corresponds to that of the original, nonprojected motion space, thus allowing to compensate for variable projection characteristics such as paning and zooming of the camera. Extended recursive least-squares and linear-quadratic tracking algorithms are used to adaptively adjust the model parameters and minimize the errors of either smoothing, filtering or prediction of the object trajectories in the projection plane. Both algorithms are derived using a second order approximation of the projection nonlinearities. All the results presented here use a generalized vectorial notation suitable for motion estimation of any finite number of object features and various approximations of the nonlinear projection. The application of the model-based motion estimator for temporal decimation/interpolation in digital video sequence compression systems is presented.

Median filters: Analysis for 2 dimensional recursively filtered signals

Median filtering is a nonlinear technique for smoothing signals. In this paper we find the output distribution of recursively median filtered two dimensional signals with additive impulsive noise. We study the edge jitter effect that recursive median filtering introduces. Finally some examples are included illustrating these results.

Sprite-based video coding using on-line segmentation

We address the problem of on-line sprite-based video coding in cases where scene segmentation is not available a priori or transmission of such segmentation information cannot be afforded due to low bit rate requirements. We propose an on-line segmentation method that can be integrated into an MPEG4 on-line sprite based video codec. The proposed method uses macroblock types as well as motion compensated residuals to perform the on-line segmentation. It produces a background mosaic without requiring a priori foreground-background segmentation information. Our results demonstrate the coding efficiency and functionality benefits of the proposed approach.

Motion estimation: the concept of velocity bandwidth

This paper shows that like spatial and temporal frequencies, velocity is subject to aliasing. The analysis is based on studying a two dimensional sinusoidal signal moving at constant velocity. It is shown that given the spatio-temporal grid used to sample the image, displacement of a sinusoidal signal must remain confined to a well defined two dimensional domain to avoid velocity aliasing. Analytical derivations to the aliasing-free domains are provided for various sampling lattices such as the rectangular field-interlaced and offset field-interlaced (quincunx) sampling grids. The paper concludes with a few suggestions regarding how the concept of velocity bandwidth can be helpful in the estimation of object displacement in digital video sequences. In particular, bounds for the search domains use in multiscale block matching- based motion estimation algorithms are derived.

Motion estimation with variable velocity bandwidth

A new phase-based motion algorithm is introduced. The estimation is performed such that only non-aliased temporal frequencies are included in the calculation of the translation vector describing the displacement of an object in a digital video sequence. A full description of the algorithm is presented. It is based on using a two-dimensional fast Fourier transform in two consecutive video fields or frames. Simulation results are included to illustrate the capabilities of the algorithm in an object-tracking application featuring fast frame-to-frame motion. It is shown that the proposed algorithm out-performs a conventional phase-based motion estimation algorithm.

Adaptive multi-feature motion estimation

A general discrete-time, adaptive, model-based motion estimation framework is introduced. The model is built to describe the dynamics of a set of points belonging to the same object over long periods of time. It incorporates two motion components, one for capturing the slowly-varying motion modes of the object and the second for accounting for the external forces and any other rapidly-changing motion modes, respectively. Time-varying mechanisms are also provided to handle occlusions or exits from the field of view. Applications of the motion model include temporal decimation/interpolation and noise filtering. In particular it is shown that the model provides a flexible framework for filtering out uncertainties in the feature matches.<<ETX>>