Serge Ayer

Compact representations of videos through dominant and multiple motion estimation

An explosion of on-line image and video data in digital form is already well underway. With the exponential rise in interactive information exploration and dissemination through the World-Wide Web (WWW), the major inhibitors of rapid access to on-line video data are costs and management of capture and storage, lack of real-time delivery, and nonavailability of content-based intelligent search and indexing techniques. The solutions for capture, storage, and delivery may be on the horizon or a little beyond. However, even with rapid delivery, the lack of efficient authoring and querying tools for visual content-based indexing may still inhibit as widespread a use of video information as that of text and traditional tabular data is currently. In order to be able to nonlinearly browse and index into videos through visual content, it is necessary to develop authoring tools that can automatically separate moving objects and significant components of the scene, and represent these in a compact form. Given that video data comes in torrents-almost a megabyte every 30th of a second-it will be highly inefficient to search for objects and scenes in every frame of a video. In this paper, we present techniques to automatically derive compact representations of scenes and objects from the motion information. Image motion is a significant cue in videos for the separation of scenes into their significant components and for the separation of moving objects. Motion analysis is useful in capturing the visual content of videos for indexing and browsing in two different ways. First, separation of the static scene from moving objects can be accomplished by employing dominant 2D/3D motion estimation methods. Alternatively, if the goal is to be able to represent the fixed scene too as a composition of significant structures and objects, then simultaneous multiple motion methods might be more appropriate. In either case, view-based summarized representations of the scene can be created by video compositing/mosaicing based on the estimated motions. We present robust algorithms for both kinds of representations: 1) dominant motion estimation based techniques which exploit a fairly common occurrence in videos that a mostly fixed background (scene) is imaged with or without independently moving objects, and 2) simultaneous multiple motion estimation and representation of motion video using layered representations. Ample examples of the representations achieved by each method are included in the paper.

Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding

Representing and modeling the motion and spatial support of multiple objects and surfaces from motion video sequences is an important intermediate step towards dynamic image understanding. One such representation, called layered representation, has recently been proposed. Although a number of algorithms have been developed for computing these representations, there has not been a consolidated effort into developing a precise mathematical formulation of the problem. This paper presents one such formulation based on maximum likelihood estimation (MLE) of mixture models and the minimum description length (MDL) encoding principle. The three major issues in layered motion representation are: (i) how many motion models adequately describe image motion, (ii) what are the motion model parameters, and (iii) what is the spatial support layer for each motion model.<<ETX>>

Segmentation of moving objects by robust motion parameter estimation over multiple frames

A method for detecting and segmenting accurately moving objects in monocular image sequences is proposed. It consists of two modules, namely a motion estimation and a motion segmentation module. The motion estimation problem is formulated as a time varying motion parameter estimation over multiple frames. Robust regression techniques are used to estimate these parameters. The motion parameters for the different moving objects are obtained by successive estimations on regions for which the previously estimated motion parameters are not valid. The segmentation module combines all motion parameters and the gray level information in order to obtain the motion boundaries and to improve them by using time integration. Experimental results on real image sequences with static or moving camera in the presence of multiple moving objects are reported.

Soft caching: web cache management techniques for images

The vast majority of current Internet traffic is generated by web browsing applications. Proxy caching, which allows some of the most popular web objects to be cached at intermediate nodes within the network, has been shown to provide substantial performance improvements. In this paper we argue that image-specific caching strategies are desirable and will result in improved performance over approaches treating all objects alike. We propose that Soft Caching, where an image can be cached at one of a set of levels of resolutions, can benefit the overall performance when combined with cache management strategies that estimate, for each object, both the bandwidth to the server where the object is stored and the appropriate resolution level demanded by the user. We formalize the cache management problem under these conditions and describe an experimental system to test these techniques.

Invariant Image Retrieval Using Wavelet Maxima Moment

Wavelets have been shown to be an effective analysis tool for image indexing due to the fact that spatial information and visual features of images could be well captured in just a few dominant wavelet coefficients. A serious problem with current wavelet-based techniques is in the handling of affine transformations in the query image. In this work, to cure the problem of translation variance with wavelet basis transform while keeping a compact representation, the wavelet transform modulus maxima is employed. To measure the similarity between wavelet maxima representations, which is required in the context of image retrieval systems, the difference of moments is used. As a result, each image is indexed by a vector in the wavelet maxima moment space. Those extracted features are shown to be robust in searching for objects independently of position, size, orientation and image background.

Efficient algorithms for embedded rendering of terrain models

Digital terrains are generally large files and need to be simplified to be rendered efficiently. We propose to build an adaptive embedded triangulation based on a binary tree structure to generate multiple levels of details. We present a O(nlogn) decimation algorithm and a O(nlogn) refinement algorithm, where n is the number of elevation points. We compare them in a rate-distortion (RD) framework. The algorithms are based on an improved version of the optimal tree pruning algorithm G-BFOS allowing one to deal with constrained tree structures and non-monotonic tree functionals.

Dominant and multiple motion estimation for video representation

The major inhibitors of rapid access to online video data are costs and management of capture and storage, lack of high-speed real-time delivery and non-availability of content and context based intelligent search and indexing techniques. The solutions for capture, storage and delivery maybe on the horizon, however the lack of visual content based indexing of video and image information may still inhibit as widespread a use of this information modality as that of text or tabular data is currently. We present techniques for compact visual representation of video data that will be useful for visual content based presentation and indexing. Video data comes in torrents-almost a megabyte every 30th of a second-but also affords the exploitation of relatively smoothly changing information over time. The techniques presented exploit the motion information across video frames to represent the underlying scene in a compact visual form as it is seen across many slowly varying frames in a video. Two classes of techniques are presented: (i) dominant motion estimation based techniques which exploit a fairly common occurrence in videos that a mostly fixed background (scene) is imaged with or without independently moving objects, and (ii) simultaneous multiple motion estimation and representation of motion video using layered representations.

Morphological shape representation of segmented images based on temporally modeled motion vectors

Region based coding schemes are among the most promising compression techniques for very low bit-rates. They consist of image segmentation, contour and texture coding. In this paper, a new shape representation for segmented images based on the geodesic morphological skeleton is presented. It is used for the coding of contour prediction residues obtained after motion compensation based on temporally modeled motion vectors. A non-reversible pre-skeletonization filter removes contour noise. A reversible post-skeletonization filter allows progressive transmission of contour information. It is based on the spatial distribution of the skeleton points, which are found to be close to known contours.<<ETX>>

A framework for interactive courses and virtual laboratories

Theory and experimentation are both complementary in sciences. Where the former builds up a formal framework, the latter helps humans to develop their intuition. Hard-printed books are great supports for theorems and formulas repositories, but stay desperately static when providing examples. This paper presents some directions towards a framework for digital publishing, distance learning and computer-aided teaching. The interactive digital signal processing (DSP) book is built in this framework and provides a concrete example to the reader. The book, reachable at http://lcavwww.epfl.ch/DSPBook, is used as a digital support for a course taught by our lab to a third year undergraduate class in electrical engineering.

Tracking Based on Hierarchical Multiple Motion Estimation and Robust Regression

Keywords: LTS1 Reference LTS-ARTICLE-1993-006 Record created on 2006-06-14, modified on 2016-08-08