Riccardo Leonardi

Recent results in high-compression image coding (Invited Papaer)

The digital representation of an image requires a very large number of bits. The goal of image coding is to reduce this number as much as possible, and to reconstruct a faithful duplicate of the original picture. Early efforts in image coding, solely guided by information theory, led to a plethora of methods. The compression ratio reached a plateau of about 10:1 several years ago. Recent progress in the study of the brain mechanism of vision and of scene analysis has opened new vistas in picture coding. The concept of directional sensitivity of neurones in the visual cortex combined with the separate processing of contours and textures has led to a new class of coding methods, called second generation, capable of achieving compression ratios as high as 100:1. In this paper, recent results on object-based coding methods are reported, exhibiting improvements in the previous second-generation methods.

Distributed Monoview and Multiview Video Coding

Growing percentage of the world population now uses image and video coding technologies on a regular basis. These technologies are behind the success and quick deployment of services and products such as digital pictures, digital television, DVDs, and Internet video communications. Todays digital video coding paradigm represented by the ITU-T and MPEG standards mainly relies on a hybrid of block- based transform and interframe predictive coding approaches. In this coding framework, the encoder architecture has the task to exploit both the temporal and spatial redundancies present in the video sequence, which is a rather complex exercise. As a consequence, all standard video encoders have a much higher computational complexity than the decoder (typically five to ten times more complex), mainly due to the temporal correlation exploitation tools, notably the motion estimation process. This type of architecture is well-suited for applications where the video is encoded once and decoded many times, i.e., one-to-many topologies, such as broadcasting or video-on-demand, where the cost of the decoder is more critical than the cost of the encoder.

Scene break detection: a comparison

The automatic organization of video databases according to the semantic content of data is a key aspect for efficient indexing and fast retrieval of audio-visual material. In order to generate indices that can be used to access a video database, a description of each video sequence is necessary. The identification of objects present in a frame and the track of their motion and interaction in space and time, is attractive but not yet very robust. For this reason, since the early 90s, attempts have been applied in trying to segment a video in shots. For each shot a representative frame of the shot, called k-frame, is usually chosen and the video can be analyzed through its k-frames. Even if abrupt scene changes are relatively easy to detect, it is more difficult to identify special effects, such as dissolve, that were operated in the editing stage to merge two shots. Unfortunately, these special effects are normally used to stress the importance of the scene change (from a content point of view), so they are extremely relevant, therefore they should not be missed. It is very important to determine precisely the beginning and the end of the transition in the case of dissolves and fades. In this work, two new parameters are proposed. These characterize the precision of boundaries of special effects when the scene change involves more than two frames. They are combined with the common recall and precision parameters. Three types of algorithms for cut detection are considered: histogram-based, motion-based and contour-based. These algorithms are tested and compared on several video sequences. Results show that the best performance is achieved by the global histogram-based method which uses color information.

Semantic indexing of multimedia documents

We propose two approaches for semantic indexing of audio-visual documents, based on bottom-up and top-down strategies. We base the first approach on a finite-state machine using low-level motion indices extracted from an MPEG compressed bitstream. The second approach innovatively performs semantic indexing through Hidden Markov Models.

State-of-the-Art and Trends in Scalable Video Compression With Wavelet-Based Approaches

Scalable video coding (SVC) differs form traditional single point approaches mainly because it allows to encode in a unique bit stream several working points corresponding to different quality, picture size and frame rate. This work describes the current state-of-the-art in SVC, focusing on wavelet based motion-compensated approaches (WSVC). It reviews individual components that have been designed to address the problem over the years and how such components are typically combined to achieve meaningful WSVC architectures. Coding schemes which mainly differ from the space-time order in which the wavelet transforms operate are here compared, discussing strengths and weaknesses of the resulting implementations. An evaluation of the achievable coding performances is provided considering the reference architectures studied and developed by ISO/MPEG in its exploration on WSVC. The paper also attempts to draw a list of major differences between wavelet based solutions and the SVC standard jointly targeted by ITU and ISO/MPEG. A major emphasis is devoted to a promising WSVC solution, named STP-tool, which presents architectural similarities with respect to the SVC standard. The paper ends drawing some evolution trends for WSVC systems and giving insights on video coding applications which could benefit by a wavelet based approach.

Distributed video coding: Selecting the most promising application scenarios

Distributed Video Coding (DVC) is a new video coding paradigm based on two major Information Theory results: the Slepian-Wolf and Wyner-Ziv theorems. Recently, practical DVC solutions have been proposed with promising results; however, there is still a need to study in a more systematic way the set of application scenarios for which DVC may bring major advantages. This paper intends to contribute for the identification of the most DVC friendly application scenarios, highlighting the expected benefits and drawbacks for each studied scenario. This selection is based on a proposed methodology which involves the characterization and clustering of the applications according to their most relevant characteristics, and their matching with the main potential DVC benefits.

Audio as a support to scene change detection and characterization of video sequences

A challenging problem to construct video databases is the organization of video information. The development of algorithms able to organize video information according to semantic content of the data is getting more and more important. This will allow algorithms such as indexing and retrieval to work more efficiently. Until now, an attempt to extract semantic information has been performed using only video information. As a video sequence is constructed from a 2-D projection of a 3-D scene, video processing has shown its limitations especially in solving problems such as object identification or object tracking, reducing the ability to extract semantic characteristics. A possibility to overcome the problem is to use additional information. The associated audio signal is then the most natural way to obtain this information. This paper presents a technique which combines video and audio information together for classification and indexing purposes. The classification is performed on the audio signal; a general framework that uses the results of such classification is then proposed for organizing video information.

Image compression using binary space partitioning trees

For low bit-rate compression applications, segmentation-based coding methods provide, in general, high compression ratios when compared with traditional (e.g., transform and subband) coding approaches. In this paper, we present a new segmentation-based image coding method that divides the desired image using binary space partitioning (BSP). The BSP approach partitions the desired image recursively by arbitrarily oriented lines in a hierarchical manner. This recursive partitioning generates a binary tree, which is referred to as the BSP-tree representation of the desired image. The most critical aspect of the BSP-tree method is the criterion used to select the partitioning lines of the BSP tree representation, In previous works, we developed novel methods for selecting the BSP-tree lines, and showed that the BSP approach provides efficient segmentation of images. In this paper, we describe a hierarchical approach for coding the partitioning lines of the BSP-tree representation. We also show that the image signal within the different regions (resulting from the recursive partitioning) can be represented using low-order polynomials. Furthermore, we employ an optimum pruning algorithm to minimize the bit rate of the BSP tree representation (for a given budget constraint) while minimizing distortion. Simulation results and comparisons with other compression methods are also presented.

Identification of story units in audio-visual sequences by joint audio and video processing

A novel technique, which uses a joint audio-visual analysis for scene identification and characterization, is proposed. The paper defines four different scene types: dialogues, stories, actions, and generic scenes. It then explains how any audio-visual material can be decomposed into a series of scenes obeying the previous classification, by properly analyzing and then combining the underlying audio and visual information. A rule-based procedure is defined for such purpose. Before such rule-based decision can take place, a series of low-level pre-processing tasks are suggested to adequately measure audio and visual correlations. As far as visual information is concerned, it is proposed to measure the similarities between non-consecutive shots using a learning vector quantization approach. An outlook on a possible implementation strategy for the overall scene identification task is suggested, and validated through a series of experimental simulations on real audio-visual data.

Adaptive region growing technique using polynomial functions for image approximation

Abstract This paper presents an approximation algorithm for two-dimensional signals (e.g., images) using polynomial functions. The proposed algorithm is based on an adaptive segmentation of the original signal into adjacent regions and on the approximation of the signal in each region by a two-dimensional polynomial function. The segmentation is obtained by an adaptive region growing technique which allows perfect adaptation between the chosen approximation and the inner structure of the signal. Results of this technique are presented in the context of image coding applications.