Roberto Castagno

Video segmentation based on multiple features for interactive multimedia applications

We present a scheme for interactive video segmentation. A key feature of the system is the distinction between two levels of segmentation, namely, regions and object segmentation. Regions are homogeneous areas of the images, which are extracted automatically by the computer. Semantically meaningful objects are obtained through user interaction by grouping of regions according to the specific application. This splitting relieves the computer of ill-posed semantic problems, and allows a higher level of flexibility of the method. The extraction of regions is based on the multidimensional analysis of several image features by a spatially constrained fuzzy C-means algorithm. The local level of reliability of the different features is taken into account in order to adaptively weight the contribution of each feature to the segmentation process. Results on the extraction of regions as well as on the tracking of spatiotemporal objects are presented.

A method for motion adaptive frame rate up-conversion

A frame interpolation algorithm for frame rate up-conversion of progressive image sequences is proposed. The algorithm is based on simple motion compensation and linear interpolation. A motion vector is searched for each pixel in the interpolated image and the resulting motion field is median filtered to remove inconsistent vectors. Averaging along the motion trajectory is used to produce the interpolated pixel values. The main novelty of the proposed method is the motion compensation algorithm which has been designed with low computational complexity as an important criterion. Subsampled blocks are used in block matching and the vector search range is constrained to the most likely motion vectors. Simulation results show that good visual quality has been obtained with moderate complexity. The algorithm has been designed mainly for 50 Hz to 75 Hz frame rate up-conversion with applications in a multimedia environment, but it can also be used in advanced television receivers to remove artifacts due to low scan rate.

Transport of MPEG—4 over IP/RTP

In this paper we review and discuss some of the current efforts and proposals in the standardization bodies IETF and MPEG for the delivery of MPEG—4 over IP. We address the issues of payload format and media control by describing two architectures that allow for MPEC—4 streaming. Finally, we describe some MPEG—4 over IP streaming experiments in which the RTP MPEG—4 payload format is used in conjunction with per—stream Forward Error Correction support, in presence of packet loss.

Dynamic approach to visual data compression

This paper presents the Swiss Federal Institute of Technology (EPFL) proposal to MPEG-4 video coding standardization activity. The proposed technique is based on a novel approach to audio-visual data compression entitled dynamic coding. The newly born multimedia environment supports a plethora of applications which cannot be covered adequately by a single compression technique. Dynamic coding offers the opportunity to combine several compression techniques and segmentation strategies. Given a particular application, these two degrees of freedom can be constrained and assembled in order to produce a particular profile which meets the set of specifications dictated by the application. The basic principles of this approach are presented together with the data representation system. The major characteristics of dynamic coding are reviewed, along with simulation results showing the performance of such an approach in a very low bit-rate video coding environment.

Dynamic video coding-an overview

In this paper, we present an overview of the dynamic coding approach, together with recent developments carried out in this framework. Dynamic coding is a general approach to the problem of visual data representation in the context of multimedia. This approach consists of a dynamic combination of multiple representation models and segmentation strategies. Given an application, these two degrees of freedom are assembled so as to yield a specific profile which meets the specifications dictated by the application. The data is represented as the union of data segments, each described within a locally appropriate representation model. In order to illustrate this approach, a video compression system, based on the principles of dynamic coding, is proposed in the context of video-telephone/conference applications. This algorithm has been submitted to the MPEG-4 committee as a proposal for the first round of tests in November 1995. Recent developments have been added: in particular, a procedure enabling the generation of an object-oriented scalable bitstream is presented here. In order to reduce the blocking artifacts which are noticeable at high compression ratios, a post-processing technique is proposed.

Estimation of image feature reliability for an interactive video segmentation scheme

We present a two-stage approach to the estimation of the reliability of different image features in the framework of a complete video segmentation scheme. The clustering method adopted in the segmentation scheme (fuzzy C-means) requires that a distance is evaluated between points in a multidimensional feature space. The analysis presented aims at establishing an appropriate weighting strategy for the different image features used in the segmentation, such as spatial information, motion, texture etc. The a priori reliability is based on an analysis of the characteristics of the frames, and influences the relative importance attributed to motion information with respect to the other features. The a posteriori reliability analysis is based on the results of the earlier stages of the segmentation process itself, and aims at identifying the features that best characterize each region and thus improve the results of the successive stages of the algorithm.

A spline-based adaptive filter for the removal of blocking artifacts in image sequences coded at very low bitrate

In this paper, a method for the reduction of blocking effect in image sequences coded at very low bitrate is presented. The proposed operator is capable of adapting its behaviour to the local characteristics of the images. In areas containing details, the degradation introduced by block-based coding is often masked by the texture itself and a satisfactory visual effect can be achieved by simply leaving the area unchanged. In uniform areas, instead, the coarse quantization normally used at very low bitrate results in visible steps in the gray level. A local action along the border is not sufficient for the removal of the artifact (specially when the frames are later viewed in a sequence). In this case, a spline interpolation is performed on the gray levels of pixels located on a wider area across the border. The extension of the filtered zone inside the blocks is evaluated by means of an index of activity so as to maximize the smoothing effect, while preserving the details inside the blocks.

An architecture based on IETF protocols for the transport of MPEG-4 content over the Internet

Networking of MPEG-4 content is the topic of ongoing standardization efforts in the ISO MPEG and IETF. In this paper, we describe a complete architecture for the transport of MPEG-4 content over the Internet. The MPEG-4 concept of separation between applications and delivery mechanism is preserved. The use of well-established Internet protocols, such as the real time protocol (RTP) and the real time streaming protocol (RTSP) allows for robustness and flexibility in view of interoperability and future extensions. In particular, we present an original proposal for the mapping of MPEG-4 specific signaling primitives onto RTSP functions.

Interactive Object Extraction from Video Sequences for Multimedia Applications Based on Multiple Features

In this paper, we present a scheme for interactive video segmentation. A key feature of the system is the distinction between two levels of segmentation, namely Regions and Object Segmentation. Regions are homogeneous areas of the images, which are extracted automatically by the computer. Semantically meaningful objects are obtained through user interaction by grouping of regions, according to the specific application. This splitting relieves the computer of ill-posed semantic problems, and allows a higher level of flexibility of the method. The extraction of regions is based on the multidimensional analysis of several image features by a spatially constrained Fuzzy C-Means algorithm. The local level of reliability of the different features is taken into account in order to adapti-vely weight the contribution of each feature to the segmentation process. Results about the extraction of regions as well as about the tracking of spatio-temporal objects are presented.

Savoir faire et faire savoir: what I learned about innovation while I thought I was doing a Ph.D.

This short text is a reflection of lessons learned about innovation during my studies at the signal processing laboratory of the Swiss Federal Institute of Technology, while I thought I was studying video segmentation based on multiple features for interactive multimedia applications.