Slawomir Mackowiak

Spatio-temporal scalability for MPEG video coding

The existing and standardized solutions for spatial scalability are not satisfactory, therefore new approaches are very actively being explored. The goal of this paper is to improve spatial scalability of MPEG-2 for progressive video. In order to avoid problems with too large bitstreams of the base layer produced by some of the hitherto proposed spatially scalable coders, spatio-temporal scalability is proposed for video compression systems. It is assumed that a coder produces two bitstreams, where the base-layer bitstream corresponds to pictures with reduced both spatial and temporal resolution while the enhancement layer bitstream is used to transmit the information needed to retrieve images with full spatial and temporal resolution. In the base layer, temporal resolution reduction is obtained by B-frame data partitioning, i.e., by placing each second frame (B-frame) in the enhancement layer. Subband (wavelet) analysis is used to provide spatial decomposition of the signal. Full compatibility with the MPEG-2 standard is ensured in the base layer, as compared to single-layer MPEG-2 encoding at bit rates below 6 Mbits/s, the bitrate overhead for scalability is less than 15% in most cases.

Modified MPEG-2 video coders with efficient multi-layer scalability

The paper describes a multi-layer video coder based on spatiotemporal scalability and data partitioning. The coder consists of two parts: a low-resolution coder and full-resolution coder. The first one encodes pictures with reduced spatial and temporal resolution. The data are partitioned into some base layers with fine granularity. The full-resolution encoder exploits interpolated images from the base layers. Here, again the output data can be partitioned into some layers. For the two-layer system, the bitrate overhead measured relative to the single layer MPEG-2 bitstream varies about 10% - 25% for progressive television test sequences. Further layers are related to bitrate overheads of about 3% per layer. The coder structure exhibits high level of compatibility with individual building blocks of MPEG-2 coders.

Wipe transition detection based on motion activity and dominant colours descriptors

Future image and video sequence encoders will be able to use the metadata information in order to improve their efficiency or introduce a new technique of error masking in a video decoder. The paper presents a wipe transition detection algorithm based on two video descriptors: motion activity and dominant colour. The information about transition could be useful in encoder in order to optimize their encoding strategy.

Fine granularity in multi-loop hybrid coders with multi-layer scalability

The paper describes a generic multi-loop coder structure suitable for mixed spatial and temporal scalability combined with fine granular SNR scalability. The structure is suitable for various variants of hybrid video coders like MPEG-2, H.263 and H.26L. The idea of mixed spatial and temporal scalability i.e. spatio-temporal scalability is substantial for the proposal. Its application allows improving the scalable coding efficiency i.e. decreasing the scalability overhead. The coder consists of independently motion-compensated sub-coders that produce bitstreams corresponding to individual levels of spatio-temporal resolution. The bitrate can be smoothly matched to the particular channel bandwidth by use of data partitioning, which is related to drift errors in the decoder. Accumulation and propagation of these errors can be bounded by use of proper structures of groups of pictures.

Scalable MPEG video coding with improved B-frame prediction

Recently, there is a great interest in video codecs that implement the functionality of spatial scalability. Unfortunately, those MPEG-2 and MPEG-4 coders which exhibit such a functionality produce much more bits than corresponding single layer coders. This bitrate overhead can be reduced by application of spatio-temporal scalability as proposed by the authors. The base layer bitstream corresponds to pictures with reduced both spatial and temporal resolution while the enhancement layer bitstream is used to transmit the information needed to retrieve images with full spatial and temporal resolution. Full compatibility with the MPEG standards is ensured in the base layer where temporal resolution reduction is obtained by B-frame data partitioning, i.e. by placing each second frame (B-frame) in the enhancement layer only. Improved prediction of B-frames in the enhancement layer is proposed in this paper. The idea is to combine temporal forward and backward prediction with spatial interpolation. Experimental results prove a clear improvement of the MPEG-2-compatible scalable coding efficiency for the scheme proposed.

Human activity recognition in multiview video

In this paper, a novel multiview video based human activity recognition system which automatic detects of such behavior as fainting, a fight or a call for help is presented. The approach proposed in this paper used a directed graphical model based on propagation nets, a subset of dynamic Bayesian networks approaches, to model the behaviors. The performance of activity recognition is analyzed for three methods of characteristic points forming a behavior descriptor (four extreme points over contour, four extreme points over contour with different normalization process and n-evenly distributed points on the contour). The results prove high score of recognition of the system for “Calling for help”, “Faint”, “Fight”, “Falling” and “Bend at the waist” behaviors.

Football player detection in video broadcast

The paper describes a novel segmentation system based on the combination of Histogram of Oriented Gradients (HOG) descriptors and linear Support Vector Machine (SVM) classification for football video. Recently, HOG methods were widely used for pedestrian detection. However, presented experimental results show that combination of HOG and SVM is very promising for locating and segmenting players. In proposed system a dominant color based segmentation for football playfield detection and a 3D playfield modeling based on Hough transform is introduced. Experimental evaluation of the system is done for SD (720×576) and HD (1280×720) test sequences. Additionally, we test proposed system performance for different lighting conditions (non-uniform pith lightning, multiple player shadows) as well as for various positions of the cameras used for acquisition.

Practical Realization of the Content-Based Image Retrieval System Using Sketch and Example

This paper covers methods of query-forming and information-retrieval in multimedia databases, which contain data described in accordance to the MPEG-7 standard. Principles of query-by-example and query-by-sketch have been presented. Also issues of multidimensional structures indexing (vector-indexing and GEMINI-indexing) have been discussed. The authors provided a genuine application, which is connected to the Microsoft SQL Server 2000 database. The application allows to run experiments in paper of the covered substance. It provides content based image-retrieval, basing on query-by-example or query-by-sketch.

Hybrid coding of video with spatiotemporal scalability using subband decomposition

The paper deals with scalable coding of video with SDTV or HDTV resolution. A new technique of scalable coding is proposed for bitrates of about 3 - 10 Mbps. The technique has been implemented for BT.601 resolution and progressive scan, therefore problems related to an interlaced scan are omitted here. The goal is to improve spatial scalability of MPEG-2 by introducing spatio-temporal scalability. The technique proposed needs less coding overhead than in MPEG-2 spatially scalable scheme and an enhancement layer bitstream with its bitrate not less than the bitrate in a base layer. The solution proposed in the paper is based on both temporal and spatial resolution reduction performed for data transmitted in a base layer. The temporal resolution reduction is obtained by placing each second frame (B-frame) in the enhancement layer. The enhancement layer includes also high-frequency spatial subbands from other frames. A variant of the system based on three-dimensional spatio-temporal analysis is also described. In both cases the assumption is that a base layer is fully MPEG-2 compatible.

Human Behavior Recognition Using Negative Curvature Minima and Positive Curvature Maxima Points

Recently, automated human behavior recognition are studied in the context of many new applications such as content-based video annotation and retrieval, highlight extraction, video summarization and video surveillance. In this chapter a novel description of human pose - a combination of negative curvature minima (NCM) and positive curvature maxima (PCM) points are proposed. Experimental results are provided in the chapter in order to demonstrate precision of the human activity recognition versus size of the descriptor (a temporal interval durations between the nodes of the model). The experimental results are focused on recognition of call for help behavior. The results prove high score of recognition of the proposed method.