Stefaan Mys

Rate-distortion driven decoder-side bitplane mode decision for distributed video coding

Distributed video coding (DVC) features simple encoders but complex decoders, which lies in contrast to conventional video compression solutions such as H.264/AVC. This shift in complexity is realized by performing motion estimation at the decoder side instead of at the encoder, which brings a number of problems that need to be dealt with. One of these problems is that, while employing different coding modes yields significant coding gains in classical video compression systems, it is still difficult to fully exploit this in DVC without increasing the complexity at the encoder side. Therefore, in this paper, instead of using an encoder-side approach, techniques for decoder-side mode decision are proposed. A rate-distortion model is derived that takes into account the position of the side information in the quantization bin. This model is then used to perform mode decision at the coefficient level and bitplane level. Average rate gains of 13-28% over the state-of-the-art DISCOVER codec are reported, for a GOP of size four, for several test sequences.

Efficient Low-Delay Distributed Video Coding

Distributed video coding (DVC) is a video coding paradigm that allows for a low-complexity encoding process by exploiting the temporal redundancies in a video sequence at the decoder side. State-of-the-art DVC systems exhibit a structural coding delay since exploiting the temporal redundancies through motion-compensated interpolation requires the frames to be decoded out of order. To alleviate this problem, we propose a system based on motion-compensated extrapolation that allows for efficient low-delay video coding with low complexity at the encoder. The proposed extrapolation technique first estimates the motion field between the two most recently decoded frames using the Lucas-Kanade algorithm. The obtained motion field is then extrapolated to the current frame using an extrapolation grid. The proposed techniques are implemented into a novel architecture featuring hybrid block-frequency Wyner-Ziv coding as well as mode decision. Results show that having references from both temporal directions in interpolation provides superior rate-distortion performance over a single temporal direction in extrapolation, as expected. However, the proposed extrapolation method is particularly suitable for low-delay coding as it performs better than H.264/AVC intra, and it is even able to outperform the interpolation-based DVC codec from DISCOVER for several sequences.

Accounting for quantization noise in online correlation noise estimation for Distributed Video Coding

In Distributed Video Coding (DVC), compression is achieved by exploiting correlation between frames at the decoder, instead of at the encoder. More specifically, the decoder uses already decoded frames to generate side information Y for each Wyner-Ziv frame X, and corrects errors in Y using error correcting bits received from the encoder. For efficient use of these bits, the decoder needs information about the correlation between X available at the encoder and Y at the decoder. While several techniques for online estimation of correlation noise X - Y have been proposed, the quantization noise in Y has not been taken into account. As a solution, in this paper, we calculate the quantization noise of intra frames at the encoder and use this information at the decoder to improve the accuracy of the correlation noise estimation. Results indicate averageWyner-Ziv bit rate reductions up to 19.5% (Bjøntegaard delta) for coarse quantization.

Stopping criterions for turbo coding in a Wyner-Ziv video codec

Distributed video coding (DVC) targets video coding applications with low encoding complexity by generating a prediction of the video signal at the decoder. One of the most common architectures uses turbo codes to correct errors in this prediction. Unfortunately, a rigorous analysis of turbo coding in the context of DVC is missing. We have targeted one particular aspect of turbo coding: the stopping criterion. The stopping criterion indicates whether decoding was successful, i.e., whether the errors in the prediction signal have been corrected. In this paper we describe and compare several stopping criterions known from the field of channel coding and criterions currently used in DVC. As our results suggest the choice of the stopping criterion has a significant impact on the overall video-coding performance. Moreover, we have found that there are even better performing criterions than those currently used in DVC.

SNR Scalability in H.264/AVC Using Data Partitioning

Efficient bitrate reduction of video content is necessary in order to satisfy the different constraints imposed by decoding devices and transmission networks. Requantization is a fast technique for bitrate reduction, and has been successfully applied for MPEG-2 bitstreams. Because of the newly introduced intra prediction in H.264/AVC, the existing techniques are rendered useless. In this paper we examine requantization transcoding of H.264/AVC bitstreams, focusing on the intra 4x4 prediction modes. Two architectures are proposed, one in the pixel domain and the other in the frequency domain, that compensate tlie drift introduced by the requantization of intra 4x4 predicted blocks. Experimental results show that these architectures perform approximately equally well as the full decode and recode architecture for low to medium bitrates. Because of the reduced computational complexity of these architectures, in particular the frequency-domain compensation architecture, they are highly suitable for real-time adaptation of video content.Although no scalability is explicitly defined in the H.264/ AVC specification, some forms of scalability can be achieved by using the available coding tools in a creative way. In this paper we will explain how to use the data partitioning tool to perform a coarse form of SNR scalability. The impact of various parameters, including the presence of IDR frames and the number of intra-coded macroblocks per frame, on bit rate and bit rate savings and on quality and quality loss will be discussed. Furthermore we will introduce and elaborate a possible use case for the technique proposed in this paper.MPEG-21 BSDL offers a solution for exposing the structure of a binary media resource as an XML description, and for the generation of a tailored media resource using a transformed XML description. The main contribution of this paper is the introduction of a real-time work flow for the XML-driven adaptation of H.264/AVC bitstreams in the temporal domain. This real-time approach, which is in line with the vision of MPEG-21 BSDL, is made possible by two key technologies: BFlavor (BSDL + XFlavor) for the efficient generation of XML descriptions and Streaming Transformations for XML (STX) for the efficient transformation of these descriptions. Our work flow is validated in several applications, all using H.264/AVC bitstreams: the exploitation and emulation of temporal scalability, as well as the creation of video skims using key frame selection. Special attention is paid to the deployment of hierarchical B pictures and to the use of placeholder slices for synchronization purposes. Extensive performance data arc also provided.Expressive Speech Recognition and Synthesis as Enabling Technologies for Affective Robot-Child Communication.- Embodied Conversational Agents: Computing and Rendering Realistic Gaze Patterns.- DBN Based Models for Audio-Visual Speech Analysis and Recognition.- An Extensive Method to Detect the Image Digital Watermarking Based on the Known Template.- Fast Mode Decision Algorithm in H.263+/H.264 Intra Transcoder.- Binary Erasure Codes for Packet Transmission Subject to Correlated Erasures.- Image Desynchronization for Secure Collusion-Resilient Fingerprint in Compression Domain.- A Format-Compliant Encryption Framework for JPEG2000 Image Code-Streams in Broadcasting Applications.- Euclidean Distance Transform of Digital Images in Arbitrary Dimensions.- JPEG2000 Steganography Possibly Secure Against Histogram-Based Attack.- Perceptual Depth Estimation from a Single 2D Image Based on Visual Perception Theory.- A System for Generating Personalized Virtual News.- Image Fingerprinting Scheme for Print-and-Capture Model.- 16x16 Integer Cosine Transform for HD Video Coding.- Heegard-Berger Video Coding Using LMMSE Estimator.- Real-Time BSD-Driven Adaptation Along the Temporal Axis of H.264/AVC Bitstreams.- Optimal Image Watermark Decoding.- Diagonal Discrete Cosine Transforms for Image Coding.- Synthesizing Variational Direction and Scale Texture on Planar Region.- Fast Content-Based Image Retrieval Based on Equal-Average K-Nearest-Neighbor Search Schemes.- Characterizing User Behavior to Improve Quality of Streaming Service over P2P Networks.- Interacting Activity Recognition Using Hierarchical Durational-State Dynamic Bayesian Network.- Improving the Image Retrieval Results Via Topic Coverage Graph.- Relevance Feedback for Sketch Retrieval Based on Linear Programming Classification.- Hierarchical Motion-Compensated Frame Interpolation Based on the Pyramid Structure.- Varying Microphone Patterns for Meeting Speech Segmentation Using Spatial Audio Cues.- Region-Based Sub-pixel Motion Estimation from Noisy, Blurred, and Down-Sampled Sequences.- Differential Operation Based Palmprint Authentication for Multimedia Security.- A Broadcast Model for Web Image Annotation.- An Approach to the Compression of Residual Data with GPCA in Video Coding.- A Robust Approach for Object Recognition.- A Novel Method for Spoken Text Feature Extraction in Semantic Video Retrieval.- A Semantic Image Category for Structuring TV Broadcast Video Streams.- Markov Chain Monte Carlo Super-Resolution Image Reconstruction with Simultaneous Adaptation of the Prior Image Model.- Text Detection in Images Using Texture Feature from Strokes.- Robust Mandarin Speech Recognition for Car Navigation Interface.- GKDA: A Group-Based Key Distribution Algorithm for WiMAX MBS Security.- A Watermarking Algorithm for JPEG File.- SNR Scalability in H.264/AVC Using Data Partitioning.- A Real-Time XML-Based Adaptation System for Scalable Video Formats.- Generic, Scalable Multimedia Streaming and Delivery with Example Application for H.264/AVC.- Shape-Based Image Retrieval in Botanical Collections.- Macroblock Mode Decision Scheme for Fast Encoding in H.264/AVC.- A Mathematical Model for Interaction Analysis Between Multiview Video System and User.- Motion Composition of 3D Video.- EKM: An Efficient Key Management Scheme for Large-Scale Peer-to-Peer Media Streaming.- Using Earth Movers Distance for Audio Clip Retrieval.- Streaming-Mode MB-Based Integral Image Techniques for Fast Multi-view Video Illumination Compensation.- A Motion Vector Predictor Architecture for AVS and MPEG-2 HDTV Decoder.- Inter-camera Coding of Multi-view Video Using Layered Depth Image Representation.- Optimal Priority Packetization with Multi-layer UEP for Video Streaming over Wireless Network.- A Multi-channel MAC Protocol with Dynamic Channel Allocation in CDMA Ad Hoc Networks.- Fuzzy Particle Swarm Optimization Clustering and Its Application to Image Clustering.- A New Fast Motion Estimation for H.264 Based on Motion Continuity Hypothesis.- Statistical Robustness in Multiplicative Watermark Detection.- Adaptive Visual Regions Categorization with Sets of Points of Interest.- A Publishing Framework for Digitally Augmented Paper Documents: Towards Cross-Media Information Integration.- Web-Based Semantic Analysis of Chinese News Video.- A Quality-Controllable Encryption for H.264/AVC Video Coding.- Texture Synthesis Based on Minimum Energy Cut and Its Applications.- Unifying Keywords and Visual Features Within One-Step Search for Web Image Retrieval.- Dynamic Bandwidth Allocation for Stored Video Under Renegotiation Frequency Constraint.- Online Selection of Discriminative Features Using Bayes Error Rate for Visual Tracking.- Interactive Knowledge Integration in 3D Cloth Animation with Intelligent Learning System.- Multi-view Video Coding with Flexible View-Temporal Prediction Structure for Fast Random Access.- Squeezing the Auditory Space: A New Approach to Multi-channel Audio Coding.- Video Coding by Texture Analysis and Synthesis Using Graph Cut.- Multiple Description Coding Using Adaptive Error Recovery for Real-Time Video Transmission.- An Improved Motion Vector Prediction Scheme for Video Coding.- Classifying Motion Time Series Using Neural Networks.- Estimating Intervals of Interest During TV Viewing for Automatic Personal Preference Acquisition.- Image Annotations Based on Semi-supervised Clustering with Semantic Soft Constraints.- Photo Retrieval from Personal Memories Using Generic Concepts.- PanoWalk: A Remote Image-Based Rendering System for Mobile Devices.- A High Quality Robust Watermarking Scheme.- An Association Rule Mining Approach for Satellite Cloud Images and Rainfall.- AVAS: An Audio-Visual Attendance System.- Improved POCS-Based Deblocking Technique Using Wavelet Transform in Block Coded Image.- Sketch Case Based Spatial Topological Data Retrieval.- Providing Consistent Service for Structured P2P Streaming System.- Adaptive Search Range Scaling for B Pictures Coding.- Video QoS Monitoring and Control Framework over Mobile and IP Networks.- Extracting Moving / Static Objects of Interest in Video.- Building a Personalized Music Emotion Prediction System.- Video Segmentation Using Joint Space-Time-Range Adaptive Mean Shift.- EagleRank: A Novel Ranking Model for Web Image Search Engine.- Color Image Enhancement Using the Laplacian Pyramid.- 3D Mesh Construction from Depth Images with Occlusion.- An Eigenbackground Subtraction Method Using Recursive Error Compensation.- Attention Information Based Spatial Adaptation Framework for Browsing Videos Via Mobile Devices.- Style Strokes Extraction Based on Color and Shape Information.- Requantization Transcoding of H.264/AVC Bitstreams for Intra 4x4 Prediction Modes.- Prediction Algorithms in Large Scale VOD Services on Grid Infrastructure.- A Hierarchical Framework for Fast Macroblock Prediction Mode Decision in H.264.- Compact Representation for Large-Scale Clustering and Similarity Search.- Robust Recognition of Noisy and Partially Occluded Faces Using Iteratively Reweighted Fitting of Eigenfaces.- Pitching Shot Detection Based on Multiple Feature Analysis and Fuzzy Classification.- Color Changing and Fading Simulation for Frescoes Based on Empirical Knowledge from Artists.- A Novel Spatial-Temporal Position Prediction Motion-Compensated Interpolation for Frame Rate Up-Conversion.- Web Image Clustering with Reduced Keywords and Weighted Bipartite Spectral Graph Partitioning.- An Architecture to Connect Disjoint Multimedia Networks Based on Nodes Capacity.- Quantitative Measure of Inlier Distributions and Contour Matching for Omnidirectional Camera Calibration.- High-Speed All-in-Focus Image Reconstruction by Merging Multiple Differently Focused Images.- A Real-Time Video Deinterlacing Scheme for MPEG-2 to AVS Transcoding.- Persian Text Watermarking.- Three Dimensional Reconstruction of Structured Scenes Based on Vanishing Points.- Parallel Processing for Reducing the Bottleneck in Realtime Graphics Rendering.- Distributed Data Visualization Tools for Multidisciplinary Design Optimization of Aero-crafts.- An Efficient Clustering and Indexing Approach over Large Video Sequences.- An Initial Study on Progressive Filtering Based on Dynamic Programming for Query-by-Singing/Humming.- Measuring Multi-modality Similarities Via Subspace Learning for Cross-Media Retrieval.- SNR-Based Bit Allocation in Video Quality Smoothing.- Shadow Removal in Sole Outdoor Image.- 3D Head Model Classification Using KCDA.- Framework for Pervasive Web Content Delivery.- Region-Based Semantic Similarity Propagation for Image Retrieval.Scalable bitstreams are used today to contribute to the Universal Multimedia Access (UMA) philosophy, i.e., accessing multimedia anywhere, at anytime, and on any device. Bitstream structure description languages provide means to adapt scalable bitstreams in order to extract a lower quality version. This paper introduces a real-time XML-based framework for content adaptation by relying on BFlavor, a combination of two existing bitstream structure description languages (i.e., the MPEG-21 Bitstream Syntax Description Language (BSDL) and the Formal Language for Audio-Visual Representation extended with XML features (XFlavor)). In order to use BFlavor with state-of-the-art media formats, we have added support for transparent retrieval of context information and support for emulation prevention bytes. These extensions are validated by building a BFlavor code for bitstreams compliant with the scalable extension of the H.264/AVC specification. Performance measurements show that such a bitstream (containing a bitrate of 17 MBit/s) can be adapted in real-time by a BFlavor-based adaptation framework (with a speed of 27 MBit/s).

Exploiting quantization and spatial correlation in virtual-noise modeling for distributed video coding

Aiming for low-complexity encoding, video coders based on Wyner-Ziv theory are still unsuccessfully trying to match the performance of predictive video coders. One of the most important factors concerning the coding performance of distributed coders is modeling and estimating the correlation between the original video signal and its temporal prediction generated at the decoder. One of the problems of the state-of-the-art correlation estimators is that their performance is not consistent across a wide range of video content and different coding settings. To address this problem we have developed a correlation model able to adapt to changes in the content and the coding parameters by exploiting the spatial correlation of the video signal and the quantization distortion. In this paper we describe our model and present experiments showing that our model provides average bit rate gains of up to 12% and average PSNR gains of up to 0.5dB when compared to the state-of-the-art models. The experiments suggest that the performance of distributed coders can be significantly improved by taking video content and coding parameters into account.

Introducing skip mode in distributed video coding

Although it was proven in the 1970s already by Wyner and Ziv and Slepian and Wolf that, under certain conditions, the same rate-distortion boundaries exist for distributed video coding (DVC) systems as for traditional predicting systems, until now no practical DVC system has been developed that even comes close to the performance of state-of-the-art video codecs such as H.264/AVC in terms of rate-distortion. Some important factors for this are the lower accuracy of the motion estimation performed at the decoder, the inaccurate modeling of the correlation between the side information and the original frame, and the absence in most state-of-the-art DVC systems of anything conceptually similar to the notion of skipped macroblocks in predictive coding systems. This paper proposes an extension of a state-of-the-art transform domain residual DVC system with an implementation of skip mode. The skip mode has an impact at two different places: in the turbo decoder, more specifically the soft input, soft output (SISO) convolutional decoder, and in the puncturing of the parity bits. Results show average bitrate gains up to 39% (depending on the sequence) achieved by combining both approaches. Furthermore, a hybrid video codec is presented where the motion estimation task is shifted back to the encoder. This results in a drastic increase in encoder complexity, but also in a drastic performance gain in terms of rate-distortion, with average bitrate savings up to 60% relative to the distributed video codec. In the hybrid video codec, smaller but still important average bitrate gains are achieved by implementing skip mode: up to 24%.

Accurate Correlation Modeling for Transform-Domain Wyner-Ziv Video Coding

In Wyner-Ziv (WZ) video coding, low-complexity encoding is achieved by generating the prediction signal only at the decoder. An accurate model of the correlation between the original frame and its prediction is necessary for efficient coding. Firstly, we propose an improvement for the pixel-domain correlation estimation. In transform-domain WZ video coding current models estimate the necessary correlation parameters directly in the transform domain. We propose an alternative approach, where an accurate estimation in the pixel domain is followed by a novel method of transforming the pixel-domain estimation into the transform domain. The experimental results show that our model leads to average bit-rate gain of 3.5---8%.

A performance evaluation of the data partitioning tool in H.264/AVC

In order to be able to better cope with packet loss, H.264/AVC, besides offering superior coding efficiency, also comes with a number of error resilience tools. The goal of these tools is to enable the decoding of a bitstream containing encoded video, even when parts of it are missing. On top of that, the visual quality of the decoded video should remain as high as possible. In this paper, we will discuss and evaluate one of these tools, in particular the data partitioning tool. Experimental results will show that using data partitioning can significantly improve the quality of a video sequence when packet loss occurs. However, this is only possible if the channel used for transmitting the video allows selective protection of the different data partitions. In the most extreme case, an increase in PSNR of up to 9.77 dB can be achieved. This paper will also show that the overhead caused by using data partitioning is acceptable. In terms of bit rate, the overhead amounts to approximately 13 bytes per slice. In general, this is less than 1% of the total bit rate. On top of that, using constrained intra prediction, which is required to fully exploit data partitioning, causes a decrease in quality of about 0.5 dB for high quality video and between 1 and 2 dB for low quality video.

Flexible distribution of complexity by hybrid predictive-distributed video coding

There is currently limited flexibility for distributing complexity in a video coding system. While rate-distortion-complexity (RDC) optimization techniques have been proposed for conventional predictive video coding with encoder-side motion estimation, they fail to offer true flexible distribution of complexity between encoder and decoder since the encoder is assumed to have always more computational resources available than the decoder. On the other hand, distributed video coding solutions with decoder-side motion estimation have been proposed, but hardly any RDC optimized systems have been developed. To offer more flexibility for video applications involving multi-tasking or battery-constrained devices, in this paper, we propose a codec combining predictive video coding concepts and techniques from distributed video coding and show the flexibility of this method in distributing complexity. We propose several modes to code frames, and provide complexity analysis illustrating encoder and decoder computational complexity for each mode. Rate distortion results for each mode indicate that the coding efficiency is similar. We describe a method to choose which mode to use for coding each inter frame, taking into account encoder and decoder complexity constraints, and illustrate how complexity is distributed more flexibly.