Stijn Notebaert

Architectures for Fast Transcoding of H.264/AVC to Quality-Scalable SVC Streams

The scalable extension of H.264/AVC (SVC) was recently standardized, and offers scalability at a minor penalty in rate-distortion efficiency when compared to single-layer H.264/AVC coding. In SVC, a scaled version of the original video sequence can easily be extracted by dropping layers from the stream. However, most of the video content nowadays is still produced in a single-layer format. While decoding and reencoding is a possible solution to introduce scalability in the existing bitstreams, this is an approach which requires a tremendous amount of time and effort. In this paper, we show that transcoding can be used to intelligently derive scalable bitstreams from existing single-layer streams. We focus on SNR scalability, and introduce techniques that are able to create multiple quality layers in the bitstreams. We also discuss bitstream rewriting from SVC to H.264/AVC, and examine how our newly proposed architectures can benefit from the changes that were introduced for bitstream rewriting. Architectures with different rate distribution flexibility and computational complexity are discussed. Rate-distortion performance of transcoding is shown to be comparable to that of reencoding at a fraction of the time needed for the latter.

A novel hybrid requantization transcoding scheme for H.264/AVC

Transcoding is a fast and elegant solution for the adaptation of video content. In the case of bitrate adaptation, an important technique is requantization transcoding. In this paper, we extend our previous work, that focused on requantization of intra-coded pictures, to P and B pictures. We show that by using a combination of techniques, depending on the slice and macroblock type, improved results are obtained when compared to previously existent architectures. We also show that in the important case of transcoding to low bitrates, results of this hybrid architecture approximate the rate-distortion performance of the computationally complex decoder-encoder cascade.

Requantization transcoding of H.264/AVC bitstreams for intra 4 x 4 prediction modes

Efficient bitrate reduction of video content is necessary in order to satisfy the different constraints imposed by decoding devices and transmission networks. Requantization is a fast technique for bitrate reduction, and has been successfully applied for MPEG-2 bitstreams. Because of the newly introduced intra prediction in H.264/AVC, the existing techniques are rendered useless. In this paper we examine requantization transcoding of H.264/AVC bitstreams, focusing on the intra 4x4 prediction modes. Two architectures are proposed, one in the pixel domain and the other in the frequency domain, that compensate tlie drift introduced by the requantization of intra 4x4 predicted blocks. Experimental results show that these architectures perform approximately equally well as the full decode and recode architecture for low to medium bitrates. Because of the reduced computational complexity of these architectures, in particular the frequency-domain compensation architecture, they are highly suitable for real-time adaptation of video content.Although no scalability is explicitly defined in the H.264/ AVC specification, some forms of scalability can be achieved by using the available coding tools in a creative way. In this paper we will explain how to use the data partitioning tool to perform a coarse form of SNR scalability. The impact of various parameters, including the presence of IDR frames and the number of intra-coded macroblocks per frame, on bit rate and bit rate savings and on quality and quality loss will be discussed. Furthermore we will introduce and elaborate a possible use case for the technique proposed in this paper.MPEG-21 BSDL offers a solution for exposing the structure of a binary media resource as an XML description, and for the generation of a tailored media resource using a transformed XML description. The main contribution of this paper is the introduction of a real-time work flow for the XML-driven adaptation of H.264/AVC bitstreams in the temporal domain. This real-time approach, which is in line with the vision of MPEG-21 BSDL, is made possible by two key technologies: BFlavor (BSDL + XFlavor) for the efficient generation of XML descriptions and Streaming Transformations for XML (STX) for the efficient transformation of these descriptions. Our work flow is validated in several applications, all using H.264/AVC bitstreams: the exploitation and emulation of temporal scalability, as well as the creation of video skims using key frame selection. Special attention is paid to the deployment of hierarchical B pictures and to the use of placeholder slices for synchronization purposes. Extensive performance data arc also provided.Expressive Speech Recognition and Synthesis as Enabling Technologies for Affective Robot-Child Communication.- Embodied Conversational Agents: Computing and Rendering Realistic Gaze Patterns.- DBN Based Models for Audio-Visual Speech Analysis and Recognition.- An Extensive Method to Detect the Image Digital Watermarking Based on the Known Template.- Fast Mode Decision Algorithm in H.263+/H.264 Intra Transcoder.- Binary Erasure Codes for Packet Transmission Subject to Correlated Erasures.- Image Desynchronization for Secure Collusion-Resilient Fingerprint in Compression Domain.- A Format-Compliant Encryption Framework for JPEG2000 Image Code-Streams in Broadcasting Applications.- Euclidean Distance Transform of Digital Images in Arbitrary Dimensions.- JPEG2000 Steganography Possibly Secure Against Histogram-Based Attack.- Perceptual Depth Estimation from a Single 2D Image Based on Visual Perception Theory.- A System for Generating Personalized Virtual News.- Image Fingerprinting Scheme for Print-and-Capture Model.- 16x16 Integer Cosine Transform for HD Video Coding.- Heegard-Berger Video Coding Using LMMSE Estimator.- Real-Time BSD-Driven Adaptation Along the Temporal Axis of H.264/AVC Bitstreams.- Optimal Image Watermark Decoding.- Diagonal Discrete Cosine Transforms for Image Coding.- Synthesizing Variational Direction and Scale Texture on Planar Region.- Fast Content-Based Image Retrieval Based on Equal-Average K-Nearest-Neighbor Search Schemes.- Characterizing User Behavior to Improve Quality of Streaming Service over P2P Networks.- Interacting Activity Recognition Using Hierarchical Durational-State Dynamic Bayesian Network.- Improving the Image Retrieval Results Via Topic Coverage Graph.- Relevance Feedback for Sketch Retrieval Based on Linear Programming Classification.- Hierarchical Motion-Compensated Frame Interpolation Based on the Pyramid Structure.- Varying Microphone Patterns for Meeting Speech Segmentation Using Spatial Audio Cues.- Region-Based Sub-pixel Motion Estimation from Noisy, Blurred, and Down-Sampled Sequences.- Differential Operation Based Palmprint Authentication for Multimedia Security.- A Broadcast Model for Web Image Annotation.- An Approach to the Compression of Residual Data with GPCA in Video Coding.- A Robust Approach for Object Recognition.- A Novel Method for Spoken Text Feature Extraction in Semantic Video Retrieval.- A Semantic Image Category for Structuring TV Broadcast Video Streams.- Markov Chain Monte Carlo Super-Resolution Image Reconstruction with Simultaneous Adaptation of the Prior Image Model.- Text Detection in Images Using Texture Feature from Strokes.- Robust Mandarin Speech Recognition for Car Navigation Interface.- GKDA: A Group-Based Key Distribution Algorithm for WiMAX MBS Security.- A Watermarking Algorithm for JPEG File.- SNR Scalability in H.264/AVC Using Data Partitioning.- A Real-Time XML-Based Adaptation System for Scalable Video Formats.- Generic, Scalable Multimedia Streaming and Delivery with Example Application for H.264/AVC.- Shape-Based Image Retrieval in Botanical Collections.- Macroblock Mode Decision Scheme for Fast Encoding in H.264/AVC.- A Mathematical Model for Interaction Analysis Between Multiview Video System and User.- Motion Composition of 3D Video.- EKM: An Efficient Key Management Scheme for Large-Scale Peer-to-Peer Media Streaming.- Using Earth Movers Distance for Audio Clip Retrieval.- Streaming-Mode MB-Based Integral Image Techniques for Fast Multi-view Video Illumination Compensation.- A Motion Vector Predictor Architecture for AVS and MPEG-2 HDTV Decoder.- Inter-camera Coding of Multi-view Video Using Layered Depth Image Representation.- Optimal Priority Packetization with Multi-layer UEP for Video Streaming over Wireless Network.- A Multi-channel MAC Protocol with Dynamic Channel Allocation in CDMA Ad Hoc Networks.- Fuzzy Particle Swarm Optimization Clustering and Its Application to Image Clustering.- A New Fast Motion Estimation for H.264 Based on Motion Continuity Hypothesis.- Statistical Robustness in Multiplicative Watermark Detection.- Adaptive Visual Regions Categorization with Sets of Points of Interest.- A Publishing Framework for Digitally Augmented Paper Documents: Towards Cross-Media Information Integration.- Web-Based Semantic Analysis of Chinese News Video.- A Quality-Controllable Encryption for H.264/AVC Video Coding.- Texture Synthesis Based on Minimum Energy Cut and Its Applications.- Unifying Keywords and Visual Features Within One-Step Search for Web Image Retrieval.- Dynamic Bandwidth Allocation for Stored Video Under Renegotiation Frequency Constraint.- Online Selection of Discriminative Features Using Bayes Error Rate for Visual Tracking.- Interactive Knowledge Integration in 3D Cloth Animation with Intelligent Learning System.- Multi-view Video Coding with Flexible View-Temporal Prediction Structure for Fast Random Access.- Squeezing the Auditory Space: A New Approach to Multi-channel Audio Coding.- Video Coding by Texture Analysis and Synthesis Using Graph Cut.- Multiple Description Coding Using Adaptive Error Recovery for Real-Time Video Transmission.- An Improved Motion Vector Prediction Scheme for Video Coding.- Classifying Motion Time Series Using Neural Networks.- Estimating Intervals of Interest During TV Viewing for Automatic Personal Preference Acquisition.- Image Annotations Based on Semi-supervised Clustering with Semantic Soft Constraints.- Photo Retrieval from Personal Memories Using Generic Concepts.- PanoWalk: A Remote Image-Based Rendering System for Mobile Devices.- A High Quality Robust Watermarking Scheme.- An Association Rule Mining Approach for Satellite Cloud Images and Rainfall.- AVAS: An Audio-Visual Attendance System.- Improved POCS-Based Deblocking Technique Using Wavelet Transform in Block Coded Image.- Sketch Case Based Spatial Topological Data Retrieval.- Providing Consistent Service for Structured P2P Streaming System.- Adaptive Search Range Scaling for B Pictures Coding.- Video QoS Monitoring and Control Framework over Mobile and IP Networks.- Extracting Moving / Static Objects of Interest in Video.- Building a Personalized Music Emotion Prediction System.- Video Segmentation Using Joint Space-Time-Range Adaptive Mean Shift.- EagleRank: A Novel Ranking Model for Web Image Search Engine.- Color Image Enhancement Using the Laplacian Pyramid.- 3D Mesh Construction from Depth Images with Occlusion.- An Eigenbackground Subtraction Method Using Recursive Error Compensation.- Attention Information Based Spatial Adaptation Framework for Browsing Videos Via Mobile Devices.- Style Strokes Extraction Based on Color and Shape Information.- Requantization Transcoding of H.264/AVC Bitstreams for Intra 4x4 Prediction Modes.- Prediction Algorithms in Large Scale VOD Services on Grid Infrastructure.- A Hierarchical Framework for Fast Macroblock Prediction Mode Decision in H.264.- Compact Representation for Large-Scale Clustering and Similarity Search.- Robust Recognition of Noisy and Partially Occluded Faces Using Iteratively Reweighted Fitting of Eigenfaces.- Pitching Shot Detection Based on Multiple Feature Analysis and Fuzzy Classification.- Color Changing and Fading Simulation for Frescoes Based on Empirical Knowledge from Artists.- A Novel Spatial-Temporal Position Prediction Motion-Compensated Interpolation for Frame Rate Up-Conversion.- Web Image Clustering with Reduced Keywords and Weighted Bipartite Spectral Graph Partitioning.- An Architecture to Connect Disjoint Multimedia Networks Based on Nodes Capacity.- Quantitative Measure of Inlier Distributions and Contour Matching for Omnidirectional Camera Calibration.- High-Speed All-in-Focus Image Reconstruction by Merging Multiple Differently Focused Images.- A Real-Time Video Deinterlacing Scheme for MPEG-2 to AVS Transcoding.- Persian Text Watermarking.- Three Dimensional Reconstruction of Structured Scenes Based on Vanishing Points.- Parallel Processing for Reducing the Bottleneck in Realtime Graphics Rendering.- Distributed Data Visualization Tools for Multidisciplinary Design Optimization of Aero-crafts.- An Efficient Clustering and Indexing Approach over Large Video Sequences.- An Initial Study on Progressive Filtering Based on Dynamic Programming for Query-by-Singing/Humming.- Measuring Multi-modality Similarities Via Subspace Learning for Cross-Media Retrieval.- SNR-Based Bit Allocation in Video Quality Smoothing.- Shadow Removal in Sole Outdoor Image.- 3D Head Model Classification Using KCDA.- Framework for Pervasive Web Content Delivery.- Region-Based Semantic Similarity Propagation for Image Retrieval.Scalable bitstreams are used today to contribute to the Universal Multimedia Access (UMA) philosophy, i.e., accessing multimedia anywhere, at anytime, and on any device. Bitstream structure description languages provide means to adapt scalable bitstreams in order to extract a lower quality version. This paper introduces a real-time XML-based framework for content adaptation by relying on BFlavor, a combination of two existing bitstream structure description languages (i.e., the MPEG-21 Bitstream Syntax Description Language (BSDL) and the Formal Language for Audio-Visual Representation extended with XML features (XFlavor)). In order to use BFlavor with state-of-the-art media formats, we have added support for transparent retrieval of context information and support for emulation prevention bytes. These extensions are validated by building a BFlavor code for bitstreams compliant with the scalable extension of the H.264/AVC specification. Performance measurements show that such a bitstream (containing a bitrate of 17 MBit/s) can be adapted in real-time by a BFlavor-based adaptation framework (with a speed of 27 MBit/s).

Requantization transcoding for H.264/AVC video coding

In this paper, efficient solutions for requantization transcoding in H.264/AVC are presented. By requantizing residual coefficients in the bitstream, different error components can appear in the transcoded video stream. Firstly, a requantization error is present due to successive quantization in encoder and transcoder. In addition to the requantization error, the loss of information caused by coarser quantization will propagate due to dependencies in the bitstream. Because of the use of intra prediction and motion-compensated prediction in H.264/AVC, both spatial and temporal drift propagation arise in transcoded H.264/AVC video streams. The spatial drift in intra-predicted blocks results from mismatches in the surrounding prediction pixels as a consequence of requantization. In this paper, both spatial and temporal drift components are analyzed. As is shown, spatial drift has a determining impact on the visual quality of transcoded video streams in H.264/AVC. In particular, this type of drift results in serious distortion and disturbing artifacts in the transcoded video stream. In order to avoid the spatially propagating distortion, we introduce transcoding architectures based on spatial compensation techniques. By combining the individual temporal and spatial compensation approaches and applying different techniques based on the picture and/or macroblock type, overall architectures are obtained that provide a trade-off between computational complexity and rate-distortion performance. The complexity of the presented architectures is significantly reduced when compared to cascaded decoder-encoder solutions, which are typically used for H.264/AVC transcoding. The reduction in complexity is particularly large for the solution which uses spatial compensation only. When compared to traditional solutions without spatial compensation, both visual and objective quality results are highly improved.

Efficient spatial resolution reduction transcoding for H.264/AVC

In this paper, we present a spatial resolution reduction transcoding architecture for H.264/AVC, which extends open-loop transcoding with a low-complexity compensation technique in the reduced-resolution domain. The proposed architecture removes visual artifacts from the transcoded sequence, while keeping complexity significantly lower than more traditional cascaded decoder-encoder architectures. The refinement step of the proposed architecture can be used to further improve rate-distortion performance, at the cost of additional complexity. In this way, a dynamic-complexity transcoder is rendered possible.

Advanced bitstream rewriting from H.264/AVC to SVC

In previous work, we introduced an H.264/AVC-to-SVC transcoder for creating SVC streams with multiple quality layers from a single-layer H.264/AVC stream. This architecture was able to restrain the error drift due to requantization of the residual coefficients. In this paper, we show that it is possible to further reduce the complexity, and completely eliminate the drift in the enhancement layer, by making use of the bitstream rewriting functionality in SVC. We propose different novel architectures, which are able to flexibly distribute the data among the different created layers.

GPU-assisted decoding of video samples represented in the YCoCg-R color space

Although pixel shaders were designed for the creation of programmable rendering effects, they can also be used as generic processing units for vector data. In this paper, attention is paid to an implementation of the YCoCg-R to RGB color space transform, as defined in the H.264/AVC Fidelity Range Extensions, by making use of pixel shaders. Our results show that a significant speedup can be achieved by relying on the processing power of the GPU, relative to the CPU. To be more specific, high definition video (1080p), represented in the YCoCg-R color space, could be decoded to RGB at 30 Hz on a PC with an AMD Athlon XP 2800+ CPU, an AGP bus and an NVIDIA GeForce 6800 graphics card, an effort that could not be realized in real-time by the CPU.

Transcoding from H.264/AVC to SVC with CGS Layers

In this paper, we show that it is possible to efficiently transcode single-layer H.264/AVC bitstreams to SNR-scalable SVC streams with CGS layers. Using requantization error compensation techniques, our architecture is able to restrain the drift that arises due to the absence of a closed prediction loop at the different dependency layers. Implementation results show that transcoding can generate SVC bit-streams with rate-distortion performance approaching that of the rate-distortion optimal encoder within 1 to 2 dB. Gains of more than 2 dB are obtained when compared to open-loop requantization.

Requantization transcoding of H.264/AVC bitstreams for intra 4×4 prediction modes

Efficient bitrate reduction of video content is necessary in order to satisfy the different constraints imposed by decoding devices and transmission networks. Requantization is a fast technique for bitrate reduction, and has been successfully applied for MPEG-2 bitstreams. Because of the newly introduced intra prediction in H.264/AVC, the existing techniques are rendered useless. In this paper we examine requantization transcoding of H.264/AVC bitstreams, focusing on the intra 4×4 prediction modes. Two architectures are proposed, one in the pixel domain and the other in the frequency domain, that compensate the drift introduced by the requantization of intra 4×4 predicted blocks. Experimental results show that these architectures perform approximately equally well as the full decode and recode architecture for low to medium bitrates. Because of the reduced computational complexity of these architectures, in particular the frequency-domain compensation architecture, they are highly suitable for real-time adaptation of video content.

Dyadic spatial resolution reduction transcoding for H.264/AVC

In this paper, we examine spatial resolution downscaling transcoding for H.264/AVC video coding. A number of advanced coding tools limit the applicability of techniques, which were developed for previous video coding standards. We present a spatial resolution reduction transcoding architecture for H.264/AVC, which extends open-loop transcoding with a low-complexity compensation technique in the reduced-resolution domain. The proposed architecture tackles the problems in H.264/AVC and avoids visual artifacts in the transcoded sequence, while keeping complexity significantly lower than more traditional cascaded decoder–encoder architectures. The refinement step of the proposed architecture can be used to further improve rate-distortion performance, at the cost of additional complexity. In this way, a dynamic-complexity transcoder is rendered possible. We present a thorough investigation of the problems related to motion and residual data mapping, leading to a transcoding solution resulting in fully compliant reduced-size H.264/AVC bitstreams.