Jan De Cock

Parallel Deblocking Filtering in MPEG-4 AVC/H.264 on Massively Parallel Architectures

The deblocking filter in the MPEG-4 AVC/H.264 standard is computationally complex because of its high content adaptivity, resulting in a significant number of data dependencies. These data dependencies interfere with parallel filtering of multiple macroblocks (MBs) on massively parallel architectures. In this letter, we introduce a novel MB partitioning scheme for concurrent deblocking in the MPEG-4 AVC/H.264 standard, based on our idea of deblocking filter independency, a corrected version of the limited error propagation effect proposed in the letter. Our proposed scheme enables concurrent MB deblocking of luma samples with limited synchronization effort, independently of slice configuration, and is compliant with the MPEG-4 H.264/AVC standard. We implemented the method on the massively parallel architecture of the graphics processing unit (GPU). Experimental results show that our GPU implementation achieves faster-than real-time deblocking at 1309 frames per second for 1080p video pictures. Both software-based deblocking filters and state-of-the-art GPU-enabled algorithms are outperformed in terms of speed by factors up to 10.2 and 19.5, respectively, for 1080p video pictures.

Encryption for High Efficiency Video Coding with video adaptation capabilities

In this paper, we describe encryption possibilities for the High Efficiency Video Coding (HEVC) standard under development. Bitstream elements which maintain HEVC compatibility after encryption are listed and their impact on video adaptation is described. From this list, three bitstream elements are selected, namely intra prediction mode difference, motion vector difference sign, and residual sign. These elements provide good protection of the video information and result in 0.0% Bjontegaard delta bitrate increase because of their equal probability entropy encoding property.

Motion-based temporal transcoding from H.264/AVC-to-SVC in baseline profile

Video contents needs to be compressed in order to reduce the resources required for storage and network transmission. It is desirable that the encoded bitstreams are adaptable to the varying characteristics of consumer electronics devices and heterogeneous networks. Scalable Video Coding provides temporal, spatial and quality scalability by organizing the encoded bitstream into layers. Since the majority of the produced video content is currently encoded using H.264/AVC, it will be necessary to implement techniques for converting from single-layer H.264/AVC to scalable bitstreams. In this paper, a technique for transcoding from H.264/AVC-to-SVC with temporal scalability in Baseline Profile is discussed. Applying the presented approach, a reduction of 64% of the coding complexity is achieved while maintaining the coding efficiency.

A compressed-domain approach for shot boundary detection on H.264/AVC bit streams

The amount of digital video content has grown extensively during recent years, resulting in a rising need for the development of systems for automatic indexing, summarization, and semantic analysis. A prerequisite for video content analysis is the ability to discover the temporal structure of a video sequence. In this paper, a novel shot boundary detection technique is introduced that operates completely in the compressed domain using the H.264/AVC video standard. As this specification contains a number of new coding tools, the characteristics of a compressed bit stream are different from prior video specifications. Furthermore, the H.264/AVC specification introduces new coding structures such as hierarchical coding patterns, which can have a major influence on video analysis algorithms. First, a shot boundary detection algorithm is proposed which can be used to segment H.264/AVC bit streams based on temporal dependencies and spatial dissimilarities. This algorithm is further enhanced to exploit hierarchical coding patterns. As these sequences are characterized by a pyramidal structure, only a subset of frames needs to be considered during analysis, allowing the reduction of the computational complexity. Besides the increased efficiency, experimental results also show that the proposed shot boundary detection algorithm achieves a high accuracy.

Improved intra mode signaling for HEVC

In the current development of HEVC, compression performance improved significantly compared to H.264/AVC for both inter pictures and intra pictures. With intra compression, the main reason for this improvement is the large increase in intra prediction directions (up to 34). The downside of having a larger number of modes is that they increase the signaling overhead in the bitstream. In this paper, a low complexity intra mode prediction algorithm is proposed which improves the mode prediction accuracy. This is achieved by exploiting the correlation between the prediction directions of the neighboring prediction units and that of the encoded prediction unit. As a result, more efficient intra mode signaling can be achieved with minimal impact on encoder and decoder complexity. On average, 0.33% bitrate improvement is obtained by employing the proposed algorithm. For sequences that are encoded with a high number of directional intra modes, around 1% bitrate improvement is measured.

Requantization transcoding of H.264/AVC bitstreams for intra 4 x 4 prediction modes

Efficient bitrate reduction of video content is necessary in order to satisfy the different constraints imposed by decoding devices and transmission networks. Requantization is a fast technique for bitrate reduction, and has been successfully applied for MPEG-2 bitstreams. Because of the newly introduced intra prediction in H.264/AVC, the existing techniques are rendered useless. In this paper we examine requantization transcoding of H.264/AVC bitstreams, focusing on the intra 4x4 prediction modes. Two architectures are proposed, one in the pixel domain and the other in the frequency domain, that compensate tlie drift introduced by the requantization of intra 4x4 predicted blocks. Experimental results show that these architectures perform approximately equally well as the full decode and recode architecture for low to medium bitrates. Because of the reduced computational complexity of these architectures, in particular the frequency-domain compensation architecture, they are highly suitable for real-time adaptation of video content.Although no scalability is explicitly defined in the H.264/ AVC specification, some forms of scalability can be achieved by using the available coding tools in a creative way. In this paper we will explain how to use the data partitioning tool to perform a coarse form of SNR scalability. The impact of various parameters, including the presence of IDR frames and the number of intra-coded macroblocks per frame, on bit rate and bit rate savings and on quality and quality loss will be discussed. Furthermore we will introduce and elaborate a possible use case for the technique proposed in this paper.MPEG-21 BSDL offers a solution for exposing the structure of a binary media resource as an XML description, and for the generation of a tailored media resource using a transformed XML description. The main contribution of this paper is the introduction of a real-time work flow for the XML-driven adaptation of H.264/AVC bitstreams in the temporal domain. This real-time approach, which is in line with the vision of MPEG-21 BSDL, is made possible by two key technologies: BFlavor (BSDL + XFlavor) for the efficient generation of XML descriptions and Streaming Transformations for XML (STX) for the efficient transformation of these descriptions. Our work flow is validated in several applications, all using H.264/AVC bitstreams: the exploitation and emulation of temporal scalability, as well as the creation of video skims using key frame selection. Special attention is paid to the deployment of hierarchical B pictures and to the use of placeholder slices for synchronization purposes. Extensive performance data arc also provided.Expressive Speech Recognition and Synthesis as Enabling Technologies for Affective Robot-Child Communication.- Embodied Conversational Agents: Computing and Rendering Realistic Gaze Patterns.- DBN Based Models for Audio-Visual Speech Analysis and Recognition.- An Extensive Method to Detect the Image Digital Watermarking Based on the Known Template.- Fast Mode Decision Algorithm in H.263+/H.264 Intra Transcoder.- Binary Erasure Codes for Packet Transmission Subject to Correlated Erasures.- Image Desynchronization for Secure Collusion-Resilient Fingerprint in Compression Domain.- A Format-Compliant Encryption Framework for JPEG2000 Image Code-Streams in Broadcasting Applications.- Euclidean Distance Transform of Digital Images in Arbitrary Dimensions.- JPEG2000 Steganography Possibly Secure Against Histogram-Based Attack.- Perceptual Depth Estimation from a Single 2D Image Based on Visual Perception Theory.- A System for Generating Personalized Virtual News.- Image Fingerprinting Scheme for Print-and-Capture Model.- 16x16 Integer Cosine Transform for HD Video Coding.- Heegard-Berger Video Coding Using LMMSE Estimator.- Real-Time BSD-Driven Adaptation Along the Temporal Axis of H.264/AVC Bitstreams.- Optimal Image Watermark Decoding.- Diagonal Discrete Cosine Transforms for Image Coding.- Synthesizing Variational Direction and Scale Texture on Planar Region.- Fast Content-Based Image Retrieval Based on Equal-Average K-Nearest-Neighbor Search Schemes.- Characterizing User Behavior to Improve Quality of Streaming Service over P2P Networks.- Interacting Activity Recognition Using Hierarchical Durational-State Dynamic Bayesian Network.- Improving the Image Retrieval Results Via Topic Coverage Graph.- Relevance Feedback for Sketch Retrieval Based on Linear Programming Classification.- Hierarchical Motion-Compensated Frame Interpolation Based on the Pyramid Structure.- Varying Microphone Patterns for Meeting Speech Segmentation Using Spatial Audio Cues.- Region-Based Sub-pixel Motion Estimation from Noisy, Blurred, and Down-Sampled Sequences.- Differential Operation Based Palmprint Authentication for Multimedia Security.- A Broadcast Model for Web Image Annotation.- An Approach to the Compression of Residual Data with GPCA in Video Coding.- A Robust Approach for Object Recognition.- A Novel Method for Spoken Text Feature Extraction in Semantic Video Retrieval.- A Semantic Image Category for Structuring TV Broadcast Video Streams.- Markov Chain Monte Carlo Super-Resolution Image Reconstruction with Simultaneous Adaptation of the Prior Image Model.- Text Detection in Images Using Texture Feature from Strokes.- Robust Mandarin Speech Recognition for Car Navigation Interface.- GKDA: A Group-Based Key Distribution Algorithm for WiMAX MBS Security.- A Watermarking Algorithm for JPEG File.- SNR Scalability in H.264/AVC Using Data Partitioning.- A Real-Time XML-Based Adaptation System for Scalable Video Formats.- Generic, Scalable Multimedia Streaming and Delivery with Example Application for H.264/AVC.- Shape-Based Image Retrieval in Botanical Collections.- Macroblock Mode Decision Scheme for Fast Encoding in H.264/AVC.- A Mathematical Model for Interaction Analysis Between Multiview Video System and User.- Motion Composition of 3D Video.- EKM: An Efficient Key Management Scheme for Large-Scale Peer-to-Peer Media Streaming.- Using Earth Movers Distance for Audio Clip Retrieval.- Streaming-Mode MB-Based Integral Image Techniques for Fast Multi-view Video Illumination Compensation.- A Motion Vector Predictor Architecture for AVS and MPEG-2 HDTV Decoder.- Inter-camera Coding of Multi-view Video Using Layered Depth Image Representation.- Optimal Priority Packetization with Multi-layer UEP for Video Streaming over Wireless Network.- A Multi-channel MAC Protocol with Dynamic Channel Allocation in CDMA Ad Hoc Networks.- Fuzzy Particle Swarm Optimization Clustering and Its Application to Image Clustering.- A New Fast Motion Estimation for H.264 Based on Motion Continuity Hypothesis.- Statistical Robustness in Multiplicative Watermark Detection.- Adaptive Visual Regions Categorization with Sets of Points of Interest.- A Publishing Framework for Digitally Augmented Paper Documents: Towards Cross-Media Information Integration.- Web-Based Semantic Analysis of Chinese News Video.- A Quality-Controllable Encryption for H.264/AVC Video Coding.- Texture Synthesis Based on Minimum Energy Cut and Its Applications.- Unifying Keywords and Visual Features Within One-Step Search for Web Image Retrieval.- Dynamic Bandwidth Allocation for Stored Video Under Renegotiation Frequency Constraint.- Online Selection of Discriminative Features Using Bayes Error Rate for Visual Tracking.- Interactive Knowledge Integration in 3D Cloth Animation with Intelligent Learning System.- Multi-view Video Coding with Flexible View-Temporal Prediction Structure for Fast Random Access.- Squeezing the Auditory Space: A New Approach to Multi-channel Audio Coding.- Video Coding by Texture Analysis and Synthesis Using Graph Cut.- Multiple Description Coding Using Adaptive Error Recovery for Real-Time Video Transmission.- An Improved Motion Vector Prediction Scheme for Video Coding.- Classifying Motion Time Series Using Neural Networks.- Estimating Intervals of Interest During TV Viewing for Automatic Personal Preference Acquisition.- Image Annotations Based on Semi-supervised Clustering with Semantic Soft Constraints.- Photo Retrieval from Personal Memories Using Generic Concepts.- PanoWalk: A Remote Image-Based Rendering System for Mobile Devices.- A High Quality Robust Watermarking Scheme.- An Association Rule Mining Approach for Satellite Cloud Images and Rainfall.- AVAS: An Audio-Visual Attendance System.- Improved POCS-Based Deblocking Technique Using Wavelet Transform in Block Coded Image.- Sketch Case Based Spatial Topological Data Retrieval.- Providing Consistent Service for Structured P2P Streaming System.- Adaptive Search Range Scaling for B Pictures Coding.- Video QoS Monitoring and Control Framework over Mobile and IP Networks.- Extracting Moving / Static Objects of Interest in Video.- Building a Personalized Music Emotion Prediction System.- Video Segmentation Using Joint Space-Time-Range Adaptive Mean Shift.- EagleRank: A Novel Ranking Model for Web Image Search Engine.- Color Image Enhancement Using the Laplacian Pyramid.- 3D Mesh Construction from Depth Images with Occlusion.- An Eigenbackground Subtraction Method Using Recursive Error Compensation.- Attention Information Based Spatial Adaptation Framework for Browsing Videos Via Mobile Devices.- Style Strokes Extraction Based on Color and Shape Information.- Requantization Transcoding of H.264/AVC Bitstreams for Intra 4x4 Prediction Modes.- Prediction Algorithms in Large Scale VOD Services on Grid Infrastructure.- A Hierarchical Framework for Fast Macroblock Prediction Mode Decision in H.264.- Compact Representation for Large-Scale Clustering and Similarity Search.- Robust Recognition of Noisy and Partially Occluded Faces Using Iteratively Reweighted Fitting of Eigenfaces.- Pitching Shot Detection Based on Multiple Feature Analysis and Fuzzy Classification.- Color Changing and Fading Simulation for Frescoes Based on Empirical Knowledge from Artists.- A Novel Spatial-Temporal Position Prediction Motion-Compensated Interpolation for Frame Rate Up-Conversion.- Web Image Clustering with Reduced Keywords and Weighted Bipartite Spectral Graph Partitioning.- An Architecture to Connect Disjoint Multimedia Networks Based on Nodes Capacity.- Quantitative Measure of Inlier Distributions and Contour Matching for Omnidirectional Camera Calibration.- High-Speed All-in-Focus Image Reconstruction by Merging Multiple Differently Focused Images.- A Real-Time Video Deinterlacing Scheme for MPEG-2 to AVS Transcoding.- Persian Text Watermarking.- Three Dimensional Reconstruction of Structured Scenes Based on Vanishing Points.- Parallel Processing for Reducing the Bottleneck in Realtime Graphics Rendering.- Distributed Data Visualization Tools for Multidisciplinary Design Optimization of Aero-crafts.- An Efficient Clustering and Indexing Approach over Large Video Sequences.- An Initial Study on Progressive Filtering Based on Dynamic Programming for Query-by-Singing/Humming.- Measuring Multi-modality Similarities Via Subspace Learning for Cross-Media Retrieval.- SNR-Based Bit Allocation in Video Quality Smoothing.- Shadow Removal in Sole Outdoor Image.- 3D Head Model Classification Using KCDA.- Framework for Pervasive Web Content Delivery.- Region-Based Semantic Similarity Propagation for Image Retrieval.Scalable bitstreams are used today to contribute to the Universal Multimedia Access (UMA) philosophy, i.e., accessing multimedia anywhere, at anytime, and on any device. Bitstream structure description languages provide means to adapt scalable bitstreams in order to extract a lower quality version. This paper introduces a real-time XML-based framework for content adaptation by relying on BFlavor, a combination of two existing bitstream structure description languages (i.e., the MPEG-21 Bitstream Syntax Description Language (BSDL) and the Formal Language for Audio-Visual Representation extended with XML features (XFlavor)). In order to use BFlavor with state-of-the-art media formats, we have added support for transparent retrieval of context information and support for emulation prevention bytes. These extensions are validated by building a BFlavor code for bitstreams compliant with the scalable extension of the H.264/AVC specification. Performance measurements show that such a bitstream (containing a bitrate of 17 MBit/s) can be adapted in real-time by a BFlavor-based adaptation framework (with a speed of 27 MBit/s).

Requantization transcoding for H.264/AVC video coding

In this paper, efficient solutions for requantization transcoding in H.264/AVC are presented. By requantizing residual coefficients in the bitstream, different error components can appear in the transcoded video stream. Firstly, a requantization error is present due to successive quantization in encoder and transcoder. In addition to the requantization error, the loss of information caused by coarser quantization will propagate due to dependencies in the bitstream. Because of the use of intra prediction and motion-compensated prediction in H.264/AVC, both spatial and temporal drift propagation arise in transcoded H.264/AVC video streams. The spatial drift in intra-predicted blocks results from mismatches in the surrounding prediction pixels as a consequence of requantization. In this paper, both spatial and temporal drift components are analyzed. As is shown, spatial drift has a determining impact on the visual quality of transcoded video streams in H.264/AVC. In particular, this type of drift results in serious distortion and disturbing artifacts in the transcoded video stream. In order to avoid the spatially propagating distortion, we introduce transcoding architectures based on spatial compensation techniques. By combining the individual temporal and spatial compensation approaches and applying different techniques based on the picture and/or macroblock type, overall architectures are obtained that provide a trade-off between computational complexity and rate-distortion performance. The complexity of the presented architectures is significantly reduced when compared to cascaded decoder-encoder solutions, which are typically used for H.264/AVC transcoding. The reduction in complexity is particularly large for the solution which uses spatial compensation only. When compared to traditional solutions without spatial compensation, both visual and objective quality results are highly improved.

GPU-assisted decoding of video samples represented in the YCoCg-R color space

Although pixel shaders were designed for the creation of programmable rendering effects, they can also be used as generic processing units for vector data. In this paper, attention is paid to an implementation of the YCoCg-R to RGB color space transform, as defined in the H.264/AVC Fidelity Range Extensions, by making use of pixel shaders. Our results show that a significant speedup can be achieved by relying on the processing power of the GPU, relative to the CPU. To be more specific, high definition video (1080p), represented in the YCoCg-R color space, could be decoded to RGB at 30 Hz on a PC with an AMD Athlon XP 2800+ CPU, an AGP bus and an NVIDIA GeForce 6800 graphics card, an effort that could not be realized in real-time by the CPU.

Fast transrating for high efficiency video coding based on machine learning

To incorporate the newly developed High Efficiency Video Coding (HEVC) standard in real-life network applications, efficient transrating algorithms are required. We propose a fast transrating scheme, based on the early prediction of the partition split-flags in P pictures. Using machine learning techniques, the correlation between co-located partitions at different quantizations is investigated. This results in a model which predicts the split-flag and gives the associated prediction accuracy so that the splitting process in the transcoder is optimized. At each partition depth, the model indicates whether the full rate-distortion cost evaluations should be performed at the current depth, or if the partition can be split immediately. Experimental results show that the proposed transcoder reduces the complexity of the transrating process by 76.04%, while maintaining the coding efficiency of a cascaded decoder-encoder.

Exploiting quantization and spatial correlation in virtual-noise modeling for distributed video coding

Aiming for low-complexity encoding, video coders based on Wyner-Ziv theory are still unsuccessfully trying to match the performance of predictive video coders. One of the most important factors concerning the coding performance of distributed coders is modeling and estimating the correlation between the original video signal and its temporal prediction generated at the decoder. One of the problems of the state-of-the-art correlation estimators is that their performance is not consistent across a wide range of video content and different coding settings. To address this problem we have developed a correlation model able to adapt to changes in the content and the coding parameters by exploiting the spatial correlation of the video signal and the quantization distortion. In this paper we describe our model and present experiments showing that our model provides average bit rate gains of up to 12% and average PSNR gains of up to 0.5dB when compared to the state-of-the-art models. The experiments suggest that the performance of distributed coders can be significantly improved by taking video content and coding parameters into account.