Sebastien Lasserre

Annealed learning based block transforms for HEVC video coding

Most of the recent video compression standards employ the Discrete Cosine Transform (DCT) for transforming the residual signal in order to remove spatial correlation and to achieve higher compression efficiency. However, by careful adaptation of transforms to the video content, a better set of integer transforms can be obtained. This paper proposes a new on-the-fly block-based transform optimization technique which involves first the classification of the residual blocks based on the cost of encoding the block, and then the generation of new optimized transforms for each class. An annealing based learning technique is further proposed in this paper in order to improve the performance of the optimization algorithm. The algorithm is tested using the latest HEVC test software where an optimized set of transforms is learned on the first frame of the HEVC test sequences and then applied to the subsequent frames in a Random Access (RA) and All Intra (AI) configuration. The results shows that this method can gain over 2% in terms of Bjontegaard Delta (BD)-rate compared to standard HEVC encoder in AI configuration and nearly 1.5% in RA.

Backward Compatible HDR Video Compression System

A new video compression scheme for High Dynamic Range (HDR) contents is presented. It is assumed that both Standard Dynamic Range (SDR) and HDR devices coexist in the ecosystem, and the scheme is constructed to ensure backward compatibility such that a single video stream carries both the original HDR video and an automatically generated SDR version of this video without noticeable overhead relatively to the distribution of the HDR video only. Compression performance are shown to be solidly improved compared to the conservative and non-backward-compatible approach using the HEVC Main10 codec and the Perceptual Quantizer (PQ) Electro-Optical Transfer Function (OETF) as non-linearity.

Transform Coding for On-the-Fly Learning Based Block Transforms

Block based video codecs developed in last decades have employed Discrete Cosine Transform (DCT) as a candidate transform for compacting the energy of the predicted residual data into fewer coefficients. However, DCT is not optimal for blocks with directional structure and high texture. Content-adaptive transforms can be used to achieve better compression. However, these adaptive transforms must be sent to the decoder, increasing the bit-rate in return. It is therefore important to take care of the overhead of sending the transforms in order to benefit from the better compaction of the adaptive transforms. This paper addresses this issue and provides methods to efficiently encode these adaptive transforms that are learned from the content. A statistical model is derived to determine the precision required to encode a transform with a negligible loss in the performance of adaptive transforms. Results shows that the overhead can be reduced by an average 70% for low QP and 86% for high QP values compared to encoding the complete transform with a little performance loss.

Low-Complexity Intra Coding for Scalable Extension of HEVC Based on Content Statistics

This paper presents a new approach for a scalable extension of High Efficiency Video Coding (HEVC) intra pictures. The proposed scalable intra codec targets very low complexity together with coding efficiency, and thus employs only one prediction mode, known as inter layer intra prediction. The main novelty of the codec is a new fast rate-distortion optimization approach, which is founded on the rate-distortion theory. It exploits precomputed offline information for the reduction of the computational complexity, and a new allocation process for the balancing of the coded information. The precomputed information is a set of optimal quantifiers and associated rate-distortion curves for predetermined statistical models that are used for efficiently coding the Discrete Cosine Transform channels. This low-complexity intra coding tool has been proposed to the scalable HEVC call for proposals. In the scalable standard testing conditions, the proposed intra codec shows compression performance with a bit-rate increase of around 10% only compared with the HEVC single-layer encoder.

CNN-based transform index prediction in multiple transforms framework to assist entropy coding

Recent work in video compression has shown that using multiple 2D transforms instead of a single transform in order to de-correlate residuals provides better compression efficiency. These transforms are tested competitively inside a video encoder and the optimal transform is selected based on the Rate Distortion Optimization (RDO) cost. However, one needs to encode a syntax to indicate the chosen transform per residual block to the decoder for successful reconstruction of the pixels. Conventionally, the transform index is binarized using fixed length coding and a CABAC context model is attached to it. In this work, we provide a novel method that utilizes Convolutional Neural Network to predict the chosen transform index from the quantized coefficient block. The prediction probabilities are used to binarize the index by employing a variable length coding instead of a fixed length coding. Results show that by employing this modified transform index coding scheme inside HEVC, one can achieve up to 0.59% BD-rate gain.

Improved coefficient coding for adaptive transforms in HEVC

Adaptive transform learning schemes have been extensively studied in the literature with a goal to achieve better compression efficiency compared to extensively used Discrete Cosine Transforms (DCT) inside a video codec. These transforms are learned offline on a large training set and are tested either in competition with or in place of the core transforms i.e. DCT. In our previous work, we proposed an alternative approach where a set of non-separable content-adaptive transforms are learned on-the-fly on a sequence and then tested on the same sequence in competition with the core transforms. In this paper, we propose to further improve the previously proposed learning scheme by improving the coding of transformed coefficients in context to the online learning of transforms. The first proposed method improves the convergence of the learning scheme by re-ordering the transformed coefficients at each iteration. The second proposed method improves the compression efficiency by modifying the coding of last significant coefficient position in context to adaptive transforms. The results shows that by combining the above proposed methods, one can achieve 1.2% gain on top of the previously proposed scheme.

Single-layer HDR video coding with SDR backward compatibility

The migration from High Definition (HD) TV to Ultra High Definition (UHD) is already underway. In addition to an increase of picture spatial resolution, UHD will bring more color and higher contrast by introducing Wide Color Gamut (WCG) and High Dynamic Range (HDR) video. As both Standard Dynamic Range (SDR) and HDR devices will coexist in the ecosystem, the transition from Standard Dynamic Range (SDR) to HDR will require distribution solutions supporting some level of backward compatibility. This paper presents a new HDR content distribution scheme, named SL-HDR1, using a single layer codec design and providing SDR compatibility. The solution is based on a pre-encoding HDR-to-SDR conversion, generating a backward compatible SDR video, with side dynamic metadata. The resulting SDR video is then compressed, distributed and decoded using standard-compliant decoders (e.g. HEVC Main 10 compliant). The decoded SDR video can be directly rendered on SDR displays without adaptation. Dynamic metadata of limited size are generated by the pre-processing and used to reconstruct the HDR signal from the decoded SDR video, using a post-processing that is the functional inverse of the pre-processing. Both HDR quality and artistic intent are preserved. Pre- and post-processing are applied independently per picture, do not involve any inter-pixel dependency, and are codec agnostic. Compression performance, and SDR quality are shown to be solidly improved compared to the non-backward and backward-compatible approaches, respectively using the Perceptual Quantization (PQ) and Hybrid Log Gamma (HLG) Opto-Electronic Transfer Functions (OETF).

High Dynamic Range and Wide Color Gamut Video Standardization — Status and Perspectives

With the advent of ultra-high-definition TV services, high dynamic range (HDR) and wide color gamut (WCG) have become two highly desired image quality improvements for delivering immersive video experiences to the consumer mass market. Capture and rendering technologies have reached a level of maturity that now allows HDR and WCG enhanced video to be deployed across all distribution channels — including: over-the-top providers, broadcasters, and packaged-media distributors (eg, Blu-ray discs). For this reason, standards bodies have made intense efforts to meet aggressive timelines so that the industry may have relevant standards in place for implementation of acceptable, concrete, and future-proof solutions that will work effectively on existing workflows and ecosystems in the short term and mid term. The aim of this chapter is to offer a global snapshot of the status of these standardization activities. This overview includes a specific focus on the High Efficiency Video Coding (aka H.265) standard, which already supports tools and features dedicated to HDR and WCG.