Evgeny Kaminsky

Dynamic computational complexity and bit allocation for optimizing H.264/AVC video compression

In this work we present a novel approach for optimizing H.264/AVC video compression by dynamically allocating computational complexity (such as a number of CPU clocks) and bits for encoding each coding element (basic unit) within a video sequence, according to its predicted MAD (mean absolute difference). Our approach is based on a computational complexity-rate-distortion (C-R-D) analysis, which adds a complexity dimension to the conventional rate-distortion (R-D) analysis. Both theoretically and experimentally, we prove that by implementing the proposed approach better results are achieved. In addition, we present a method and system for implementing the proposed approach, and for controlling computational complexity and bit allocation in real-time and off-line video coding. For allocating a corresponding group of coding modes and the quantization step-size, we develop computational complexity - complexity step - rate (C-I-R) and rate - quantization step-size - computational complexity (R-Q-C) models.

ROI adaptive scalable video coding for limited bandwidth wireless networks

Much of the attention in the field of video adaptation has been directed to the Scalable Video Coding (SVC), which is the extension of the H.264/AVC standard, since the bit-stream scalability for video is a desirable feature for many multimedia applications. The need for the scalability mainly arises from the need for spatial formats, bit rates or power. To fulfill these requirements, it would be beneficial to simultaneously transmit or store video in variety of spatial/temporal resolutions and qualities, leading to the video bit-stream scalability. Regions-of-interest (ROI) coding is a desirable feature in future applications of Scalable Video Coding. For those SVC applications, users at the decoder side usually wish to receive a high-quality decoded video stream, containing the desired ROI, which should be adaptively selected from the pre-encoded scalable bit-stream. In this work, we present a novel ROI adaptive scalable video coding scheme, enabling to adaptively set desirable ROI location, size, resolution and bit-rate, according to the limited network bandwidth and predefined settings. This, in turn, will enable providing an effective rate control for multiple ROIs, thereby enabling adaptively selecting the required ROI from multiple ROIs in the scalable bit-stream, and adaptively changing ROI spatial resolution, ROI visual quality or amount of bits allocated for each ROI, according to the network bandwidth and users settings (i.e., users display resolution, etc.).

Rate distortion optimization for efficient watermarking in the DCT domain

In this work we concentrate on a unique algorithm for watermarking, which is based on a modulo operation method. This method hides an integer number in the sum of the quantized DCT coefficients in a JPEG block. We use the Lagrangian optimization algorithm, which enables us to select the possible coefficients that will be changed in order to insert the watermark while controlling the bit rate and the distortion level (MSE). The uniqueness of this work is that by integrating the rate-distortion optimization method with the watermarking process we can control the rate of the watermarked image with minimum additional distortion. Initial results of our algorithm show that we can add 4-5 bits per DCT block (8times8 pixels), with a distortion level of 0.7 [dB] at JPEG compression quality above 75%.

Adaptive bit-rate control for Region-of-Interest Scalable Video Coding

In this work, we present a novel approach for providing an adaptive bit-rate control for the Region-of-Interest (ROI) Scalable Video Coding (SVC). Recently, much of the attention in the field of video adaptation has been directed to the SVC, which is the extension of the H.264/AVC standard. The need for the scalability mainly arises from the need for spatial formats, bit rates or power. To fulfill these requirements, it would be beneficial to simultaneously transmit or store video in variety of spatial/temporal resolutions and qualities, leading to the video bit-stream scalability. The ROI coding is a desirable feature in future SVC applications. For those SVC applications, users at the decoder side usually wish to receive a high-quality decoded video stream, containing the desired ROI, which should be adaptively selected from the pre-encoded scalable bit-stream. By implementing the proposed adaptive bit-rate control for the ROI SVC, smaller quantization parameters can be used, while obtaining the same compression rate, especially, in order to optimally employ the decoder computational resources and in order to achieve optimal video presentation quality. As a result, the SVC visual presentation quality at the decoder side will be significantly improved even for the decoders with very limited computational resources, which can be very useful for various mobile devices, such as cellular phones. Further, the proposed adaptive bit-rate control scheme is especially useful for future Internet and 4G applications with limited computational resources and/or with a limited channel bandwidth, such as video conferencing (between two or more mobile device users), video transrating, video transcoding between video coding standards, and many other applications.

Dynamically adjustable and scalable ROI video coding

Regions-of-interest (ROI) coding is a desirable feature in future applications of Scalable Video Coding (SVC), which is the extension of the H.264/AVC standard. For those SVC applications, users at the decoder side usually wish to receive a high-quality decoded video stream, containing the desired ROI, which should be adaptively selected from the pre-encoded scalable bit-stream. In this work, we present a novel improved dynamically adjustable and scalable ROI video coding scheme, enabling to adaptively set desirable ROI location, size, resolution and bit-rate, according to the network bandwidth and predefined settings. This, in turn, will enable providing an effective rate control for multiple ROIs, thereby enabling adaptively selecting the required ROI from multiple ROIs in the scalable bit-stream, and adaptively changing ROI spatial resolution, ROI visual quality or amount of bits allocated for each ROI, according to the bandwidth and users settings (i.e., according to the users display resolution, etc.).

Optimal transrating via DCT coefficients modification and dropping

Coefficient dropping is a common tool for video transrating in order to adapt it to various network bandwidth constraints. Several recent works propose Lagrangian optimization for finding the optimal retained coefficients for each coded block to achieve the desired bit-rate with minimum distortion. In this paper we extend the Lagrangian optimization procedure by modifying the coefficients prior to dropping them. The purpose of this Lagrangian extension is to provide higher PSNR than other coefficient dropping methods without increasing computational complexity significantly.

DCT-Domain Coder for Digital Video Applications

Recent home networking services such as next generation game consoles, video-on-demand, HDTV, etc. require faster video encoders and transcoders that avoid over-complicated transformations. In most popular digital video coding standards such as MPEG-X or H.26x, discrete cosine transform (DCT), inverse DCT (IDCT) transform, and block matching algorithms are essential elements in the coding process. The transforms between the pixel and DCT domains, are taking place during the operations and being only intermediate calculations, which complicate the encoding process. Thus, it is advantageous to reduce the number of transform operations between the spatial and frequency domains in order to simplify the overall encoder structure. In this paper we propose a DCT domain video encoder for reducing processing time. This is achieved by excluding the intermediate DCT and IDCT transforms from the conventional hybrid encoder. This is achieved by calculating the motion estimation and motion compensation via a spatial relationship between the DCT coefficients of a specific block and its sub- blocks. Experimental results of this method were compared to those of the conventional hybrid video encoder in the sense of visual quality and computational complexity. The comparison shows that the computational complexity of the proposed encoder is lower by 12.5% in comparison to conventional hybrid video coder for the same visual quality.

Buffer control in H.264/AVC applications by implementing dynamic complexity-rate-distortion analysis

In this work we present a novel optimal buffer control approach for H.264/AVC low-delay applications by dynamically allocating computational complexity (such as a number of CPU clocks) and bits for encoding each coding element (basic unit) within a video sequence, according to its predicted MAD (Mean Absolute Difference), while considering buffer size limitations (preventing underflow/overflow of the buffer) and buffer delay. Our buffer control approach is based on a computational complexity-rate-distortion (C-R-D) analysis, which adds a complexity dimension to the conventional rate-distortion (R-D) analysis. Both theoretically and experimentally, we prove that by implementing the proposed buffer control approach better results are achieved. In addition, we present an optimal buffer control method and system for implementing the proposed approach, and for controlling computational complexity and bit allocation in real-time and off-line video coding. We divide each frame into one or more basic units, wherein each basic unit consists of at least one macroblock (MB), whose contents are related to a number of coding modes. We determine how much computational complexity and bits should be allocated for encoding each basic unit, while considering buffer size limitations and buffer delay, and then we allocate a corresponding group of coding modes and a quantization step-size, according to the estimated distortion (calculated by a linear regression model) of each basic unit and according to the remaining computational complexity and bits for encoding remaining basic units. For allocating a corresponding group of coding modes and the quantization step-size, we develop computational complexity - complexity step - rate (C-I-R) and rate - quantization step-size - computational complexity (R-Q-C) models.

Efficient Real-Time Video-in-Video Insertion into a Pre-Encoded Video Stream

This work relates to the developing and implementing of an efficient method and system for the fast real-time Video-in-Video (ViV) insertion, thereby enabling efficiently inserting a video sequence into a predefined location within a pre-encoded video stream. The proposed method and system are based on dividing the video insertion process into two steps. The first step (i.e., the Video-in-Video Constrained Format (ViVCF) encoder) includes the modification of the conventional H.264/AVC video encoder to support the visual content insertion Constrained Format (CF), including generation of isolated regions without using the Frequent Macroblock Ordering (FMO) slicing, and to support the fast real-time insertion of overlays. Although, the first step is computationally intensive, it should to be performed only once even if different overlays have to be modified (e.g., for different users). The second step for performing the ViV insertion (i.e., the ViVCF inserter) is relatively simple (operating mostly in a bit-domain), and is performed separately for each different overlay. The performance of the presented method and system is demonstrated and compared with the H.264/AVC reference software (JM 12); according to our experimental results, there is a significantly low bit-rate overhead, while there is substantially no degradation in the PSNR quality.

Efficient real-time Video-in-Video insertion into a pre-encoded video stream for the H.264/AVC

This work relates to the developing and implementing of an efficient method and system for the fast real-time Video-in-Video (ViV) insertion, thereby enabling efficiently inserting a video sequence into a predefined location within a pre-encoded video stream (e.g., for inserting advertisements). The proposed ViV method enables achieving a significant performance over the conventional brute-force approaches (in terms of the bit-rate and insertion run-time) and enables supporting multiple rectangular overlays of various sizes (e.g., 16N × 16M sizes, where N and M are integers). The proposed method and system are based on dividing the video insertion process into two steps. The first step (i.e., the ViVCF encoder) includes modification of the conventional H.264/AVC video encoder to support the visual content insertion Constrained Format (CF), including generation of isolated regions without using the FMO (Frequent Macroblock Ordering) slicing, and to support the fast real-time insertion of overlays. Although, the first step is computationally intensive, it should to be performed only once even if different overlays have to be modified (e.g., for different users). The second step (i.e., the ViVCF inserter) is relatively simple (operating mostly in the bit-domain), and is performed separately for each different overlay.