Johan De Praeter

Efficient Bit Rate Transcoding for High Efficiency Video Coding

High efficiency video coding (HEVC) shows a significant advance in compression efficiency and is considered to be the successor of H.264/AVC. To incorporate the HEVC standard into real-life network applications and a diversity of other applications, efficient bit rate adaptation (transrating) algorithms are required. A current problem of transrating for HEVC is the high computational complexity associated with the encoder part of such a cascaded pixel domain transcoder. This paper focuses on deriving an optimal strategy for reducing the transcoding complexity with a complexity-scalable scheme. We propose different transcoding techniques which are able to reduce the transcoding complexity in both CU and PU optimization levels. At the CU level, CUs can be evaluated in top-to-bottom or bottom-to-top flows, in which the coding information of the input video stream is utilized to reduce the number of evaluations or to early terminate certain evaluations. At the PU level, the PU candidates are adaptively selected based on the probability of PU sizes and the co-located input PU partitioning. Moreover, with the use of different proposed methods, a complexity-scalable transrating scheme can be achieved. Furthermore, the transcoding complexity can be effectively controlled by the machine learning based approach. Simulations show that the proposed techniques provide a superior transcoding performance compared to the state-of-the-art related works. Additionally, the proposed methods can achieve a range of trade-offs between transrating complexity and coding performance. From the proposed schemes, the fastest approach is able to reduce the complexity by 82% while keeping the bitrate loss below 3%.

Efficient transcoding for spatially misaligned compositions for HEVC

The visualization of (ultra) high-resolution compositions created from multiple input bitstreams requires several decoders at the receiving device. Therefore, not all devices can properly display such compositions. To address this problem, the input streams are decoded, merged into a single video, and re-encoded by a transcoder in the network. However, this approach requires a computationally complex re-encoding step. To reduce this complexity, information from the input bit-streams can be reused during transcoding. In HEVC, simply reusing the original encoding information is not compression efficient if the inserted content is not aligned with the grid of coded blocks. In this paper, we applied feature selection based on a decision tree, which was used in a fast HEVC transcoding model for misaligned content. The performance varies depending on the shift and average transform size in the original sequence, resulting in complexity reductions of up to 76%.

Fast simultaneous video encoder for adaptive streaming

Content providers create different versions of a video to accommodate different end-user devices and network conditions. However, each of these versions requires a resource intensive encoding process. To reduce the computational complexity of the encodings, this paper proposes a fast simultaneous encoder. This encoder takes a single video as input and creates a number of bit streams encoded with different parameters. Only one version of the video is created with a full encode, whereas encoding of the other versions is accelerated by exploiting the correlation with the fully encoded version using machine learning techniques. In a practical scenario, the fast simultaneous encoder achieves a complexity reduction of 67.3% with a bit rate increase of 5.2% compared to performing a full encode of each version.

Machine learning for arbitrary downsizing of pre-encoded video in HEVC

In this paper, we propose a machine learning based transcoding scheme for arbitrarily downsizing a pre-encoded High Efficiency Video Coding video. The spatial scaling factor can be freely selected to adapt the output bit rate to the bandwidth of the network. Furthermore, machine learning techniques can exploit the correlation between input and output coding information to predict the split-flag of coding units in a P-frame. We analyzed the performance of both offline and online training in the learning phase of transcoding. The experimental results show that the proposed techniques significantly reduce the transcoding complexity and achieve trade-offs between coding performance and complexity. In addition, we demonstrate that online training performs better than offline training.

Performance analysis of machine learning for arbitrary downsizing of pre-encoded HEVC video

Nowadays, broadcasters deliver ultra-high resolution video to their consumers. This live video is sent to a set-top box for display on a television. However, if one or more users in the home want to view the same video on their personal mobile devices with a lower display resolution and limited processing power, decoding the original ultra-high resolution video would result in stuttering and quickly drain the battery life on these devices. To enable a satisfactory consumer experience, the resolution of the video stream should be adapted to the target mobile device at the set-top box. The aim of this paper is to investigate the performance of different machine learning strategies to arbitrary downsize video pre-encoded with the high efficiency video coding standard (HEVC). These machine learning techniques exploit correlation between input and output coding information to predict the splitting behavior of HEVC coding units. Several machine learning algorithms are optimized. Additionally, both online and offline training strategies are tested. Of the tested algorithms, online-trained random forests achieve the best compression-efficiency with a bit rate increase of 5.4% and an average complexity reduction of 70%1.

Efficient Encoding of Interactive Personalized Views Extracted from Immersive Video Content

Traditional television limits people to a single viewpoint. However, with new technologies such as virtual reality glasses, the way in which people experience video will change. Instead of being limited to a single viewpoint, people will demand a more immersive experience that gives them a sense of being present in a sports stadium, a concert hall, or at other events. To satisfy these users, video such as 360-degree or panoramic video needs to be transported to their homes. Since these videos have an extremely high resolution, sending the entire video requires a high bandwidth capacity and also results in a high decoding complexity at the viewer. The traditional approach to this problem is to split the original video into tiles and only send the required tiles to the viewer. However, this approach still has a large bit rate overhead compared to sending only the required view. Therefore, we propose to send only a personalized view to each user. Since this paper focuses on reducing the computational cost of such a system, we accelerate the encoding of each personalized view based on coding information obtained from a pre-analysis on the entire ultra-high-resolution video. By doing this using the High Efficiency Video Coding Test Model (HM), the complexity of each individual encode of a personalized view is reduced by more than 96.5% compared to a full encode of the view. This acceleration results in a bit rate overhead of at most 19.5%, which is smaller compared to the bit rate overhead of the tile-based method.

Spatially misaligned HEVC transcoding with computational-complexity scalability

Two methods to transcode a spatially misaligned video sequence are proposed.The machine learning method outperforms the trivial method for all shifts.Both methods perform best when the shifts align with a grid.Predicting both CU and PU information is more efficient than CU only. In control rooms, video walls display footage from multiple sources. Often, a composition of these sources is sent to other devices in a single video stream. To minimize the computational complexity of this composition process, information from the original bitstreams can be reused. However, in High Efficiency Video Coding (HEVC), simply copying the original encoding decisions is not compression efficient if the individual videos are spatially misaligned with the grid of coded blocks of the composition. Our proposed HEVC-based transcoder reduces the computational complexity by predicting encoding decisions of misaligned sequences by using a trivial method or a more adaptive, computational-complexity scalable machine learning method. Higher compression efficiency is observed when more alignment is preserved with the original block grid. Overall, the machine learning method achieves a higher compression efficiency than the trivial method. Both methods attain a complexity reduction of up to 82% compared to the reference software.

Simultaneous encoder for high-dynamic-range and low-dynamic-range video

High-dynamic-range (HDR) technology is an emerging video technology that allows displays to produce a higher range of luminance to better approximate the range of brightness perceived by the human eye. However, during the transition to this new technology, not all consumer devices will support the full range of luminance values offered by HDR. In order to also support these devices with lower dynamic ranges, content providers have to supply multiple dynamic range versions to provide the best experience to all viewers. This means that the processing cost to compress these versions will be multiplied by the number of versions. As a solution, this paper proposes a simultaneous encoder based on high efficiency video coding. This encoder reuses parts of the coding information generated during compression of an HDR video to accelerate the encoding of a low-dynamicrange (LDR) version of the same video. The proposed method speeds up the encoder 299 times with a bit rate increase of 12.4% compared to a non-accelerated encode of the LDR version. This is more than 90 times faster compared to stateof- the-art fast encoding algorithms and allows simultaneous encoding of the two versions for approximately the computational cost of a single encoder.

Fast encoding for personalized views extracted from beyond high definition content

Broadcast providers are looking for new opportunities to increase user experience and user interaction on their content. Their main goal is to attract and preserve viewer attention to create a big and stable audience. This could be achieved with a second screen application that lets the users select their own viewpoint in an extremely high resolution video to direct their own first screen. By allowing the users to create their own personalized video stream, they become involved with the content creation itself. However, encoding a personalized view for each user is computationally complex. This paper describes a machine learning approach to speed up the encoding of each personal view. Simulation results of zoom, pan and tilt scenarios show bit rate increases between 2% and 9% for complexity reductions between 69% and 79% compared to full encoding.

A Motion Vector Re-Use Algorithm for H.264/AVC and HEVC Simultaneous Video Encoding

The rapid emergence and rise of a wide range of electronic devices has led to the need for providing very different levels of video transmission services. The capabilities and performance of these devices determine the type of video streams that they are able to decode. As a way to fulfil their requirements, this paper presents a heterogeneous simultaneous encoding framework of H.264/Advanced Video Coding (AVC) and High Efficiency Video Coding (HEVC) that shares information between encoders in order to reduce the overall encoding time. In this regard, the proposed approach utilizes the H.264/AVC motion vectors of the 16x16 pixels macroblocks as predictors for the HEVC prediction units. As a consequence, the size of the motion estimation search area can be significantly reduced. Results show that an encoding time reduction of 8.95% can be achieved with negligible losses in terms of rate-distortion.