Madhu Krishnan

Performance comparison of AV1, HEVC, and JVET video codecs on 360 (spherical) video

This paper compares the coding efficiency performance on 360 videos, of three software codecs: (a) AV1 video codec from the Alliance for Open Media (AOM); (b) the HEVC Reference Software HM; and (c) the JVET JEM Reference SW. Note that 360 video is especially challenging content, in that one codes full res globally, but typically looks locally (in a viewport), which magnifies errors. These are tested in two different projection formats ERP and RSP, to check consistency. Performance is tabulated for 1-pass encoding on two fronts: (1) objective performance based on end-to-end (E2E) metrics such as SPSNR-NN, and WS-PSNR, currently developed in the JVET committee; and (2) informal subjective assessment of static viewports. Constant quality encoding is performed with all the three codecs for an unbiased comparison of the core coding tools. Our general conclusion is that under constant quality coding, AV1 underperforms HEVC, which underperforms JVET. We also test with rate control, where AV1 currently underperforms the open source X265 HEVC codec. Objective and visual evidence is provided.

Deep learning techniques in video coding and quality analysis

Video coding is a powerful enabling technology for networked multimedia transmission and communication, that has been in constant improvement for decades. The upcoming VVC video codec, due in 2020, from the ITU|ISO/IEC standards committees, aims to achieve on the order of 1000:1 compression on high resolution and high dynamic range video, a stunning landmark. But the basic structure of codecs has remained largely unchanged over time, the gains obtained mainly through complexity increases. Moreover, video encoders have for decades used the same mean squared error, or sum of absolute differences, measure to optimize coding decisions. At the same time, the rapid rise of deep learning (DL) techniques poses the question: can DL fundamentally reshape how video is coded. While that question is highly complex, we first see a path for DL methods to make inroads into how video quality is measured. This in turn can also change how it is coded. In particular, we study a recently introduced video quality metric called VMAF and find ways to improve it further, which can lead to more powerful encoder designs that employ these measures in the coding decisions.

HDR compression in the JVET codec

This paper presents an advanced approach to HDR/WCG video coding developed at FastVDO called FVHDR, and built on top of the Versatile Video Coding (VVC) VTM1.0 test model of the Joint Video Exploration Team, a joint committee of ITU|ISO/IEC. A fully automatic adaptive video process that differs from a known HDR video processing chain (analogous to HDR10, and herein called “anchor”) developed recently in the standards committee JCTVC, is used. FVHDR works entirely within the framework of the VTM software model, but adds additional tools. These tools can become an integral part of a future video coding standard, or be extracted as additional pre- and post-processing chains. Reconstructed video sequences using FVHDR show an improved subjective visual quality to the output of the anchor. Moreover, the resultant SDR content generated by the data adaptive grading process is backward compatible.

Performance comparison of VVC, AV1, and HEVC on 8-bit and 10-bit content

This paper presents a study comparing the coding efficiency performance of three video codecs: (a) the Versatile Video Coding (VVC) Bench Mark Set 1 (BMS1); (b) AV1 codec of the Alliance for Open Media (AOM); and (c) the HEVC Main Profile Reference Software. Two approaches to coding were used: (i) constant quality (QP); and (ii) target bit rate (VBR). Constant quality encoding is performed with all the three codecs for an unbiased comparison of the core coding tools. Whereas, target bitrate coding is done with the AV1 codec to study the compression efficiency achieved with rate control, which can and does have a significant impact. Performance is tabulated for on two fronts: (1) objective performance based on PSNR’s and (2) informal subjective assessment. Our general conclusion derived from the assessment of objective metrics and subjective evaluation is that VVC (BMS1) appears to be superior to AV1 and HEVC under both constant quality and target bitrate coding constraints. AV1 shows superior coding gains with respect to HEVC under target bitrate coding, but in general has increased computational complexity and henceforth an encode time factor of 20 – 30 over HEVC.

Advanced single-stream HDR coding using JVET JEM with backward compatibility options

This paper presents a state of the art approach in HDR/WCG video coding developed at FastVDO called FVHDR, and based on the JEM 6 Test Model of the Joint Exploration Team, a joint committee of ITU|ISO/IEC. A fully automatic adaptive video process that differs from a known HDR video processing chain (analogous to HDR10, herein called “anchor”) developed recently in the standards committee JVET, is used. FVHDR works entirely within the framework of JEM software model, but adds additional tools. These tools can become an integral part of a future video coding standard, or be extracted as additional pre- and post-processing chains. Reconstructed video sequences using FVHDR show a subjective visual quality superior to the output of the anchor. Moreover the resultant SDR content generated by the data adaptive grading process is backward compatible. Representative objective results for the system include: results for DE100, and PSNRL100, were -13.4%, and -3.8% respectively.

FastVDO enhancements of the AV1 codec and comparison to HEVC and JVET codecs

This paper describes a study to investigate possible ways to improve the AV1 codec, in several directions, most particularly in the context of 10-bit HDR video content, and 8/10 bit image content. Applications to SDR video, and 360 content are discussed elsewhere. For HDR content, a data adaptive grading technique in conjunction with the AV1 codec is studied. For image content, lapped biorthogonal transforms for (near) lossless compression is studied. For scalability-type applications, we introduce advanced resampling filters which outperform current ones. It is asserted that useful improvements are possible in each of these categories. In particular, substantial value is offered in the coding of HDR content, very competitive with HEVC HDR10, in a coding framework offering backwards compatibility with SDR. We also provide a rudimentary comparison of AV1 to the standard HEVC as well as the developing JVET codecs.

Performance comparison of HEVC reference SW, x265 and VPX on 8-bit 1080p content

This paper presents a study comparing the coding efficiency performance of three software codecs: (a) the HEVC Main Profile Reference Software; (b) the x265 codec; and (c) VP10. Note here that we are specifically testing only 8-bit performance. Performance is tabulated for 1-pass encoding on two fronts: (1) objective performance (PSNR), (2) informal subjective assessment. Finally, two approaches to coding were used: (i) constant quality; and (ii) fixed bit rate. Constant quality encoding is performed with all the three codecs for an unbiased comparison of the core coding tools. Whereas target bitrate coding is done to study the compression efficiency achieved with rate control, which can and does have a significant impact. Our general conclusion is that under constant quality coding, the HEVC reference software appears to be superior to the other two, whereas with rate control and fixed rate coding, these codecs are more on an equal footing. We remark that this latter result may be partly or mainly due to the maturity of the various rate control mechanisms in these codecs.

FVP10: enhancements of VPX for SDR/HDR applications

This paper describes a study to investigate possible ways to improve the VPX codecs in the context of both 8-bit SDR video and 10-bit HDR video content, for two types of applications: streaming and high quality (near lossless) coding for content contribution editing. For SDR content, the following tools are investigated: (a) lapped biorthogonal transforms for near lossless applications; and (b) optimized resampling filter pairs for adaptive resolution coding in streaming applications. For HDR content, a data adaptive grading technique in conjunction with the VP9/VP10 encoder is studied. Both the objective metrics (measured using BD rate) and informal subjective visual quality assessments are recorded. It is asserted that useful improvements are possible in each of these categories. In particular, substantial value is offered in the coding of HDR content, and especially in creating a coding scheme offering backwards compatibility with SDR.

FV10: An efficient single-layer approach to HDR coding, with backward compatibility options

High Dynamic Range and Wide Color Gamut (HDR/WCG) video is now at the forefront of modern broadcast and other video delivery systems. The efficient transmission and display of such video over diverse networks and systems is an important problem. This paper presents a novel, state of the art approach in HDR/WCG video coding (called FV10) which uses a new, fully automatic video data adaptive regrading process, which converts HDR to Standard Dynamic Range (SDR). Our method differs from one developed recently in standards committees (the Joint Collaborative Team on Video Coding, or JCT-VC, of ITU|ISO/IEC), based on the HEVC Main10 Profile as the core codec, which is an HDR10 compliant system (“anchor”). FV10 also works entirely within the framework of HEVC Main10 Profile, but makes greater use of existing SEI messages. Reconstructed video using our methods show a subjective visual quality superior to the output of an example HDR10 anchor. Moreover, a usable backwards compatible SDR video is obtained as a byproduct in the processing chain, allowing service efficiencies. Representative objective results for the system include: results for RGB-PSNR, DE100, MD100, tOSNR-XYZ were -46.0%, -21.6%, -29.6%, 16.2% respectively.

Improvements on HDR10

This paper presents two approaches to coding HDR/WCG video, by modifying certain components of an HDR video processing chain developed recently in standards committees (the Joint Collaborative Team on Video Coding, or JCT-VC, of ITU|ISO/IEC), which is an HDR10 compliant system (the “anchor”). One approach, called FastVDO_ECHDR, differs from the anchor in two tools: the intermediate color representation, and the chroma resampling filters. A second approach, called FastVDO_HDR, further uses a new video data adaptive tuning process, which differs from the ST.2084 transfer function used in the anchor. It is asserted that both systems perform well, with subjective visual quality superior to the output of an HDR10 anchor. Representative objective results for these systems include: (a) FastVDO_ECHDR, results for RGB-PSNR, DE100, MD100 were -1.1%, 10.3%, 38% respectively; (b) FastVDO_HDR, results for RGB-PSNR, DE100, MD100, were - 43.7%, -3.3%, -33.7% respectively.