Qifei Wang

Free Viewpoint Video Coding With Rate-Distortion Analysis

To improve free viewpoint video (FVV) coding efficiency and optimize the quality of the synthesized virtual view video, this paper proposes a depth-assisted FVV coding framework and analyzes the rate-distortion (R-D) property of the synthesized virtual view video in FVV coding. In the depth-assisted FVV coding framework, the depth assigned disparity compensated prediction is introduced to exploit the correlation between multiview video (MVV) and depth. To model the R-D property of the synthesized virtual view video, a region-based view synthesis distortion estimation approach is investigated with respect to the distortion of MVV and depth. Subsequently, the general R-D property estimation models of MVV and depth are analyzed. Finally, a rate-allocation scheme is designed to optimize the quantization parameter pair of MVV and depth in FVV coding. The simulation results demonstrate that the proposed depth-assisted FVV coding framework can improve the FVV coding efficiency. The region-based view synthesis distortion estimation approach and the general R-D model are able to precisely approximate the R-D property of synthesized virtual view video in the multiview video plus depth based FVV coding frameworks. The proposed rate-allocation scheme can optimize the overall FVV coding efficiency to achieve a high-quality reconstructed video at the desired viewpoint with a given rate constraint.

Reduced-complexity search for video coding geometry partitions using texture and depth data

In this paper, a texture-space geometry partitioning approach is proposed to reduce the computational complexity of searching for geometry partitions for video coding. Additionally, for systems that capture both video and depth data, an enhanced geometry partitioning approach using both texture and depth information is proposed to further improve the partitioning accuracy and reduce the search complexity. Compared with a full-search approach, the proposed geometry partition search approaches achieve about 94% reduction of the encoding time while retaining similar rate-distortion performance.

Online distribution and interaction of video data in social multimedia network

As the emerging multimedia on the latest social multimedia network, high dimensional video (HDV) data distribution is one of the most challenging problems in social multimedia applications. In the paper, we provide a state-of-the-art analysis on the streaming and interactive technologies for the two most popular representations of HDV, video plus depth map and multi-view video since both of them can enable the view-point interaction and behavior analysis in social media applications. Additionally, view-point switching techniques in HDV streaming with emphasis on interactive functionality are discussed, including SP-frame, Wyner-ziv frame and view switchable stream generation. Moreover, the overall performance of key tools focusing on HDV streaming is evaluated and potential improvements are discussed.

Complexity Reduction and Performance Improvement for Geometry Partitioning in Video Coding

Geometry partitioning for video coding involves establishing a partition line boundary within each block-shaped region and applying motion-compensated prediction to the two sub-regions created by the partition line. This paper presents techniques for enhancing the effectiveness and reducing the complexity of geometry partitioning schemes. A texture-difference-based approach is described to simplify the process of selecting the partition lines. Applying this approach together with a described skipping strategy for blocks with uniform texture can achieve a 94% reduction of encoding time while retaining a similar rate-distortion (R-D) performance to the full-search partitioning approach, when implemented for wedge-based geometry partitioning (WGP) in the context of H.264/MPEG-4 AVC JM 16.2. A bit rate improvement of approximately 6% is shown relative to not using geometry partitioning. For further R-D improvement, we describe a background-compensated prediction scheme to reduce the number of overhead bits used for motion vectors. Additionally, for systems in which high-quality depth maps are available, we incorporate depth map usage into the described approaches to generate a more accurate partitioning. Using these approaches with object-boundary-based geometry partitioning can achieve about 9% bit rate savings relative to using WGP, while keeping a similar computational complexity to the described complexity-reduced WGP.

Region Based Rate-Distortion Analysis for 3D Video Coding

In 3D video coding, to provide the high-quality interactive viewpoint video to audience, it is necessary to jointly optimize coding efficiency of color and depth images at a given bit-rates. In this paper, a region based distortion model is proposed to precisely estimate the error of synthesized virtual view. Furthermore, combined with the rate-distortion (R-D) model of color and depth images coding, an overall R-D model is built up for 3D video coding. Experimental results exhibit that the proposed approach can efficiently measure the R-D property of 3D video coding.

Complexity-reduced geometry partition search and high efficiency prediction for video coding

To reduce the complexity of searching for wedge-based geometry partitions in video coding, we propose a texture-difference based partition line selection approach with a skipping strategy. Applying this approach can reduce the encoding time by 90% while retaining the similar rate-distortion performance to that of exhaustive searching. We also propose a background-compensated prediction scheme to improve the rate-distortion performance of the geometry partitioning prediction by reducing its motion vector overhead. Incorporating our proposed approach into object-boundary-based geometry partitioning can achieve about 10% bit-rate savings relative to the full-search approach while keeping the complexity at about the same level as our proposed complexity-reduced wedge-based geometry partitioning.

3D spatial reconstruction and communication from vision field

Vision field describes the real world visual information by summarizing the seven-dimensional plenoptic function into three domains: view, light, time, from which a better understanding of previous 3D capture and reconstruction systems can be provided. In this paper, we first show how to reconstruct 3D spatial information from all the three attributes of the vision field, namely full-space vision field reconstruction. Then, based on Laplacian iterative geometry prediction, a 3D mesh coding algorithm with cascaded quantization is presented to facilitate the communication of the reconstructed 3D models from vision field. At last, experimental results of both the 3D spatial reconstruction and the 3D mesh coding are demonstrated.

Anterior cruciate ligament reconstruction model based on anatomical position locating

In the reconstruction surgery of Anterior Cruciate Ligament (ACL), how to locate the anatomical position is a very hard point to clinician occupational therapists. In this paper, we propose an Anatomical Position Locating (APL) approach based on Expectation Maximization (EM) algorithm. Firstly, the proposed intersection set operation algorithm is proposed to compute the attachment region between the injured ACL and femur or tibia. Then, the anatomical position is located by the 3D points cloud with the Gaussian spatial distribution. The last, the attachment spatial distribution and the barycenter, which are also viewed as the candidates of the anatomical position by a proposed EM algorithm, is partition. Experimental results verify our assumption and demonstrate that the located anatomical position has great serviceability.

Wyner-Ziv Coding of 3D Dynamic Meshes

Existing 3-D dynamic mesh compression methods directly explore temporal redundancy by predictive coding and the coded bitstreams are sensitive to transmission errors. In this paper, an efficient and error-resilient compression paradigm based on Wyner-Ziv coding (WZC) is proposed. We first apply an anisotropic wavelet transform (AWT) on each frame to explore their spatial redundancy. Then the wavelet coeffcients of every frame are compressed by a Wyner-Ziv codec which is composed of a nested scalar quantizer and a turbo codes based Slepian-Wolf codec. Benefiting from the inherent robustness of WZC, the proposed coding scheme can alleviates the problem of error-propagation associated with conventional predictive coding scheme. Furthermore, based on wavelet transform, our method can be extended to support progressive coding which is desirable for the streaming of 3D meshes. Experimental results show that our scheme is competitive with other compression methods in compression performance. Moreover, our method is more robust when transmission error occurs.