Bruno Macchiavello

A simple reversed-complexity Wyner-Ziv video coding mode based on a spatial reduction framework

A spatial-resolution reduction based framework for incorporation of a Wyner-Ziv frame coding mode in existing video codecs is presented, to enable a mode of operation with low encoding complexity. The core Wyner-Ziv frame coder works on the Laplacian residual of a lower-resolution frame encoded by a regular codec at reduced resolution. The quantized transform coefficients of the residual frame are mapped to cosets to reduce the bit-rate. A detailed rate-distortion analysis and procedure for obtaining the optimal parameters based on a realistic statistical model for the transform coefficients and the side information is also presented. The decoder iteratively conducts motion-based side-information generation and coset decoding, to gradually refine the estimate of the frame. Preliminary results are presented for application to the H.263+ video codec.

Loss-Resilient Coding of Texture and Depth for Free-Viewpoint Video Conferencing

Free-viewpoint video conferencing allows a participant to observe the remote 3D scene from any freely chosen viewpoint. An intermediate virtual viewpoint image is typically synthesized using two pairs of transmitted texture and depth maps from two neighboring captured viewpoints via depth-image-based rendering (DIBR). To maintain high quality of synthesized images, it is imperative to contain the adverse effects of network packet losses that may arise during texture and depth video transmission. Towards this goal, we develop an integrated approach that exploits the representation redundancy inherent in the multiple streamed videos-a voxel in the 3D scene visible to two captured views is sampled and coded twice in the two views. In particular, at the receiver we first develop an error concealment strategy that adaptively blends corresponding pixels in the two captured views during DIBR, so that pixels from the more reliable transmitted view are weighted more heavily. We then couple it with a sender-side optimization of reference picture selection (RPS) during real-time video coding, so that blocks containing pixel samples of voxels that are visible in both views are more error-resiliently coded in one view only, given adaptive blending will mitigate errors in the other view. Further, synthesized view distortion sensitivities to texture versus depth errors are analyzed, so that relative importance of texture and depth code blocks can be computed for system-wide RPS optimization. Finally, quantization parameter (QP) is adaptively selected per frame, optimally trading off source distortion due to compression with channel distortion due to potential packet losses. Experimental results show that the proposed scheme can outperform previous work by up to 2.9 dB at 5% packet loss rate.

Side-information generation for temporally and spatially scalable Wyner-Ziv codecs

The distributed video coding paradigm enables video codecs to operate with reversed complexity, in which the complexity is shifted from the encoder toward the decoder. Its performance is heavily dependent on the quality of the side information generated by motio estimation at the decoder. We compare the rate-distortion performance of different side-information estimators, for both temporally and spatially scalable Wyner-Ziv codecs. For the temporally scalable codec we compared an established method with a new algorithm that uses a linear-motion model to produce side-information. As a continuation of previous works, in this paper, we propose to use a super-resolution method to upsample the nonkey frame, for the spatial scalable codec, using the key frames as reference. We verify the performance of the spatial scalable WZ coding using the state-of-the-art video coding standard H.264/AVC.

Iterative Side-Information Generation in a Mixed Resolution Wyner-Ziv Framework

We propose a mixed resolution framework based on full resolution key frames and spatial-reduction-based Wyner-Ziv coding of intermediate nonreference frames. Improved rate-distortion performance is achieved by enabling better side-information generation at the decoder side and better rate-allocation at the encoder side. The framework enables reduced encoding complexity by low resolution encoding of the nonreference frames, followed by Wyner-Ziv coding of the Laplacian residue. The quantized transform coefficients of the residual frame are mapped to cosets without the use of a feedback channel. A study to select optimal coding parameters in the creation of the memoryless cosets is made. Furthermore, a correlation estimation mechanism that guides the parameter choice process is proposed. The decoder first decodes the low resolution base layer and then generates a super-resolved side-information frame at full resolution using past and future key frames. Coset decoding is carried using side-information to obtain a higher quality version of the decoded frame. Implementation results are presented for the H.264/AVC codec.

An H.264/AVC to HEVC video transcoder based on mode mapping

The emerging video coding standard, HEVC, was developed to replace the current standard, H.264/AVC. However, in order to promote inter-operability with existing systems using the H.264/AVC, transcoding from H.264/AVC to the HEVC codec is highly needed. This paper presents a transcoding solution that uses machine learning techniques in order to map H.264/AVC macroblocks into HEVC coding units (CUs). Two alternatives to build the machine learning model are evaluated. The first uses a static training, where the model is built offline and used to transcode any video sequence. The other uses a dynamic training, with two well-defined stages: a training stage and a transcoding stage. In the training stage, full re-encoding is performed while the H.264/AVC and the HEVC information are gathered. This information is then used to build a model, which is used in the transcoding stage to classify the HEVC CU partitioning. Both solutions are tested with well-known video sequences and evaluated in terms of rate-distortion (RD) and complexity. The proposed method is on average 2.26 times faster than the trivial transcoder using fast motion estimation, while yielding a RD loss of only 3.6% in terms of bitrate.

CQR codes: Colored quick-response codes

This paper proposes a new way to store/transmit information using a Colored QR Code structure. Instead of using only black and white modules, the proposed code is designed to employ 5 different RGB colors (red, green, blue, black and white), which enables twice as much storage capacity compared to traditional binary QR Codes. Reed-Solomon error-correcting code with a theoretical correction capability of 38.41% is also applied. In our experiments, each Colored QR Code in the test set is printed, scanned using a 3.2 megapixel digital camera and decoded. We show that the proposed scheme can consistently decode 1024 bits of information stored on a 1.3 cm × 1.3 cm printed area.

Semi-automatic algorithm for construction of the left ventricular area variation curve over a complete cardiac cycle

BackgroundTwo-dimensional echocardiography (2D-echo) allows the evaluation of cardiac structures and their movements. A wide range of clinical diagnoses are based on the performance of the left ventricle. The evaluation of myocardial function is typically performed by manual segmentation of the ventricular cavity in a series of dynamic images. This process is laborious and operator dependent. The automatic segmentation of the left ventricle in 4-chamber long-axis images during diastole is troublesome, because of the opening of the mitral valve.MethodsThis work presents a method for segmentation of the left ventricle in dynamic 2D-echo 4-chamber long-axis images over the complete cardiac cycle. The proposed algorithm is based on classic image processing techniques, including time-averaging and wavelet-based denoising, edge enhancement filtering, morphological operations, homotopy modification, and watershed segmentation. The proposed method is semi-automatic, requiring a single user intervention for identification of the position of the mitral valve in the first temporal frame of the video sequence. Image segmentation is performed on a set of dynamic 2D-echo images collected from an examination covering two consecutive cardiac cycles.ResultsThe proposed method is demonstrated and evaluated on twelve healthy volunteers. The results are quantitatively evaluated using four different metrics, in a comparison with contours manually segmented by a specialist, and with four alternative methods from the literature. The methods intra- and inter-operator variabilities are also evaluated.ConclusionsThe proposed method allows the automatic construction of the area variation curve of the left ventricle corresponding to a complete cardiac cycle. This may potentially be used for the identification of several clinical parameters, including the area variation fraction. This parameter could potentially be used for evaluating the global systolic function of the left ventricle.

Reference frame selection for loss-resilient texture & depth map coding in multiview video conferencing

In a free-viewpoint video conferencing system, the viewer can choose any desired viewpoint of the 3D scene for observation. Rendering of images for arbitrarily chosen viewpoint can be achieved through depth-image-based rendering (DIBR), which typically employs “texture-plus-depth” video format for 3D data exchange. Robust and timely transmission of multiple texture and depth maps over bandwidth-constrained and loss-prone networks is a challenging problem. In this paper, we optimize transmission of multiview video in texture-plus-depth format over a lossy channel for free viewpoint synthesis at decoder. In particular, we construct a recursive model to estimate the distortion in synthesized view due to errors in both texture and depth maps, and formulate a rate-distortion optimization problem to select reference pictures for macroblock encoding in H.264 in a computation-efficient way, in order to provide unequal protection to different macroblocks. Results show that the proposed scheme can outperform random insertion of intra refresh blocks by up to 0.73 dB at 5% loss.

Motion-Based Side-Information Generation for a Scalable Wyner-Ziv Video Coder

A motion-based side-information generation scheme with semi super-resolution for a scalable Wyner-Ziv coder framework is introduced. It is known that the performance of any Wyner-Ziv coder is heavily dependent on the efficiency of the side-information generation. We propose an iterative block based scheme to generate a semi super-resolution frame using the past and future reference frames which should be coded at full-resolution. To enable this side-information generation the framework should allow for low encoding complexity, reducing the spatial resolution only in the non-reference frames. The enhancement layer is produced using a residual frame of the reduced resolution encoded frame. The decoder first decodes the low resolution base layer and iteratively generates the side-information, along with channel decoding, to obtain a higher quality version of the decoded frame. Results of the implementation of the framework using the motion-based side-information in the H.263+ and H.264 standards are presented.

Fast H.264/AVC to HEVC transcoding based on machine learning

Since the HEVC codec has become an ITU-T and ISO/IEC standard, efficient transcoding from previous standards, such as the H.264/AVC, to HEVC is highly needed. In this paper, we build on our previous work with the goal to develop a faster transcoder from H.264/AVC to HEVC. The transcoder is built around an established two-stage transcoding. In the first stage, called the training stage, full re-encoding is performed while the H.264/AVC and the HEVC information are gathered. This information is then used to build a CU classification model that is used in the second stage (called the transcoding stage). The solution is tested with well-known video sequences and evaluated in terms of rate-distortion and complexity. The proposed method is 3.4 times faster, on average, than the trivial transcoder, and 1.65 times faster than a previous transcoding solution.