Sylvain Marcelino

Lost block reconstruction in depth maps using color image contours

This paper presents a method to recover lost blocks in depth maps affected by data loss in 3D image/video communications over error prone networks. The proposed method relies on the color image for accurate reconstruction of the lost contour segments in the corresponding depth map areas. Such reconstructed depth map contours are then used as boundaries at different depth planes to recover the missing depth values through weighted interpolation. The method performance is evaluated by the objective quality (PSNR) of the synthesised views. Images decoded with the reconstructed depth maps are compared with those of the reference method. The proposed method exhibits PSNR gains up to 1.49dB higher than the reference one, and a better performance is consistently achieved for different 3D image content.

Error recovery of image-based depth maps using Bézier curve fitting

This paper proposes a method to recover lost regions in image-based depth maps used in video plus depth 3D format. This method performs depth maps reconstruction taking into account depth contours within the lost regions. This is achieved by extracting the contours and recovering their lost segments based on Bézier curve fitting, followed by spatial interpolation. The proposed method maintains contour smoothness and uses them as the boundary limits of homogeneous depth regions, which are then filled through weighted pixel interpolation. The experimental results show that the proposed method yields better synthesized images than classic spatial concealment methods, uniquely based on pixel interpolation techniques. The method presented in this paper is able to outperform the reference method, in terms of PSNR by up to 1.91dB. The subjective quality is also shown as being significantly better.

Spatial error concealment for intra-coded depth maps in multiview video-plus-depth

Transmission errors or packet loss in depth maps have great impact on the decoding quality and view synthesis of 3D and multiview video. Thus efficient methods to recover corrupted depth data are critical functions for accurate view rendering. This paper proposes an error concealment method for intra-coded depth maps, based on spatial intra and inter-view methods, which exploit neighbouring data of depth and colour images received error-free. A novel three-stage processing algorithm is devised to reconstruct sharp depth transitions (i.e. lost depth contours), using a disparity map and geometric interpolation based on parametric Bézier curves. The simulation results obtained from different views of various MVD sequences, for different packetisation modes and a wide range of packet loss rates (PLR), show that the proposed method consistently leads to quality improvement of synthesised images in comparison with reference methods.

Efficient MV prediction for zonal search in video transcoding

This paper proposes a method to efficiently find motion vector predictions for zonal search motion re-estimation in fast video transcoders. The motion information extracted from the incoming video stream is processed to generate accurate motion vector predictions for transcoding with reduced complexity. Our results demonstrate that motion vector predictions computed by the proposed method outperform those generated by the highly efficient EPZS (Enhanced Predictive Zonal Search) algorithm in H.264/AVC transcoders. The computational complexity is reduced up to 59.6% at negligible cost in R-D performance. The proposed method can be useful in multimedia systems and applications using any type of transcoder, such as transrating and/or spatial resolution downsizing.

Efficient depth error concealment for 3D video over error-prone channels

This paper addresses the problem of multivew video-plus-depth (MVD) decoding with corrupted depth maps due to transmission errors. A method for spatial error concealment of depth maps in MVD is proposed to efficiently recover lost blocks. The proposed method relies on the colour image and geometric curve fitting for accurate reconstruction of the lost contour segments in the corresponding depth map areas. Such reconstructed depth map contours are then used as boundaries at different depth planes to recover the missing depth values through weighted interpolation. The method performance is evaluated by the objective quality (PSNR) of the synthesised views. Images decoded with the reconstructed depth maps are compared with those of a reference method based on bidirectional interpolation. The proposed method exhibits PSNR gains up to 1.66dB higher than the reference one and better performance is consistently achieved for different visual content.

Depth map concealment using interview warping vectors from geometric transforms

This paper deals with reconstruction of corrupted depth maps received by multiview video-plus-depth (MVD) decoders from error prone channels. An interview-based method is proposed using warping vectors obtained through a block matching approach with geometric transforms (BMGT) between two colour views. It is shown that BMGT is able to find efficient warping vectors for reconstruction of lost regions in the depth maps associated with the colour views. The proposed concealment method uses an additional contour reconstruction technique, applied to arbitrary shapes within the lost regions, which is used for weighted interpolation. In comparison to a reference method based on simple weighted interpolation, the proposed method is able to achieve PSNR gains in synthesised views up to 5.61 dB, at data loss ratios up to 40%.

Robust decoding of MDC depth maps for enhanced 3D video over hybrid broadcasting networks

This paper addresses the problem of robust decoding of 3D video over hybrid broadcast networks, using Multiple Description Coding (MDC). Despite the fact that MDC allows decoding of any description even when the others are lost in the multipath network, the resulting coarsely decoded depth maps produce unacceptable distortion in synthesised views. This is particularly harmful in the case of intra-coded depth slices because of subsequent error propagation. To achieve improved quality in the virtual views synthesised from MDC depth maps received with lost descriptions, a method is proposed based on the depth geometric information extracted from the received descriptions and also from texture motion. Accurate recovery of either lost or coarsely decoded depth slices from one single description, is achieved by jointly using motion information, depth map edges and spatially neighbouring depth values. Comparing with MDC decoding without enhanced reconstruction of lost descriptions, the proposed method reaches quality gains up to 2.29dB for loss rates of 10%.

Digital sound processing using arduino and MATLAB

Over the last decade, impelled by the huge open source software community support, the low cost Arduino platform presents itself as an alternative for digital sound processing. Although Arduino is generally used for small applications for the artistic and maker community, its built-in Analog to digital converter can be used for sound capturing, processing and reproduction. Equipped with a powerful AVR 8 bit RISC microcontroller, the Arduino, can achieve up to 200kHz with a 10 bit resolution according to the Atmel ATmega328P datasheet that is the AVR core that we are going to focus on this article. Realizing the hardware potential, software suppliers like Matworks or National instruments, have included the Arduino packages on the software accessories of MATLAB and LABView. This work presents some of the sound capabilities and specific limitations of the Arduino platform, enfacing its connection and installation with MATLAB software. A series of examples of the Arduino interface with MATLAB are detail and shown in order to facilitate users initiation of MATLAB and Arduino Digital Sound Processing enhancing education fostering.

Quality evaluation of depth map error concealment using a perceptually-aware objective metric

This paper presents a quality evaluation study on the performance of error concealment methods for depth maps used in multiview video-plus-depth (MVD). The research deals with the problem of decoding corrupted depth maps received from error-prone networks, where the quality of the reconstructed depth data is not always directly related to the quality of the virtual views synthesised by those maps. Even after error concealment such distortions are not particularly perceived as other known types, such as coding distortion. Thus, traditional quality metrics are not adequate to capture all the relevant features. In this work, the performance of two error concealment methods for depth maps is evaluated using a perceptually-aware objective metric. This metric is validated through subjective assessment of virtual views synthesised with concealed depth maps. Each subjective test is performed by comparing the relative quality between between two synthesised images using different error concealment methods. The perceptual impact of reconstruction in corrupted depth of MVD is evaluated under various loss rates, using several colour images and depth maps encoded at multiple quantisation steps. The achieved results reveal that the proposed objective quality metric is mostly inline with user preferences, in respect to the relative performance of each error concealment method.

Error Concealment Methods for Multiview Video and Depth

The different media representation formats and coding techniques currently used to deliver 3D visual information across diverse networks require specific approaches and methods to minimise the perceptual impact of data loss, that may occur along the communications path. This chapter addresses this type of problem by presenting recent advances in error concealment methods, expanding conventional techniques used for 2D video to multiview (MVC) and multiview video-plus-depth (MVD) coded formats. The methods described in the chapter exploit the specific characteristics of multiview formats to achieve highly efficient error concealment performance and, consequently, to improve the perceptual quality delivered to end users, in the presence of transmission losses. In the case of MVC, besides spatial and inter-frame, inter-view correlations are also exploited, while in MVD, the most efficient methods use both the texture (view) and depth information to improve the error concealment performance. The most relevant contributions in this field are described in detail, where the performance of these advanced solutions is discussed along with comparisons between different methods and benchmarking.