Brian W. Micallef

Cross-Layer Design for Optimized Region of Interest of Ultrasound Video Data Over Mobile WiMAX

The application of advanced error concealment techniques applied as a postprocess to conceal lost video information in error-prone channels, such as the wireless channel, demands additional processing at the receiver. This increases the delivery delay and needs more computational power. However, in general, only a small region within medical video is of interest to the physician and thus if only this area is considered, the number of computations can be curtailed. In this paper, we present a technique whereby the region of interest specified by the physician is used to delimit the area where the more complex concealment techniques are applied. A cross-layer design approach in mobile worldwide interoperability for microwave access wireless communication environment is adopted in this paper to provide an optimized quality of experience in the region that matters most to the mobile physician while relaxing the requirements in the background, ensuring real-time delivery. Results show that a diagnostically acceptable peak signal-to-noise-ratio of about 36 dB can still be achieved within reasonable decoding time.

An analysis on the effect of transmission errors in real-time H.264-MVC Bit-streams

This paper studies the quality of transmitted multi-view video when the corrupted packets are not discarded by the underlying protocols of the decoder. It assumes a wireless channel where the errors can be significant and implements solutions within the current H.264-MVC to reduce their impact on the video quality perceived by the user. The results show that transmission errors drastically reduce the quality of the reconstructed 3D video and confirm that a new type of error propagation between views exists. Furthermore, employing the Context Adaptive Variable Length Coding (CAVLC) entropy encoder, coding and transmitting the video streams in smaller packets, and having a small cyclic-Intra coded period, all improve the error resilience of the system.

Exploiting depth information for fast multi-view video coding

Multi-view video coding exploits inter-view redundancies to compress the video streams and their associated depth information. These techniques utilize disparity estimation techniques to obtain disparity vectors (DVs) across different views. However, these methods contribute to the majority of the computational power needed for multi-view video encoding. This paper proposes a solution for fast disparity estimation based on multi-view geometry and depth information. A DV predictor is first calculated followed by an iterative or a fast search estimation process which finds the optimal DV in the search area dictated by the predictor. Simulation results demonstrate that this predictor is reliable enough to determine the area of the optimal DVs to allow a smaller search range. Furthermore, results show that the proposed approach achieves a speedup of 2.5 while still preserving the original rate-distortion performance.

Error concealment techniques for multi-view video

The H.264/AVC multi-view extension provides for high compression ratios of multi-view sequences. The coding scheme used exploits spatial, temporal and inter-view dependability for this scope. However, in the event of transmission errors, this leads to the propagation of the distorted macroblocks, degrading the quality of the video perceived by the user. In this paper we introduce error resilient coding and error concealment techniques in Multi-view Video Coding to reduce this effect. The results obtained demonstrate that better multiview video reconstruction is obtained when Intra-coded frames are spatially concealed while Inter-coded frames are concealed using motion compensation techniques, within the multi-view prediction structure. Furthermore, additional gain in quality can be achieved when Anchor frames are concealed using a combination of spatial and motion compensation techniques.

Exploiting depth information for fast motion and disparity estimation in Multi-view Video Coding

Multi-view Video Coding (MVC) employs both motion and disparity estimation within the encoding process. These provide a significant increase in coding efficiency at the expense of a substantial increase in computational requirements. This paper presents a fast motion and disparity estimation technique that utilizes the multi-view geometry together with the depth information and the corresponding encoded motion vectors from the reference view, to produce more reliable motion and disparity vector predictors for the current view. This allows for a smaller search area which reduces the computational cost of the multi-view encoding system. Experimental results confirm that the proposed techniques can provide a speed-up gain of up to 4.2 times, with a negligible loss in the rate-distortion performance for both the color and the depth MVC.

Exploiting depth information for efficient Multi-View Video Coding

The Multi-view Video Coding (MVC) technique provides significantly better coding efficiency compared to simulcast transmission of different camera view-points. This is done by exploiting both motion and disparity compensation techniques to compress the different view-point videos. This paper proposes a more efficient MVC technique that transmits the optimally selected compensation replacements with respect to more accurate compensation vector predictors obtained using the multi-view geometry and the depth information. In this technique, the SKIP mode is also modified to adapt its compensation direction from both the temporal and the view-point reference frames. Experimental results show that the proposed MVC technique gives an average gain in video quality of about 0.5dB, for the interview predicted view-points.

Fast inter-mode decision in multi-view video plus depth coding

Motion and disparity estimations are employed in Multi-view Video Coding (MVC) to remove redundancies present between temporal and different viewpoint frames, respectively, in both the color and the depth multi-view videos. These constitute the major computational expensive tasks of the video encoder, as iterative search for the optimal mode and its appropriate compensation vectors is employed to reduce the Rate-Distortion Optimization (RDO) cost function. This paper proposes a solution to limit the number of modes that are tested for RDO to encode the inter-view predicted views. The decision is based on the encoded information obtained from the corresponding Macroblock in the Base view, identified accurately by using the multi-view geometry together with the depth data. Results show that this geometric technique manages to reduce about 70% of the estimations computational time and can also be used with fast geometric estimations to reduce up to 95% of the original encoding time. These gains are obtained with little degradation on the multi-view video quality for both color and depth MVC.

Fast multi-view video plus depth coding with hierarchical bi-prediction

The Multi-view Video Coding (MVC) standard was developed for efficient encoding of multi-view videos. Part of it requires the calculation of both disparity and motion estimations using a bi-prediction structure. These estimations involve an exhaustive search for the optimal compensation vectors from multiple forward and backward reference frames which, while being very efficient in terms of compression, results in high computational costs. This paper proposes a solution that utilizes the multi-view geometry along with the available depth data, to calculate more accurate predictors for both motion and disparity estimations, and for both directions of the prediction structure. Simulation results demonstrate that this technique is reliable enough to allow a substantial reduction in the search areas in all the reference frames. This in turn results in a significant speed-up gain of 3.2 times with a negligible influence on the coding efficiency, while encoding both the color and the depth MVVs.

Reducing 3D video coding complexity through more efficient disparity estimation

3D video coding for transmission exploits the Disparity Estimation (DE) to remove the inter-view redundancies present within both the texture and the depth map multi-view videos. Good estimation accuracy can be achieved by partitioning the macro-block into smaller subblocks partitions. However, the DE process must be performed on each individual sub-block to determine the optimal mode and their disparity vectors, in terms of rate-distortion efficiency. This vector estimation process is heavy on computational resources, thus, the coding computational cost becomes proportional to the number of search points and the inter-view modes tested during the rate-distortion optimization. In this paper, a solution that exploits the available depth map data, together with the multi-view geometry, is proposed to identify a better DE search area; such that it allows a reduction in its search points. It also exploits the number of different depth levels present within the current macro-block to determine which modes can be used for DE to further reduce its computations. Simulation results demonstrate that this can save up to 95% of the encoding time, with little influence on the coding efficiency of the texture and the depth map multi-view video coding. This makes 3D video coding more practical for any consumer devices, which tend to have limited computational power.

Error Concealment Techniques in Multi-view Video Applications

The demand for immersive multimedia experiences is driving researchers and industry to develop solutions that capture, deliver, and display 3D video in an efficient way. The key to the success of this technology is the removal of redundant data through effective video coding and its transmission. One solution that can meet this coding requirement is Multi-view Video Coding (MVC). Besides the removal of spatial and temporal redundancies already used in legacy single-view video transmission, this scheme also considers the redundancies present in between views. Such redundancies exist since the different spatially separated capturing devices are shooting the same view. Removal of the latter significantly enhances the coding efficiency compared to encoding separately each view. This huge reduction in data to be transmitted comes at a price. Practical channels are not error-free, and thus, because of the dependencies generated during encoding, errors in the channel will result in artifacts in the video streams that will propagate in space, time, and views until the dependencies are interrupted. This demands solutions that allow the reconstruction of missing data at the receiver to guarantee a good quality of experience (QoE) which is paramount to the success of 3D video applications. To reduce the occurrence of erroneous data, the information transmitted needs to be protected using error control strategies. These will typically introduce some redundancies that infer knowledge on missing information such that it can be recovered. However, not all the erroneous data can be recovered requesting algorithms that can limit and estimate the missing information. The latter techniques are known as error resilience coding and error concealment methods, respectively, in which the error propagation is restricted and the missing content is estimated from the available information. This chapter will examine the effects of errors experienced during transmission of multi-view video content. This is followed by the application of error resilient and error concealment techniques that introduce enhancements to the quality of the received video content.