Chaminda T. E. R. Hewage

Quality Evaluation of Color Plus Depth Map-Based Stereoscopic Video

In the near future, many conventional video applications are likely to be replaced by immersive video to provide a sense of ldquobeing there.rdquo This transition is facilitated by the recent advancement of 3D capture, coding, transmission, and display technologies. Stereoscopic video is the simplest form of 3D video available in the literature. ldquoColor plus depth maprdquo based stereoscopic video has attracted significant attention, as it can reduce storage and bandwidth requirements for the transmission of stereoscopic content over communication channels. However, quality assessment of coded video sequences can currently only be performed reliably using expensive and inconvenient subjective tests. To enable researchers to optimize 3D video systems in a timely fashion, it is essential that reliable objective measures are found. This paper investigates the correlation between subjective and objective evaluation of color plus depth video. The investigation is conducted for different compression ratios, and different video sequences. Transmission over Internet protocol (IP) is also investigated. Subjective tests are performed to determine the image quality and depth perception of a range of differently coded video sequences, with packet loss rates ranging from 0% to 20%. The subjective results are used to determine more accurate objective quality assessment metrics for 3D color plus depth video.

Quality analysis for 3D video using 2D video quality models

The 3D (3-dimensional) video technologies are emerging to provide more immersive media content compared to conventional 2D (2-dimensional) video applications. More often 3D video quality is measured using rigorous and time-consuming subjective evaluation test campaigns. This is due to the fact that 3D video quality can be described as a combination of several perceptual attributes such as overall image quality, perceived depth, presence, naturalness and eye strain, etc. Hence this paper investigates the relationship between subjective quality measures and several objective quality measures like PSNR, SSIM, and VQM for 3D video content. The 3D video content captured using both stereo camera pair (two cameras for left and right views) and colour-and-depth special range cameras are considered in this study. The results show that, VQM quality measures of individual left and right views (rendered left and right views for colour-and-depth sequences) can be effectively used in predicting the overall image quality and statistical measures like PSNR and SSIM of left and right views illustrate good correlations with depth perception of 3D video.

Frame concealment algorithm for stereoscopic video using motion vector sharing

Stereoscopic video is one of the simplest forms of multi view video, which can be easily adapted for communication applications. Much current research is based on colour and depth map stereoscopic video, due to its reduced bandwidth requirements and backward compatibility. Existing immersive media research is more focused on application processing than aspects related to transfer of immersive content over communication channels. As video over packet networks is affected by missing frames, caused by packet loss, this paper proposes a frame concealment method for colour and depth map based stereoscopic video. The proposed method exploits the motion correlation of colour and depth map image sequences. The colour motion information is reused for prediction during depth map coding. The redundant motion information is then used to conceal transmission errors at the decoder. The experimental results show that the proposed frame concealment scheme performs better than applying error concealment for colour and depth map video separately in a range of packet error conditions.

Reduced-reference quality metric for 3D depth map transmission

Due to the technological advancement of 3D video technologies and the availability of other supportive services such as high bandwidth communication links, introduction of immersive video services to the mass market is imminent. However, in order to provide better service to demanding customers, the transmission system parameters need to be changed “on the fly”. Measured 3D video quality at the receiver side can be used as feedback information to fine tune the system. However, measuring 3D video quality using Full-reference quality metrics will not be feasible due to the need of original 3D vide sequence at the receiver side. Therefore, this paper proposed a Reduced-reference quality metric for 3D depth map transmission using the extracted edge information. This work is motivated by the fact that the edges and contours of the depth map can represent different depth levels and hence can be used in quality evaluations. Performance of the method is evaluated across a range of Packet Loss Rates (PLRs) and shows acceptable results compared to its counterpart Full-reference quality metric.

Quality of experience for 3D video streaming

New 3D video applications and services are emerging to fulfill increasing user demand. This effort is well supported by the increasing 3D video content including user generated content (e.g., through 3D capture/display enabled mobile phones), technological advancements (e.g., HD 3D video capture and processing methods), affordable 3D displays, and standardization activities. However, not much attention has been given to how these technologies, along the end-to-end chain from content capture to display, affect user perception and whether the overall experience of 3D video users is satisfactory or not. 3D video streaming also introduces artifacts on the reconstructed 3D video at the receiver end, leading to inferior quality and user experience. In this article we present and discuss in detail how artifacts introduced during 3D video streaming affect the end-user perception and how we could use realtime quality evaluation methodologies to overcome these effects. The observations presented can underpin the design of future QoE-aware 3D video streaming systems.

Flexible macroblock ordering for context-aware ultrasound video transmission over mobile WiMAX

The most recent network technologies are enabling a variety of new applications, thanks to the provision of increased bandwidth and better management of Quality of Service. Nevertheless, telemedical services involving multimedia data are still lagging behind, due to the concern of the end users, that is, clinicians and also patients, about the low quality provided. Indeed, emerging network technologies should be appropriately exploited by designing the transmission strategy focusing on quality provision for end users. Stemming from this principle, we propose here a context-aware transmission strategy for medical video transmission over WiMAX systems. Context, in terms of regions of interest (ROI) in a specific session, is taken into account for the identification of multiple regions of interest, and compression/transmission strategies are tailored to such context information. We present a methodology based on H.264 medical video compression and Flexible Macroblock Ordering (FMO) for ROI identification. Two different unequal error protection methodologies, providing higher protection to the most diagnostically relevant data, are presented.

Scalable multiple description video coding for stereoscopic 3D

Scalable multiple description video coding (MDC) provides adaptability to bandwidth variations and receiving device characteristics, and can improve error robustness in multimedia networks. In this paper, a scalable MDC scheme is proposed for stereoscopic 3D video. Scalable MDC has previously been applied to 2D video but not to 3D video. The proposed algorithm enhances the error resilience of the base layer of the scalable video coding (SVC) standard using even and odd frame based MDC. The performance of the algorithm is evaluated in error free and error prone environments. Simulation results show improved performance using the proposed scalable MDC at high error rates compared to the original SVC. To improve the rate distortion performance, down-sampling of the depth information is proposed for SVC and scalable MDC. The proposed method reduces the overall bit rates and consequently: (1) improves their rate distortion, particularly at lower bit rates in error free channels; and (2) improves their performance in error prone channels.

Image quality assessment based on edge preservation

Objective image/video quality metrics which accurately represent the subjective quality of processed images are of paramount importance for the design and assessment of an image compression and transmission system. In some scenarios, it is also important to evaluate the quality of the received image with minimal reference to the transmitted one. For instance, for closed-loop optimization of a transmission system, the image quality measure can be evaluated at the receiver and provided as feedback information to the system controller. The original image - prior to compression and transmission - is not usually available at the receiver side, and it is important to rely at the receiver side on an objective quality metric that does not need reference or needs minimal reference to the original image. The observation that the human eye is very sensitive to edge and contour information of an image underpins the proposal of our reduced reference (RR) quality metric, which compares edge information between the distorted and the original image. Results highlight that the metric correlates well with subjective observations, also in comparison with commonly used full-reference metrics and with a state-of-the-art reduced reference metric.

3D medical video transmission over 4G networks

Enabling 3D video in e-health applications results in the provision of more natural viewing conditions and improved diagnosis. Recent advances in wireless communications, network technologies and 3D video capture/display mechanisms enable the replacement of 2D video applications of current healthcare services with 3D video, to provide improved perception and diagnosis. Remote surgery and surgery training can benefit in particular from 3D video due to the added dimension of depth. This paper envisages a wireless health system which transmits 3D surgical video over a 3GPP LTE network. The quality of the received 3D video is measured both objectively and subjectively for several packet sizes.

Reduced-reference quality evaluation for compressed depth maps associated with colour plus depth 3D video

3D video quality assessment is a key task in transmission of immersive video, both for the assessment of end-to-end system performance and for quality evaluation “on the fly”. For example, measured 3D video quality at the receiver side can be used as feedback information for controlling transmission parameters. However, measuring video quality at the receiver side using Full-Reference quality metrics (e.g. PSNR) is less practical due to the need of original image sequence at the receiver. Hence, Reduced-Reference and No-Reference quality metrics are being developed to evaluate the quality. Since edges and contours of the depth map represent different depth levels, this paper proposes a Reduced-Reference quality metric for the depth maps associated with colour plus depth 3D video using edge detection. Edge/Side information (i.e. binary edge mask) generated using sobel filtering is utilized. Results show that the proposed metric can be used in place of a Full-Reference metric with an acceptable accuracy.