H. Kodikara Arachchi

A new mode selection technique for coding Depth maps of 3D video

Compression of Depth maps that are used in 3D video systems based on Depth Image Based Rendering (DIBR) poses a new challenge in video coding, since it is not a sequence of images for final viewing by end users rather an aid for rendering. Therefore, compressing depth maps using existing video coding techniques yields unacceptable distortions while rendering virtual views. In this paper we propose a novel mode selection method for offline compression of depth maps by selecting modes collaboratively considering an entire row of macroblocks together. For selecting these modes while encoding, we propose a novel distortion criteria that incorporates rendering distortions instead of distortion of depth map itself. A genetic algorithm based optimization technique is used for the mode selection. The simulation results suggest that the proposed technique can improve the PSNR up to 1.6dB in the rendered stereoscopic views in comparison to the block wise mode selection method based on Lagrange Optimization and the distortion of the depth map itself.

Adaptation-aware encryption of scalable H.264/AVC video for content security

Data encryption is one of the key information security technologies used for safeguarding multimedia content from unauthorised access and manipulation in end-to-end delivery and access chains. This technology, combined with the appropriate cryptographic methods, effectively prevents the content against malicious attacks, so as to protect its authenticity as well as integrity. While encryption-based security is ensuring the authorised consumption of the multimedia content, content adaptation technologies have the primary goal of providing means for wider dissemination of the content across diverse networks, devices and users, and thus enriching user satisfaction and experience of the delivered content within a given set of usage environment constraints. Traditionally, protected contents can only be adapted at trusted adaptation engines residing between the source and end-users, since they have to be fully decrypted before performing the necessary adaptation operations. The drawback of such a process is that it significantly limits the availability and flexibility of adaptation engines applicable for adapting protected contents on the fly. Thus, this paper proposes a novel scalable H.264/advanced video coding (AVC)-compatible video encryption technique, which is also transparent to adaptation engines in an end-to-end video delivery scenario. The proposed technology relies on keeping syntax elements required for performing the adaptation operations clear (i.e., not encrypted). The effectiveness of the proposed technique has been successfully verified in scenarios, where both conventional joint scalable video model (JSVM) bit stream extracting and random packet dropping mechanisms are used.

Impact of depth map spatial resolution on 3D video quality and depth perception

Despite the burgeoning advances in 3D video technologies, 3D video adaptation is still in its infancy. 3D video has a multidimensional nature. To support the growth of 3D adaptation technologies, key factors that characterise this nature should be thoroughly studied. Spatial resolution is one of the important factors characterising this nature. Thus, particularly the effect of different spatial resolutions of depth maps on video quality and depth perception should be investigated. Similarly, the influence of the content related factors on the perception of different spatial resolutions of the depth maps should also be investigated. Therefore, evaluation studies are conducted using spatially scaled depth maps encoded at different bit rates and their original colour texture counterparts. The depth maps utilised in the evaluations present different spatial and motion complexity characteristics. The results demonstrate that when the spatial resolutions of the depth maps increase, both the video quality and depth perception improve. The amount of improvement is highly correlated with the structural and motion complexity characteristics of the depth maps.

Psycho-physical limits of interocular blur suppression and its application to asymmetric stereoscopic video delivery

It is well known that when the two eyes are provided with two views of different resolutions the overall perception is dominated by the high resolution view. This property, known as binocular suppression, is effectively used to reduce the bit rate required for stereoscopic video delivery, where one view of the stereo pair is encoded at a much lower quality than the other. There have been significant amount of effort in the recent past to measure the just noticeable level of asymmetry between the two views, where asymmetry is achieved by encoding views at two quantization levels. However, encoding artifacts introduce both blurring and blocking artifacts in to the stereo views, which are perceived differently by the human visual system. Therefore, in this paper, we design a set of psycho-physical experiments to measure the just noticeable level of asymmetric blur at various spatial frequencies, luminance contrasts and orientations. The subjective results suggest that humans could tolerate a significant amount of asymmetry introduced by blur, and the level of tolerance is independent of the spatial frequency or luminance contrast. Furthermore, the results of this paper illustrate that when asymmetry is introduced by unequal quantization, the just noticeable level of asymmetry is driven by the blocking artifacts. In general, stereoscopic asymmetry introduced by way of asymmetric blurring is preferred over asymmetric compression. It is expected that the subjective results of this paper will have important use cases in objective measurement of stereoscopic video quality and asymmetric compression and processing of stereoscopic video.

Using context to assist the adaptation of protected multimedia content in virtual collaboration applications

This paper proposes a framework for a virtual classroom application based on a virtual collaboration system (VCS), which is being developed under the VISNET II network of excellence (NoE), and discusses adaptation technologies that enable seamless access to classroom sessions while intellectual property and digital rights are managed. The proposed virtual classroom framework enables academic institutions to conduct their collaborative lecture series, to which registered students will be able to attend remotely and interactively over the Internet. Furthermore, the general public may also follow the classroom sessions under certain restrictions imposed by the participating institutions. In order to facilitate seamless access to a heterogeneous audience that is composed of users with various preferences and privileges accessing the classroom sessions over different network infrastructures and using terminal devices with diverse capabilities, context-aware content adaptation is required to meet constraints imposed by the usage context and enhance the quality of the user experience. Thus, this paper describes the concepts and functionalities of a context-aware content adaptation platform that suits the requirements of such multimedia application scenarios. This platform is able to consume low-level contextual information to infer higher-level contexts, and thus decide the need for and type of adaptation operations to be performed upon the content. In this way, it is aimed to meet usage constraints while also satisfying restrictions imposed by the digital rights management (DRM) to govern the use of protected content.

Seamless video access for mobile devices by content-aware utility-based adaptation

Today’s Internet multimedia services are characterized by heterogeneous networks, a wide range of terminals, diverse user preferences, and varying natural environment conditions. Heterogeneity of terminals, networks, and user preferences impose nontrivial challenges to the Internet multimedia services for providing seamless multimedia access particularly for mobile devices (e.g., laptops, tablet PCs, PDAs, mobile phones, etc.). Thus, it is essential that advanced multimedia technologies are developed to deal with these challenges. One of these technologies is video adaptation, which has gained significant importance with its main objective of enabling seamless access to video contents available over the Internet. Adaptation decision taking, which can be considered as the “brain” of video adaptation, assists video adaptation to achieve this objective. Scalable Video Coding (SVC) offers flexibility for video adaptation through providing a comprehensive set of scalability parameters (i.e., temporal, spatial, and quality) for producing scalable video streams. Deciding the best combination of scalability parameters to adapt a scalable video stream while satisfying a set of constraints (e.g., device specifics, network bandwidth, etc.) poses challenges for the existing adaptation services to enable seamless video access. To ease such challenges, an adaptation decision taking technique employing a utility-based approach to decide on the most adequate scalability parameters for adaptation operations is developed. A Utility Function (UF), which models the relationships among the scalability parameters and weights specifying the relative importance of these parameters considering video content characteristics (i.e., motion activity and structural feature), is proposed to assist the developed technique. In order to perform the developed adaptation decision taking technique, a video adaptation framework is also proposed in this paper. The adaptation experiments performed using the proposed framework prove the effectiveness of the framework to provide an important step towards enabling seamless video access for mobile devices to enhance viewing experience of users.

Robust video communication using Random Linear Network Coding with Pre-Coding and Interleaving

With the increasing popularity of online video streaming and broadcasting applications, video transmission has become the most bandwidth consumed application in the Internet. Unless resource utilisation is further optimised, existing networks would not be able to accommodate the forecasted demand for high volume video traffic. Random Linear Network Coding (RLNC) is a promising approach to increase the network throughput and robustness in order to facilitate high volume traffic. However, the performance gain obtained using RLNC decreases when the network packet loss rate increases. In response, this paper proposes to incorporate RLNC with a combined interleaving and a coding technique to improve the effectiveness of generating redundant packet combinations over classical RLNC to enhance the robustness against packet losses. Simulation results illustrate that the proposed method improves the probability of recovering lost data over the classical RLNC and the video transmission simulation results exhibit up to 5dB increase in PSNR and up to 14% increase in VQM utilising the same network resources at the same network conditions.

Sensitivity of the HVS for Binocular disparity Cue in 3D Displays under Different Ambient Illumination Conditions

In this paper, the sensitivity of the HVS towards binocular disparity depth cue, which is the most important cue in modern stereoscopic 3D displays, is investigated under different ambient illumination conditions. The experimental results indicate that as ambient illumination increases the sensitivity of the HVS for depth details increases or the Just Noticeable Difference in Depth (JNDD) decreases. It is expected that the investigation results will have important use cases in designing 3D display setups and 3D content production.

Extended VQM model for predicting 3D video quality considering ambient illumination context

To speed-up the proliferation of advanced 3-Dimensional (3D) technologies into the consumer market, the influence of these technologies on the perception of 3D video should be determined. Currently, this can only be achieved using either subjective assessment techniques or 2D objective quality evaluation models. Even though the subjective assessment techniques are better than the objective models from the accuracy point of view, they are time consuming and costly. Thus, 2D objective quality evaluation models correlating with Human Visual System (HVS) should be used to predict the 3D video quality perception of users in a reliable way with less effort. Video Quality Metric (VQM), which is a standardized 2D objective quality measurement model due to its well correlation with HVS, is used to predict 3D video quality perception of users reliably. However, ambient illumination context of the viewing environment, which has an effect on 3D video quality perception, is not considered in the quality assessments by VQM. Content adaptation is one of the key applications that need to use the perceived 3D quality assessments under different ambient illumination conditions at regular basis for ensuring improved video experience of users. Therefore, the standardized VQM model is extended using ambient illumination context and content related contexts (i.e., motion, structural feature, and luminance contrast) to predict 3D video quality measurement under a particular ambient illumination condition. The results prove that the extended VQM model can be efficiently utilized to predict the video quality perception of 3D video under a particular ambient illumination condition.

Towards minimising the coefficient vector overhead in random linear Network Coding

Network Coding is a promising approach to increase network throughput and robustness to facilitate high volume traffic. Performing network coding in dynamic network structures requires transmitting coding coefficients for information sinks to decode network coded packets. Compared to the packet sizes used in practical networks, the size of coefficient vectors can be significant. This paper exploits the properties of small and medium sized networks and proposes a novel approach to minimise the coefficient vector size of network coded packets. Simulation results exhibit better compression of coefficient vectors over existing algorithms for small and medium sized networks.