Hassan Mansour

Optimizing a Tone Curve for Backward-Compatible High Dynamic Range Image and Video Compression

For backward compatible high dynamic range (HDR) video compression, the HDR sequence is reconstructed by inverse tone-mapping a compressed low dynamic range (LDR) version of the original HDR content. In this paper, we show that the appropriate choice of a tone-mapping operator (TMO) can significantly improve the reconstructed HDR quality. We develop a statistical model that approximates the distortion resulting from the combined processes of tone-mapping and compression. Using this model, we formulate a numerical optimization problem to find the tone-curve that minimizes the expected mean square error (MSE) in the reconstructed HDR sequence. We also develop a simplified model that reduces the computational complexity of the optimization problem to a closed-form solution. Performance evaluations show that the proposed methods provide superior performance in terms of HDR MSE and SSIM compared to existing tone-mapping schemes. It is also shown that the LDR image quality resulting from the proposed methods matches that produced by perceptually-based TMOs.

A Link Adaptation Scheme for Efficient Transmission of H.264 Scalable Video Over Multirate WLANs

In this paper, we propose a cross-layer optimization scheme for delivery of scalable video over multirate wireless networks, in particular the popular 802.11 based wireless local area network (WLAN). The 802.11 based networks use a link adaptation mechanism in the physical layer (PHY) to maintain the reliability of transmission under varying channel conditions. When channel condition worsens, the reliability is maintained by employing more robust modulation and coding schemes, at the cost of reduced PHY bit rate. The reduced bit rate will result in lower available throughput for applications. For scalable video streaming applications, the conventional solution to this problem is to reduce the video bit rate by dropping the higher enhancement layers of the scalable video. We show in this article that the video quality can be improved, if the link adaptation scheme uses more intelligent reliability criteria and adjusts the PHY parameters used for delivering each video layer, according to the relative importance of that layer. Our scheme achieves better video quality without increasing the traffic load of the WLAN. For this purpose we present temporal fairness constraints and formulate an optimization problem for assigning different PHY modes to different layers of scalable video; the solution to this problem provides a set of PHY configuration parameters that achieve the highest possible video quality while meeting the admission control constraints in the network. Performance evaluations demonstrate that our method outperforms the existing mechanisms.

HDR image construction from multi-exposed stereo LDR images

In this paper, we present an algorithm that generates high dynamic range (HDR) images from multi-exposed low dynamic range (LDR) stereo images. The vast majority of cameras in the market only capture a limited dynamic range of a scene. Our algorithm first computes the disparity map between the stereo images. The disparity map is used to compute the camera response function which in turn results in the scene radiance maps. A refinement step for the disparity map is then applied to eliminate edge artifacts in the final HDR image. Existing methods generate HDR images of good quality for still or slow motion scenes, but give defects when the motion is fast. Our algorithm can deal with images taken during fast motion scenes and tolerate saturation and radiometric changes better than other stereo matching algorithms.

Dynamic Resource Allocation for MGS H.264/AVC Video Transmission Over Link-Adaptive Networks

In this paper, we address the problem of efficiently allocating network resources to support multiple scalable video streams over a constrained wireless channel. We present a resource allocation framework that jointly optimizes the operation of the link adaptation scheme in the physical layer (PHY), and that of a traffic control module in the network or medium access control (MAC) layer in multirate wireless networks, while satisfying bandwidth/capacity constraints. Multirate networks, such as IEEE 802.16 or IEEE 802.11, adjust the PHY coding and modulation schemes to maintain the reliability of transmission under varying channel conditions. Higher reliability is achieved at the cost of reduced PHY bit-rate which in turn necessitates a reduction in video stream bit-rates. The rate reduction for scalable video is implemented using a traffic control module. Conventional solutions operate unaware of the importance and loss tolerance of data and drop the higher layers of scalable video altogether. In this paper, we consider medium grain scalable (MGS) extension of H.264/AVC video and develop new rate and distortion models that characterize the coded bitstream. Performance evaluations show that our proposed framework results in significant gains over existing schemes in terms of average video PSNR that can reach 3 dB in some cases for different channel SNRs and different bandwidth budgets.

An improved error concealment algorithm for intra-frames in H.264/AVC

The highly error-prone nature of wireless environments and limited computational power of mobile devices necessitates the implementation of robust yet simple error concealment in H.264/AVC. We propose a new and effective error concealment algorithm for intra-coded frames that utilizes the temporal redundancy in a wireless video bitstream. The proposed concealment method supports both raster scan and FMO type slices. Performance evaluations show that our approach achieves significant improvement over existing methods in both PSNR and subjective picture quality.

Channel Aware Multiuser Scalable Video Streaming Over Lossy Under-Provisioned Channels: Modeling and Analysis

In this paper, we analyze the performance of media-aware multiuser video streaming strategies in capacity limited wireless channels suffering from latency problems and packet losses. Wireless video streaming applications are characterized by their bandwidth-intensity, delay-sensitivity, and loss-tolerance. Our main contributions include (i) a rate-minimized unequal erasure protection (UXP) scheme, (ii) an analytical expression for packet delay and play-out deadline of UXP protected scalable video, (iii) a loss-distortion model for hierarchical predictive video coders with picture copy concealment, (iv) an analysis of the performance and complexity of delay-aware, capacity-aware, and optimized UXP streaming scenarios, and (v) we show that the use of unequal error protection causes a rate-constrained optimization problem to be nonconvex. Performance evaluations using a 3GPP network simulator show that, for different channel capacities and packet loss rates, delay-aware nonstationary rate-allocation streaming policies deliver significant gains which range between 1.65 dB to 2 dB in average Y-PSNR of the received video streams over delay-unaware strategies. These gains come at a cost of increased offline computation which is performed prior to the start of the streaming session or in batches during transmission and therefore, do not affect the run-time performance of the streaming system.

Rate and Distortion Modeling of CGS Coded Scalable Video Content

In this paper, we derive single layer and scalable video rate and distortion models for video bitstreams encoded using the coarse grain quality scalability (CGS) feature of the scalable extension of H.264/AVC. In these models, we assume the source is Laplacian distributed and compensate for errors in the distribution assumption by linearly scaling the Laplacian parameter . Moreover, we present simplified approximations of the derived models that allow for a run-time calculation of sequence dependent model constants. Our models use the mean absolute difference (MAD) of the prediction residual signal and the encoder quantization parameter (QP) as input parameters. Consequently, we are able to estimate the residual MAD, bitrate, and distortion of a future video frame at any QP value and for both base-layer and CGS layer packets. We also present simulation results that demonstrate the accuracy of the proposed models.

Color image desaturation using sparse reconstruction

In this paper, we propose an algorithm to estimate the true values of saturated pixels in color images. Pixel saturation occurs when at least one color channel is clipped at some value below the full dynamic range of the scene, resulting in a loss in image fidelity. The proposed algorithm is based on the assumptions that images are nearly sparse in an appropriate transform domain, and that saturated pixels can be inferred from the structure of non-saturated neighboring pixels. Consequently, we use a hierarchical windowing algorithm which selects image regions containing relatively few saturated pixels for processing. Starting with small sized regions, and progressively increasing the size, we solve a sparsity promoting constrained ℓ1 minimization problem for each selected region to recover the saturated pixels. Moreover, we provide simulation results to show the effectiveness of our algorithm.

An Efficient Multiple Description Coding Scheme for the Scalable Extension of H.264/AVC (SVC)

The demand for efficient scalable video codecs has constantly been on the rise in response to the increase in the variety of services and QoS requirements in multimedia networks. Existing standardization efforts, such as the scalable video coding extension of the H.264/AVC standard, do not offer efficient error resilient protection for all the different levels of video enhancement. We developed a multiple description scalable video coding technique that offers complementary and independently decodable descriptions, offering acceptable video quality even if only one of them is successfully received. Performance evaluations show that our scheme delivers an average improvement of 5 dB for single channel decoding and an improvement of 2 dB on average for packet loss simulations when compared with the UXP protected SD-SVC and the multiple-description motion compensated temporal filtering (MD-MCTF) scheme at comparable redundancy levels

Multi-user scalable video transmission control in cognitive radio networks as a Markovian dynamic game

This paper considers the multi-user bit-rate and latency control of scalable video content in a cognitive radio multimedia network. We consider a cognitive radio network where multiple secondary users attempt to access a spectrum hole according to a predefined time division multiple access (TDMA) access rule based on the primary user activities, the channel quality and the transmission delay of each user. Scalable video rate and distortion models are used in formulating the problem as a switching control dynamic Markovian game. The video sources and channel behavior are modeled as independent Markov processes. However, the interaction between users is combined in the access rule thus resulting in a switching control game. We show that the proposed switching control game formulation results in an improvement in video quality over a myopic rate allocation scheme in video peak signal-to-noise ratio (PSNR).