Akbar Sheikh Akbari

A Novel H.264/AVC Based Multi-View Video Coding Scheme

This paper investigates extensions of H.264/AVC for compressing multi-view video sequences. The proposed technique re-sorts frames of sequences captured by multiple cameras looking at a person in a scene from different views and generates a single video sequence. The multi-frame referencing property of the H.264/AVC, which enables exploitation of the spatial and temporal redundancy contained in the multi-view sequences, is employed to implement several modes of operation in the proposed coding algorithm. To evaluate the performance of the proposed coding technique at different modes of operations, five multi-view video sequences at different frame rates were coded using the proposed and the simulcast H.264/AVC coding schemes. Experiments show the superior performance of the proposed coding scheme when coding the multi-view sequences at low and up to half of the original frame rates.

Wavelet Based Image Compression Techniques

With advances in multimedia technologies, demand for transmission and storage of voluminous multimedia data has dramatically increased and, as a consequence, data compression is now essential in reducing the amount of data prior storage or transmission. Compression techniques aim to minimise the number of bits required to represent image data while maintaining an acceptable visual quality. Image compression is achieved by exploiting the spatial and perceptual redundancies present in image data. Image compression techniques are classified into two categories, lossless and lossy. Lossless techniques refer to those that allow recovery of the original input data from its compressed representation without any loss of information, i.e. after decoding, an identical copy of the original data can be restored. Lossy techniques offer higher compression ratios but it is impossible to recover the original data from its compressed data, as some of the input information is lost during the lossy compression. These techniques are designed to minimise the amount of distortion introduced into the image data at certain compression ratios. Compression is usually achieved by transforming the image data into another domain, e.g. frequency or wavelet domains, and then quantizing and losslessy encoding the transformed coefficients (Ghanbari, 1999; Peng & Kieffer, 2004; Wang et al., 2001). In recent years much research has been undertaken to develop efficient image compression techniques. This research has led to the development of two standard image compression techniques called: JPEG and JPEG2000 (JPEG, 1994; JPEG 2000, 2000), and many nonstandard image compression algorithms (Said & Pearlman, 1996; Scargall & Dlay, 2000; Shapiro, 1993).

Stereo image representation using compressive sensing

This paper presents a compressive sensing based stereo image representation technique using wavelet transform gain. The pair of input stereo images is first decomposed into its low-pass and high-pass views using a motion compensated lifting based wavelet transform. A 2D spatial wavelet transform is then further de-correlates the low-pass view into its sub-bands. Wavelet transform gains are employed to regulate threshold value for different sub-bands. The coefficients in high frequency sub-bands and high-pass view are then hard thresholded to generate their sparse sub-bands and view. The compressive sensing method is then used to generate measurements for different resulting sparse sub-bands and view. The baseband coefficients and measurements are finally losslessly coded. The application of compressive sensing in compressing natural images is in its early stages. Therefore, their performances are usually compared with each other than standard codecs. The performance of the proposed codec is superior to the state of the art and is superior to JPEG subjectively.

Multiresolution HVS and statistically based image coding scheme

In this paper a novel multiresolution human visual system and statistically based image coding scheme is presented. It decorrelates the input image into a number of subbands using a lifting based wavelet transform. The codec employs a novel statistical encoding algorithm to code the coefficients in the detail subbands. Perceptual weights are applied to regulate the threshold value of each detail subband that is required in the statistical encoding process. The baseband coefficients are losslessly coded. An extension of the codec to the progressive transmission of images is also developed. To evaluate the performance of the coding scheme, it was applied to a number of test images and its performance with and without perceptual weights is evaluated. The results indicate significant improvement in both subjective and objective quality of the reconstructed images when perceptual weights are employed. The performance of the proposed technique was also compared to JPEG and JPEG2000. The results show that the proposed coding scheme outperforms both coding standards at low compression ratios, while offering satisfactory performance at higher compression ratios.

Multiresolution, perceptual and vector quantization based video codec

This paper presents a novel Multiresolution, Perceptual and Vector Quantization (MPVQ) based video coding scheme. In the intra-frame mode of operation, a wavelet transform is applied to the input frame and decorrelates it into its frequency subbands. The coefficients in each detail subband are pixel quantized using a uniform quantization factor divided by the perceptual weighting factor of that subband. The quantized coefficients are finally coded using a quadtree-coding algorithm. Perceptual weights are specifically calculated for the centre of each detail subband. In the inter-frame mode of operation, a Displaced Frame Difference (DFD) is first generated using an overlapped block motion estimation/compensation technique. A wavelet transform is then applied on the DFD and converts it into its frequency subbands. The detail subbands are finally vector quantized using an Adaptive Vector Quantization (AVQ) scheme. To evaluate the performance of the proposed codec, the proposed codec and the adaptive subband vector quantization coding scheme (ASVQ), which has been shown to outperform H.263 at all bitrates, were applied to six test sequences. Experimental results indicate that the proposed codec outperforms the ASVQ subjectively and objectively at all bit rates.

Effect of inter-camera angles on the performance of an H.264/AVC based multi-view video codec

This paper investigates the effect of inter-camera angles on the performance of an H.264/AVC based multi-view video codec. To achieve this, the H.264/AVC software has been modified to support multi-view video coding using its multi-frame reference property. Results were generated using a wide baseline convergent multi-view video data set: Breakdancers. To generate a set of three synchronized multi-view videos from the same scene with different inter-camera angles, all possible three camera combinations are generated and classified according to their inter-camera angles. The resulting set of multi-view videos are coded using H.264/AVC based multi-view and simulcast video codecs at different bitrates. Results demonstrate that the multi-view video codec gives superior coding performance up to 1.2dB compared to that of simulcast coding scheme at low inter-camera angles and it deteriorates as the inter camera angles increase. Finally, a range of inter-camera angles for best use of either multi-view or simulcast coding is determined.

Colour volumetric compression for realistic view synthesis applications

Colour volumetric data, which is constructed from a set of multi-view images, is capable of providing realistic immersive experience. However it is not widely applicable due to its manifold increase in bandwidth. This paper presents a novel framework to achieve scalable volumetric compression. Based on wavelet transformation, data rearrangement algorithm is proposed to compact volumetric data leading to high efficiency of transformation. The colour data is rearranged using the characteristics of human visual system. A pre-processing scheme for adaptive resolution is also proposed in this paper. The low resolution overcomes the limitation of the data transmission at low bitrates, whilst the fine resolution improves the quality of the synthesised images. Results show significant improvement of the compression performance over the traditional 3D coding. Finally, effect of using residual coding is investigated in order to show a trade off between the compression and view synthesis performance.

Disparity Compensated View Filtering Wavelet Based Multiview Image Code Using Lagrangian Optimization

This paper presents a disparity compensated view filtering wavelet based multiview image coding scheme. The proposed codec decorrelates the input views into their frequency subbands using a disparity compensated lifting based wavelet transform. The codec then applies a 2D wavelet transform on baseband-image transforming it into a number of subbands. The general form of the Lagrangian rate distortion optimization algorithm is then modified and used along with the steepest descent algorithm to assign bits among the different subbands in such a way that it minimizes the PSNR variance of the decoded images. Two sets of experimental results are generated using three sets of multiview test images. In the first set of experiments the effect of using a weighted lambda for high frequency images on the PSNR variance of the decoded images is investigated. In the second set of experiments, performance of the proposed codec in reducing the variance of the PSNR of the decoded images is investigated. Results indicate that the proposed codec gives lower PSNR variance among the decoded images compared to the basic form of the codec at all bitrates.

Multi-scale, Perceptual and Vector Quanitzation Based Video Codec

This paper presents a novel hybrid multi-scale, perceptual and vector quantization based video coding scheme. In intra mode of operation, a wavelet transform is applied to the input frame and decorrelate it into a number of subbands. The lowest frequency subband is losslessly coded. The coefficient of the high frequency subbands are pixel quantized using perceptual weights, which specifically designed for each high frequency subband. The quantized coefficients are then coded using quadtree-coding scheme. In the inter mode of operation, displaced frame difference is generated using overlapped block motion estimation/compensation to exploit the inter-frame redundancy. A wavelet transform is then applied to the displaced frame difference to decorrelate it into a number of subbands. The coefficients in the resulting subbands are coded using an adaptive vector quantization scheme. To evaluate the performance of the proposed codec, the proposed codec and the adaptive subband vector quantization coding scheme (ASVQ), which has been shown outperforms H.263 at all bitrates, were applied to a number of test sequences. Results indicate that the proposed codec outperforms ASVQ subjectively and objectively at all bit rates.

DCT image codec using variance of sub-regions

Abstract This paper presents a novel variance of subregions and discrete cosine transform based image-coding scheme. The proposed encoder divides the input image into a number of non-overlapping blocks. The coefficients in each block are then transformed into their spatial frequencies using a discrete cosine transform. Coefficients with the same spatial frequency index at different blocks are put together generating a number of matrices, where each matrix contains coefficients of a particular spatial frequency index. The matrix containing DC coefficients is losslessly coded to preserve its visually important information. Matrices containing high frequency coefficients are coded using a variance of sub-regions based encoding algorithm proposed in this paper. Perceptual weights are used to regulate the threshold value required in the coding process of the high frequency matrices. An extension of the system to the progressive image transmission is also developed. The proposed coding scheme, JPEG and JPEG2000were applied to a number of test images. Results show that the proposed coding scheme outperforms JPEG and JPEG2000 subjectively and objectively at low compression ratios. Results also indicate that the proposed codec decoded images exhibit superior subjective quality at high compression ratios compared to that of JPEG, while offering satisfactory results to that of JPEG2000.