Xinwei Gao

Hierarchical frame based spatial-temporal recovery for video compressive sensing coding

In this paper, the divide-and-conquer based hierarchical video compressive sensing (CS) coding framework is proposed, in which the whole video is independently divided into non-overlapped blocks of the hierarchical frames. The proposed framework outperforms the traditional framework through the better exploitation of frames correlation with reference frames, the unequal sample subrates setting among frames in different layers and the reduction of the error propagation. At the encoder, compared with the video/frame based CS, the proposed hierarchical block based CS matrix can be easily implemented and stored in hardware. Each measurement of the block in a different hierarchical frame is obtained with the different sample subrate. At the decoder, by considering the spatial and temporal correlations of the video sequence, a spatial-temporal sparse representation based recovery is proposed, in which the similar blocks in the current frame and these recovered reference frames are organized as a spatial-temporal group unit to be represented sparsely. Finally, the recovery problem of video compressive sensing coding can be solved by adopting the split Bregman iteration. Experimental results show that the proposed method achieves better performance against many state-of-the-art still-image CS and video CS recovery algorithms.

Low bit-rate video coding via mode-dependent adaptive regression for wireless visual communications

In this paper, a practical video coding scheme is developed to realize state-of-the-art video coding efficiency with lower encoder complexity at low bit-rate, while supporting standard compliance and error resilience. Such an architecture is particularly attractive for wireless visual communications. At the encoder, multiple descriptions of a video sequence are generated in the spatio-temporal domain by temporal multiplexing and spatial adaptive downsampling. The resulting side descriptions are interleaved with each other in temporal domain, and still with conventional square sample grids in spatial domain. As such, each side description can be compressed without any change to existing video coding standards. At the decoder, each side description is first decompressed, and then reconstructed to original resolution with the help of the other side description. In this procedure, the decoder recover the original video sequence in a constrained least squares regression process, using 2D or 3D piecewise autoregressive model according to different prediction modes. In this way, the spatial and temporal correlation is sufficiently explored to achieve superior quality. Experiment results demonstrate the proposed video coding scheme outperforms H.264 in rate-distortion performance at low bit-rates and achieves superior visual quality at medium bit-rates as well.

High-quality image interpolation via local autoregressive and nonlocal 3-D sparse regularization

In this paper, we propose a novel image interpolation algorithm, which is formulated via combining both the local autoregressive (AR) model and the nonlocal adaptive 3-D sparse model as regularized constraints under the regularization framework. Estimating the high-resolution image by the local AR regularization is different from these conventional AR models, which weighted calculates the interpolation coefficients without considering the rough structural similarity between the low-resolution (LR) and high-resolution (HR) images. Then the nonlocal adaptive 3-D sparse model is formulated to regularize the interpolated HR image, which provides a way to modify these pixels with the problem of numerical stability caused by AR model. In addition, a new Split-Bregman based iterative algorithm is developed to solve the above optimization problem iteratively. Experiment results demonstrate that the proposed algorithm achieves significant performance improvements over the traditional algorithms in terms of both objective quality and visual perception.

Block-Based Compressive Sensing Coding of Natural Images by Local Structural Measurement Matrix

Gaussian random matrix (GRM) has been widely used to generate linear measurements in compressive sensing (CS) of natural images. However, in practice, there actually exist two problems with GRM. One is that GRM is non-sparse and complicated, leading to high computational complexity and high difficulty in hardware implementation. The other is that regardless of the characteristics of signal the measurements generated by GRM are also random, which results in low efficiency of compression coding. In this paper, we design a novel local structural measurement matrix (LSMM) for block-based CS coding of natural images by utilizing the local smooth property of images. The proposed LSMM has two main advantages. First, LSMM is a highly sparse matrix, which can be easily implemented in hardware, and its reconstruction performance is even superior to GRM at low CS sampling sub rate. Second, the adjacent measurement elements generated by LSMM have high correlation, which can be exploited to greatly improve the coding efficiency. Furthermore, this paper presents a new framework with LSMM for block-based CS coding of natural images, including measurement generating, measurement coding and CS reconstruction. Experimental results show that the proposed framework with LSMM for block-based CS coding of natural images greatly enhances the existing CS coding performance when compared with other state-of-the-art image CS coding schemes.

Mode-dependent intra frame interpolation for H.264/AVC compressed video

In this paper, a mode-dependent intra frame interpolation method is proposed for H.264/AVC compressed video. The intra prediction mode information is taken into account in the interpolation filter design. For each intra prediction mode, an optimal Wiener filter is trained based on the representative video sequences. Therefore the trained filter is adaptive to the intra prediction mode. Furthermore, the quantization parameter is also explored as context information for filter selection. Extensive experiments demonstrate that the proposed method achieves better performance than the traditional methods such as Bicubic, Bilinear, LAZA and NEDI, while keeping low computational complexity.

Spatial-temporal recovery for hierarchical frame based video compressed sensing

In this paper, the hierarchical frame based video compressed sensing (CS) framework is proposed, which outperforms the traditional framework through the better exploitation of frames correlation with reference frames, the unequal sample subrates setting among frames in different layers and the reduction of the error propagation. By considering the spatial and temporal correlations of the video sequence, a spatial-temporal sparse representation based recovery is proposed for this framework. The similar blocks in both the current frame and these recovered reference frames are composed as a spatial-temporal group, which is defined as the unit of the sparse representation. By exploiting the low dimensional subspace description of each group, the video CS recovery is converted as a low-rank matrix approximation problem, which can be solved by exploiting the hard thresholding and the gradient descent. Experimental results show that the proposed method achieves better performance against both the state-of-art still-image CS recovery algorithms and the existing residual domain based video CS reconstruction approaches.

Model-based low bit-rate video coding for resource-deficient wireless visual communication

In this paper, an effective low bit-rate video coding scheme is developed to realize state-of-the-art video coding efficiency with lower encoder complexity, while supporting standard compliance and error resilience. Such an architecture is particularly attractive for application scenarios involving resource-deficient wireless video communications. At the encoder, in order to increase resilience to channel error, multiple descriptions of a video sequence are generated in the spatio-temporal domain by temporal multiplexing and spatial adaptive downsampling. The resulting side descriptions are interleaved with each other in temporal domain, while still with conventional square sample grids in spatial domain. As such, each side description can be compressed without any change to existing video coding standards. At the decoder, each side description is first decompressed, and then reconstructed to the original resolution with the help of the other side description. In this procedure, the decoder recovers the original video sequence in a constrained least squares regression process, in which 2D or 3D piecewise autoregressive model is adaptively chosen according to different predictive modes. In this way, the spatial and temporal correlation is sufficiently explored to achieve superior quality. Experimental results demonstrate that the proposed video coding scheme outperforms H.264/AVC and other state-of-the-art methods in rate-distortion performance at low bit-rates and achieves superior visual quality at medium bit rates as well, while with lower encoding computational complexity.

Motion vector refinement for frame rate up conversion on 3D video

With the rapid development of digital video technology, frame rate up conversion is widely used. In this paper, a novel motion vector refinement method for frame rate up conversion on depth based 3D video is proposed. Our method involves two major stages in frame rate up conversion which are motion estimation and motion vector filtering. In the motion estimation process, the depth constraint to block matching algorithm is introduced in the bi-directional motion estimation method to obtain the motion vectors. In the motion vector filtering process, a depth-guided filter is designed to enhance the consistence of motions in the same depth plane. The refined motion vectors are used for frame interpolation. Experimental results show that the proposed method achieves 0.45 dB gain in terms of PSNR on average and improves the visual quality of the frame rate up-converted video.

Reference image based method of region of interest enhancement for haze image

Different from general algorithms of haze removal and low lighting image enhancement, which only use the information of image to process, this paper adds a reference image to get more information for the algorithm and focuses on enhancing region of interest of an image based on the reference one. With the reference image, the haze one can be divided into Region of Interest (RoI) and Region of no Interest (non-RoI). Furthermore, the reference image can provide more useful information for computing the transmission map and atmospheric light. For the non-RoI region, a more robust transmission map and minimizing reconstruction error cost function based method to estimate atmospheric light has been proposed. Because the atmospheric light is a global variable, the optimized one is also suitable for the RoI region. With the global optimized atmospheric light, an optimized transmission map can be got for the RoI region. The RoI region can be enhanced via the optimal transmission map and atmosphere light. Theoretical analysis gives eloquent proof proving that the proposed method is definitely better than the traditional dark-channel-prior-based methods due to our better transmission map and atmosphere light. Extensive experiments also show the expected results.

Directional intra frame interpolation for HEVC compressed video

Image interpolation is one of the most elementary imaging research topics. A number of image interpolation methods have been developed and tested on uncompressed images in the literature. However, a lot of videos have already been stored or have to be transmitted in compressed format due to the storage limitation or the bandwidth limitation. The existed image interpolation methods may not be efficient when directly applied to compressed images or videos. Inspired by the success of the intra prediction in HEVC and the edge-directed image interpolation methods, a directional intra frame interpolation for HEVC compressed video is proposed. The main idea is to use the directional prediction information in compressed low-resolution video bitstreams to estimate the associated high-resolution video. For intra frames, the prediction direction information is taken into account as context in the directional interpolation. When a pixel is decompressed with a small prediction residual, the interpolation is performed along its block direction. The interpolation weight for each block direction is off-line trained by the Wiener filter based on the representative video sequences. For each pixel with a large prediction residual, a piecewise autoregressive model is used as a regularization term into the interpolation function. Extensive experiments demonstrate that the proposed method achieves better performance than the traditional methods such as Bicubic, KR, LAZA, NEDI and SAI.