Aidong Men

Intra prediction with enhanced inpainting method and vector predictor for HEVC

As the successor to H.264/AVC, High Efficiency Video Coding (HEVC) will provide 50% reduction on compression data compared to H.264/AVC. In this paper, we propose an intra prediction method based on inpainting algorithms and vector predictor for HEVC. Our method utilizes a combination of two important inpainting algorithms: Laplace partial differential equation (PDE) and total variation (TV) model. Experiment results show that, compared to HEVC Test Model (HM) 2.0, our proposal achieves an average of 1.65% bitrate reduction.

Depth map compression via edge-based inpainting

This work presents a novel intra-frame coding scheme for depth maps that makes use of the characteristics of depth images: large smooth areas separated by sharp edges. The proposed method is block-based, thus can be easily integrated into H.264/AVC framework. By using the extracted edge information, each block is divided into several regions and each region is handled independently. Most regions are predicted via image inpainting based on the Laplace equation. The others that could not be inpainted are predicted by their mean values or a default value. Both the extracted edge information and the mean values are compressed under the rate-distortion (RD) optimization principle. Compared with H.264/AVC, the results show that the proposed scheme achieves both PSNR gain in depth maps and visual quality improvement in rendered views.

On unequal error protection with low density parity check codes in scalable video coding

In this work, we propose an unequal error protection (UEP) scheme for scalable video coding using Low Density Parity Check (LDPC) codes. LDPC codes are renowned for their near-capacity performance. In the proposed framework, a video sequence is coded based on Scalable Video Coding (SVC) H.264/AVC standard document. We divide the SVC coded video stream into different sub-streams in terms of time and quality scalability. These sub-streams are channel-coded using Low Density Parity Check (LDPC) codes for robustness against channel impairments. LDPC codes with different coding rates are applied for different sub-streams according to the significance level of each sub-stream in order to provide UEP. We develop a probability model for decoding failure using LDPC codes in order to evaluate the bit error rate under different signal-to-noise ratios and delay constraints for an AWGN channel. Further motivated by optimally allocating the bits to maximize the quality of reconstructed video at the decoder against channel induced corruptions, we propose an Optimal Bit Allocation Algorithm (OBAA) that optimally allocates the channel coding rates using a dynamic programming approach. We use the proposed probability model in OBAA in order to obtain rate-distortion characteristics of LDPC coded SVC video. The results achieved clearly establish that the proposed UEP scheme can optimally allocate LDPC codes with different coding rates to different sub-streams that significantly decreases the end-to-end distortion of the system as compared to equal error protection (EEP) scheme resulting in greater system throughputs

An Improved Ranking-Based Feature Enhancement Approach for Robust Speaker Recognition

Although the field of automatic speaker or speech recognition has been extensively studied over the past decades, the lack of robustness has remained a major challenge. Feature warping is a promising approach and its effectiveness significantly depends on the relative positions of each of the features in a sliding window. However, the relative positions are changed due to the non-linear effect of noise. Aiming at the problem, this paper takes the advantage of ranking feature, which is obtained directly by sorting a feature sequence in descending order, to propose a method. It first labels the central frame in a sliding window as speech or noise dominant (“reliable” or “unreliable”). In the unreliable case, the ranking of the central frame is estimated. Subsequently, the estimated ranking is mapped to a warped feature using a desired target distribution for recognition experiments. Through the theoretical analysis and experimental results, it is found that autocorrelation of a ranking sequence is larger than that of the corresponding feature sequence. What is more, rank correlation is not easily influenced by abnormal data or data that are highly variable. Thus, this paper deals with a ranking sequence rather than a feature sequence. The proposed feature enhancement approach is evaluated in an open-set speaker recognition system. The experimental results show that it outperforms missing data method based on linear interpolation and feature warping in terms of recognition performance in all noise conditions. Furthermore, the method proposed here is a feature-based method, which may be combined with other technologies, such as model-based, scores-based, to enhance the robustness of speaker or speech recognition system.

Video saliency detection incorporating temporal information in compressed domain

Saliency detection is widely used to pick out relevant parts of a scene as visual attention regions for various image/video applications. Since video is increasingly being captured, moved and stored in compressed form, there is a need for detecting video saliency directly in compressed domain. In this study, a compressed video saliency detection algorithm is proposed based on discrete cosine transformation (DCT) coefficients and motion information within a visual window. Firstly, DCT coefficients and motion information are extracted from H.264 video bitstream without full decoding. Due to a high quantization parameter setting in encoder, skip/intra is easily chosen as the best prediction mode, resulting in a large number of blocks with zero motion vector and no residual existing in video bitstream. To address these problems, the motion vectors of skip/intra coded blocks are calculated by interpolating its surroundings. In addition, a visual window is constructed to enhance the contrast of features and to avoid being affected by encoder. Secondly, after spatial and temporal saliency maps being generated by the normalized entropy, a motion importance factor is imposed to refine the temporal saliency map. Finally, a variance-like fusion method is proposed to dynamically combine these maps to yield the final video saliency map. Experimental results show that the proposed approach significantly outperforms other state-of-the-art video saliency detection models. HighlightsWe propose a compressed video saliency model for which few attention is given to.The characteristics of codec are considered to remove the effects of QP.We use K-means clustering to statistically distinguish the motion attention level.The visual window is built to strengthen the contrast of features.The variance-like fusion method is used to compute the video saliency map.

Rate control for hierarchical B-frames in H.264/AVC

Abstract Hierarchical B -frames can bring high coding performance when introduced into H.264/AVC. However, the traditional rate control schemes can not work efficiently in such new coding framework. This article presents a rate control algorithm for hierarchical B -frames in H.264/AVC. Taking the feature of the dyadic hierarchical coding structure into consideration, the proposed algorithm includes group of pictures (GOP) layer, temporal layer and frame layer bits allocation. After frame layer bits allocation is complete, frame layer quantization parameters (QP) determination strategy is responsible for calculating the final QP. Experimental results show that compared with other rate control algorithms, the proposed one can improve the coding performance and reduce the mismatch of target bit rate and real bit rate.

No-reference image quality assessment based on phase congruency and spectral entropies

We develop an efficient general-purpose blind/no-reference image quality assessment (IQA) algorithm that utilizes curvelet domain features of phase congruency values and local spectral entropy values in distorted images. A 2-stage framework of distortion classification followed by quality assessment is used for mapping feature vectors to prediction scores. We utilize a support vector machine (SVM) to train an image distortion and quality prediction model. The resulting algorithm which we name Phase Congruency and Spectral Entropy based Quality (PCSEQ) index is capable of assessing the quality of distorted images across multiple distortion categories. We explain the advantages of phase congruency features and spectral entropy features. We also thoroughly test the algorithm on the LIVE IQA databse and find that PCSEQ correlates well with human judgments of quality. It is superior to the full-reference (FR) IQA algorithm SSIM and several top-performance no-reference (NR) IQA methods such as DIIVINE and SSEQ. We also tested PCSEQ on the TID2008 database to ascertain whether it has performance that is database independent.

A novel video saliency map detection model in compressed domain

Saliency detection in videos has attracted great attention in recent years due to its wide range of applications. In this paper, a novel spatiotemporal saliency detection model based on clustering is proposed. The discrete cosine transform coefficients are used as features to generate the spatial saliency map firstly. We utilize 2D Gaussian function to estimate the absolute feature difference in consideration of video resolution. Multiple spatial saliency maps which indicate different features are constructed and linearly combined to obtain the overall spatial saliency map. Then, a hierarchical structure is utilized to obtain the temporal saliency map using the extracted motion vectors that belong to the foreground. In addition, spatial and temporal saliency maps are clustered into non-overlapping regions automatically based on the histogram of each saliency map. Finally, an adaptive fusion method is used to merge clustered spatial and temporal saliency maps of each frame into its spatiotemporal saliency map. Based on the experimental results obtained in our study, the performance of the proposed approach is better than those of the other compared approaches.

A perceptual quality metric based rate-quality optimization of H.265/HEVC

An effective video coding should not only remove statistical redundancy but also take into account the characteristic of human visual system. Just noticeable difference (JND) represents the maximum distortion that cannot be perceived in a suitable viewing condition. In this paper, we introduce a foveated JND model combined with visual attention model. Though moving object always attracts viewers attention when watching video, viewer could not distinguish every detail of all types of movement. We classify the movement into three types to guide foveated JND model. Then we use the foveated JND model to modify block-based multi-metric fusion (BMMF). In the end, we integrate the modified BMMF metric into traditional hybrid coding scheme. Experimental results show that the proposed model can guarantee better subjective performance at the same bit rate.

A novel fusion method in distributed multi-view video coding over wireless video sensor network

To meet the special requirements of resource-limited video sensors in wireless video sensor network (WVSN), low-complexity video encoding technique is highly desired. In distributed multi-view video coding (DMVC) system, multi-view video sources are encoded separately and decoded dependently, so the burden of huge computation is shifted from the encoder side to the decoder side. The generation of the side information (SI) is an important part in the design of a DMVC system as it directly relates to the systems performance. In this paper, a new fusion method combining the histogram matching and the minimum sum of the absolute differences (SAD) criterion is proposed to generate the final SI. The simulation results show that the proposed method generates more qualified SI and improves the peak signal to noise ratio (PSNR) performance at an average 1.6dB gain for reconstructed frame when compared to the traditional fusion methods in DMVC system.