Yongbing Zhang

A Spatio-Temporal Auto Regressive Model for Frame Rate Upconversion

This paper proposes a spatio-temporal auto regressive (STAR) model for frame rate upconversion. In the STAR model, each pixel in the interpolated frame is approximated as the weighted combination of a sample space including the pixels within its two temporal neighborhoods from the previous and following original frames as well as the available interpolated pixels within its spatial neighborhood in the current to-be-interpolated frame. To derive accurate STAR weights, an iterative self-feedback weight training algorithm is proposed. In each iteration, first the pixels of each training window in the interpolated frames are approximated by the sample space from the previous and following original frames and the to-be-interpolated frame. And then the actual pixels of each training window in the original frame are approximated by the sample space from the previous and following interpolated frames and the current original frame with the same weights. The weights of each training window are calculated by jointly minimizing the distortion between the interpolated frames in the current and previous iterations as well as the distortion between the original frame and its interpolated one. Extensive simulation results demonstrate that the proposed STAR model is able to yield the interpolated frames with high performance in terms of both subjective and objective qualities.

A Motion-Aligned Auto-Regressive Model for Frame Rate Up Conversion

In this paper, a motion-aligned auto-regressive (MAAR) model is proposed for frame rate up conversion, where each pixel is interpolated as the average of the results generated by one forward MAAR (Fw-MAAR) model and one backward MAAR (Bw-MAAR) model. In the Fw-MAAR model, each pixel in the to-be-interpolated frame is generated as a linear weighted summation of the pixels within a motion-aligned square neighborhood in the previous frame. To derive more accurate interpolation weights, the aligned actual pixels in the following frame are also estimated as a linear weighted summation of the newly interpolated pixels in the to-be-interpolated frame by the same weights. Consequently, the backward-aligned actual pixels in the following frame can be estimated as a weighted summation of the corresponding pixels within an enlarged square neighborhood in the previous frame. The Bw-MAAR is performed likewise except that it is operated in the reverse direction. A damping Newton algorithm is then proposed to compute the adaptive interpolation weights for the Fw-MAAR and Bw-MAAR models. Extensive experiments demonstrate that the proposed MAAR model is able to achieve superior performance than the traditional frame interpolation methods such as MCI, OBMC, and AOBMC, and it is even better than STAR model for the most test sequences with moderate or large motions.

Interpolation-Dependent Image Downsampling

Traditional methods for image downsampling commit to remove the aliasing artifacts. However, the influences on the quality of the image interpolated from the downsampled one are usually neglected. To tackle this problem, in this paper, we propose an interpolation-dependent image downsampling (IDID), where interpolation is hinged to downsampling. Given an interpolation method, the goal of IDID is to obtain a downsampled image that minimizes the sum of square errors between the input image and the one interpolated from the corresponding downsampled image. Utilizing a least squares algorithm, the solution of IDID is derived as the inverse operator of upsampling. We also devise a content-dependent IDID for the interpolation methods with varying interpolation coefficients. Numerous experimental results demonstrate the viability and efficiency of the proposed IDID.

Packet Video Error Concealment With Auto Regressive Model

In this paper, auto regressive (AR) model is applied to error concealment for block-based packet video coding. In the proposed error concealment scheme, the motion vector for each corrupted block is first derived by any kind of recovery algorithms. Then each pixel within the corrupted block is replenished as the weighted summation of pixels within a square centered at the pixel indicated by the derived motion vector in a regression manner. Two block-dependent AR coefficient derivation algorithms under spatial and temporal continuity constraints are proposed respectively. The first one derives the AR coefficients via minimizing the summation of the weighted square errors within all the available neighboring blocks under the spatial continuity constraint. The confidence weight of each pixel sample within the available neighboring blocks is inversely proportional to the distance between the sample and the corrupted block. The second one derives the AR coefficients by minimizing the summation of the weighted square errors within an extended block in the previous frame along the motion trajectory under the temporal continuity constraint. The confidence weight of each extended sample is inversely proportional to the distance toward the corresponding motion aligned block whereas the confidence weight of each sample within the motion aligned block is set to be one. The regression results generated by the two algorithms are then merged to form the ultimate restorations. Various experimental results demonstrate that the proposed error concealment strategy is able to improve both the objective and subjective quality of the replenished blocks compared to other methods.

Nonlocal Edge-Directed Interpolation

In this paper, we proposed a new edge-directed image interpolation algorithm which can preserve the edge features and natural appearance of images efficiently. In the proposed scheme, we first get a close-form solution of the optimal interpolation coefficients under the sense of minimal mean square error by exploiting autoregressive model (AR) and the geometric duality between the low-resolution and high-resolution images .Then the coefficients of the Nonlocal Edge-directed interpolation (NLEDI) are derived with structure similarity in images, which are solutions of weighted least square equations. The new image interpolation approach preserves spatial coherence of the interpolated images better than the existing methods and it outperforms the other methods in terms of objective and subjective image quality.

CCR: Clustering and Collaborative Representation for Fast Single Image Super-Resolution

Clustering and collaborative representation (CCR) have recently been used in fast single image super-resolution (SR). In this paper, we propose an effective and fast single image super-resolution (SR) algorithm by combining clustering and collaborative representation. In particular, we first cluster the feature space of low-resolution (LR) images into multiple LR feature subspaces and group the corresponding high-resolution (HR) feature subspaces. The local geometry property learned from the clustering process is used to collect numerous neighbor LR and HR feature subsets from the whole feature spaces for each cluster center. Multiple projection matrices are then computed via collaborative representation to map LR feature subspaces to HR subspaces. For an arbitrary input LR feature, the desired HR output can be estimated according to the projection matrix, whose corresponding LR cluster center is nearest to the input. Moreover, by learning statistical priors from the clustering process, our clustering-based SR algorithm would further decrease the computational time in the reconstruction phase. Extensive experimental results on commonly used datasets indicate that our proposed SR algorithm obtains compelling SR images quantitatively and qualitatively against many state-of-the-art methods.

CONCOLOR: Constrained Non-Convex Low-Rank Model for Image Deblocking

Due to independent and coarse quantization of transform coefficients in each block, block-based transform coding usually introduces visually annoying blocking artifacts at low bitrates, which greatly prevents further bit reduction. To alleviate the conflict between bit reduction and quality preservation, deblocking as a post-processing strategy is an attractive and promising solution without changing existing codec. In this paper, in order to reduce blocking artifacts and obtain high-quality image, image deblocking is formulated as an optimization problem within maximum a posteriori framework, and a novel algorithm for image deblocking using constrained non-convex low-rank model is proposed. The lp (0 <; p <; 1) penalty function is extended on singular values of a matrix to characterize low-rank prior model rather than the nuclear norm, while the quantization constraint is explicitly transformed into the feasible solution space to constrain the non-convex low-rank optimization. Moreover, a new quantization noise model is developed, and an alternatively minimizing strategy with adaptive parameter adjustment is developed to solve the proposed optimization problem. This parameter-free advantage enables the whole algorithm more attractive and practical. Experiments demonstrate that the proposed image deblocking algorithm outperforms the current state-of-the-art methods in both the objective quality and the perceptual quality.

A No-Reference Blocking Artifacts Metric Using Selective Gradient and Plainness Measures

This paper presents a novel no-reference blocking artifacts metric using selective gradient and plainness (BAM_SGP) measures for DCT-coded images. A boundary selection criterion is introduced to distinguish the blocking artifacts boundaries from the true-edge boundaries, which ensures that the most potential artifacts boundaries are involved in the measurement. Next, the artifacts are evaluated by the gradient and plainness measures indicating different aspects of blocking artifacts characteristics. Then these two measures are fused into a metric of blocking artifacts. Compared with some existing metrics, experiments on the LIVE database and our own test set show that the proposed metric can keep better consistent with Mean Opinion Score (MOS).

Image deblocking using group-based sparse representation and quantization constraint prior

To alleviate the conflict between bit reduction and quality preservation, deblocking as a post-processing strategy is an attractive and promising solution without changing existing codec. In this paper, in order to reduce blocking artifacts and obtain high-quality image, image deblocking is formulated as an optimization problem via maximum a posteriori framework, and a novel algorithm for image deblocking using group-based sparse representation (GSR) and quantization constraint (QC) prior is proposed. GSR prior is utilized to simultaneously enforce the intrinsic local sparsity and the nonlocal self-similarity of natural images, while QC prior is explicitly incorporated to ensure a more reliable and robust estimation. A new split Bregman iteration based method with adaptively adjusted regularization parameter is developed to solve the proposed optimization problem for image deblocking. The parameter-adaptive advantage enables the whole algorithm more attractive and practical. Experiments manifest that the proposed image deblocking algorithm improves current state-of-the-art results by a large margin in both PSNR and visual perception.

Side information generation with auto regressive model for low-delay distributed video coding

In this paper, we propose an auto regressive (AR) model to generate the high quality side information (SI) for Wyner-Ziv (WZ) frames in low-delay distributed video coding, where the future frames are not used for generating SI. In the proposed AR model, the SI of each pixel within the current WZ frame t is generated as a linear weighted summation of the pixels within a window in the previous reconstructed WZ/Key frame t-1 along the motion trajectory. To obtain accurate SI, the AR model is used in both temporal directions in the reconstructed WZ/Key frames t-1 and t-2, and then the regression results are fused with traditional extrapolation result based on a probability model. In each temporal direction, a weighting coefficient set is computed by the least mean square method for each block in the current WZ frame t. In particular, due to the unavailability of future frames in low-delay distributed video coding, a centrosymmetric rearrangement is proposed for pixel generation in the backward direction. Various experimental results demonstrate that the proposed model is able to achieve a higher performance compared to the existing SI generation methods.