Xiaohai He

Learning-based super resolution using kernel partial least squares

In this paper, we propose a learning-based super resolution approach consisting of two steps. The first step uses the kernel partial least squares (KPLS) method to implement the regression between the low-resolution (LR) and high-resolution (HR) images in the training set. With the built KPLS regression model, a primitive super-resolved image can be obtained. However, this primitive HR image loses some detailed information and does not guarantee the compatibility with the LR one. Therefore, the second step compensates the primitive HR image with a residual HR image, which is the subtraction of the original and primitive HR images. Similarly, the residual LR image is obtained from the down-sampled version of the primitive HR and original LR image. The relation of the residual LR and HR images is again modeled with KPLS. Integration of the primitive and the residual HR image will achieve the final super-resolved image. The experiments with face, vehicle plate, and natural scene images demonstrate the effectiveness of the proposed approach in terms of visual quality and selected image quality metrics.

An automated vision system for container-code recognition

Highlights? An automatic container-code recognition system is developed by using computer vision. ? The characteristics of characters are made full use of to locate container-code. ? A two-step method is proposed to segment characters for various imaging conditions. Automatic container-code recognition is of great importance to the modern container management system. Similar techniques have been proposed for vehicle license plate recognition in past decades. Compared with license plate recognition, automatic container-code recognition faces more challenges due to the severity of nonuniform illumination and invalidation of color information. In this paper, a computer vision based container-code recognition technique is proposed. The system consists of three function modules, namely location, isolation, and character recognition. In location module, we propose a text-line region location algorithm, which takes into account the characteristics of single character as well as the spatial relationship between successive characters. This module locates the text-line regions by using a horizontal high-pass filter and scanline analysis. To resolve nonuniform illumination, a two-step procedure is applied to segment container-code characters, and a projection process is adopted to isolate characters in the isolation module. In character recognition module, the character recognition is achieved by classifying the extracted features, which represent the character image, with trained support vector machines (SVMs). The experimental results demonstrate the efficiency and effectiveness of the proposed technique for practical usage.

Single-image super-resolution based on Markov random field and contourlet transform

Learning-based methods are well adopted in image super-resolution. In this paper, we propose a new learning-based approach using contourlet transform and Markov random field. The proposed algorithm employs contourlet transform rather than the conventional wavelet to represent image features and takes into account the correlation between adjacent pixels or image patches through the Markov random field (MRF) model. The input low-resolution (LR) image is decomposed with the contourlet transform and fed to the MRF model together with the contourlet transform coefficients from the low- and high-resolution image pairs in the training set. The unknown high-frequency components/coefficients for the input low-resolution image are inferred by a belief propagation algorithm. Finally, the inverse contourlet transform converts the LR input and the inferred high-frequency coefficients into the super-resolved image. The effectiveness of the proposed method is demonstrated with the experiments on facial, vehicle plate, and real scene images. A better visual quality is achieved in terms of peak signal to noise ratio and the image structural similarity measurement.

Video superresolution reconstruction based on subpixel registration and iterative back projection

To improve the spatial resolution of video, a superresolution reconstruction method based on a sliding window is proposed utilizing the movement information between frames in the low-resolution video. We propose a registration algorithm based on a four-parameter transformation model through Taylor series expansion, using an iterative solving method as well as the Gaussian pyramid image model to estimate the movement parameters from coarseness to fine. Superresolution frames are reconstructed using an iterative back projection (IBP) algorithm. We also present the suitable length of the sliding window and the reasonable iteration number of the IBP algorithm in the video superresolution reconstruction. Our algorithm is compared to other algorithms on simulated images and actual color videos. Both show that our registration algorithm achieves higher subpixel accuracy than other algorithms, even in the case of large movements, and that the reconstructed video has better visual effects and stronger resolution ability. It can be extensively applied to the superresolution reconstruction of video sequences in which the frames are different from each other mainly by translation and rotation.

A pixel selection rule based on the number of different‐phase neighbours for the simulated annealing reconstruction of sandstone microstructure

Sandstone reservoir is one of the main types of oil and gas reservoirs in China. It has porous microstructure, which directly affects the transport properties of a sandstone. Hence, the study of porous microstructure is important to the exploration and exploitation of oil and gas. Three‐dimensional microstructure of a sandstone can be reconstructed using the simulated annealing method based on statistical properties of its two‐dimensional micrograph. The aim of reconstruction is to minimize the discrepancy between the statistical properties of the reconstructed microstructure and those of the two‐dimensional image. To accelerate the rate of convergence, we proposed a different‐phase neighbours (DPNs)‐based pixel selection rule to replace the random pixel selection rule of the simulated annealing reconstruction. In this rule, pixels with the largest number of DPNs have the largest selection probability. The selection probabilities of other pixels are proportional to their DPNs. Microstructure reconstructed with the DPNs‐based rule is compared with those with the random selection rule and two other biased pixel selection rules. The DPNs‐based rule is the most effective in enhancing convergence. Permeability of the microstructure reconstructed with the DPNs‐based rule is estimated by the Kozeny–Carman formula and is in good agreement with the one reconstructed with the random pixel selection rule.

Fast inter-mode decision algorithm for high-efficiency video coding based on similarity of coding unit segmentation and partition mode between two temporally adjacent frames

Abstract. High-efficiency video coding (HEVC) introduces a flexible hierarchy of three block structures: coding unit (CU), prediction unit (PU), and transform unit (TU), which have brought about higher coding efficiency than the current national video coding standard H.264/advanced video coding (AVC). HEVC, however, simultaneously requires higher computational complexity than H.264/AVC, although several fast inter-mode decisions were proposed in its development. To further reduce this complexity, a fast inter-mode decision algorithm is proposed based on temporal correlation. Because of the distinct difference of inter-prediction block between HEVC and H.264/AVC, in order to use the temporal correlation to speed up the inter prediction, the correlation of inter-prediction between two adjacent frames needs to be analyzed according to the structure of CU and PU in HEVC. The probabilities of all the partition modes in all sizes of CU and the similarity of CU segmentation and partition modes between two adjacent frames are tested. The correlation of partition modes between two CUs with different sizes in two adjacent frames is tested and analyzed. Based on the characteristics tested and analyzed, at most, two prior partition modes are evaluated for each level of CU, which reduces the number of rate distortion cost calculations. The simulation results show that the proposed algorithm further reduces coding time by 33.0% to 43.3%, with negligible loss in bitrate and peak signal-to-noise ratio, on the basis of the fast inter-mode decision algorithms in current HEVC reference software HM7.0.

Moving Object Detection With a Freely Moving Camera via Background Motion Subtraction

Detection of moving objects in a video captured by a freely moving camera is a challenging problem in computer vision. Most existing methods often assume that the background (BG) can be approximated by dominant single plane/multiple planes or impose significant geometric constraints on BG, or utilize a complex BG/foreground probabilistic model. Instead, we propose a computationally efficient algorithm that is able to detect moving objects accurately and robustly in a general 3D scene. This problem is formulated as a coarse-to-fine thresholding scheme on the particle trajectories in the video sequence. First, a coarse foreground (CFG) region is extracted by performing reduced singular value decomposition on multiple matrices that are built from bundles of particle trajectories. Next, the BG motion of pixels in the CFG region is reconstructed by a fast inpainting method. After subtracting the BG motion, the fine foreground is segmented out by an adaptive thresholding method that is capable of solving multiple-moving-objects scenarios. Finally, the detected foreground is further refined by the mean-shift segmentation method. Extensive simulations and a comparison with the state-of-the-art methods verify the effectiveness of the proposed method.

Single Image Super-Resolution via Adaptive High-Dimensional Non-Local Total Variation and Adaptive Geometric Feature

Single image super-resolution (SR) is very important in many computer vision systems. However, as a highly ill-posed problem, its performance mainly relies on the prior knowledge. Among these priors, the non-local total variation (NLTV) prior is very popular and has been thoroughly studied in recent years. Nevertheless, technical challenges remain. Because NLTV only exploits a fixed non-shifted target patch in the patch search process, a lack of similar patches is inevitable in some cases. Thus, the non-local similarity cannot be fully characterized, and the effectiveness of NLTV cannot be ensured. Based on the motivation that more accurate non-local similar patches can be found by using shifted target patches, a novel multishifted similar-patch search (MSPS) strategy is proposed. With this strategy, NLTV is extended as a newly proposed super-high-dimensional NLTV (SHNLTV) prior to fully exploit the underlying non-local similarity. However, as SHNLTV is very high-dimensional, applying it directly to SR is very difficult. To solve this problem, a novel statistics-based dimension reduction strategy is proposed and then applied to SHNLTV. Thus, SHNLTV becomes a more computationally effective prior that we call adaptive high-dimensional non-local total variation (AHNLTV). In AHNLTV, a novel joint weight strategy that fully exploits the potential of the MSPS-based non-local similarity is proposed. To further boost the performance of AHNLTV, the adaptive geometric duality (AGD) prior is also incorporated. Finally, an efficient split Bregman iteration-based algorithm is developed to solve the AHNLTV-AGD-driven minimization problem. Extensive experiments validate the proposed method achieves better results than many state-of-the-art SR methods in terms of both objective and subjective qualities.

Single Image Super-Resolution Using Local Geometric Duality and Non-Local Similarity

Super-resolution (SR) from a single image plays an important role in many computer vision applications. It aims to estimate a high-resolution (HR) image from an input low- resolution (LR) image. To ensure a reliable and robust estimation of the HR image, we propose a novel single image SR method that exploits both the local geometric duality (GD) and the non-local similarity of images. The main principle is to formulate these two typically existing features of images as effective priors to constrain the super-resolved results. In consideration of this principle, the robust soft-decision interpolation method is generalized as an outstanding adaptive GD (AGD)-based local prior. To adaptively design weights for the AGD prior, a local non-smoothness detection method and a directional standard-deviation-based weights selection method are proposed. After that, the AGD prior is combined with a variational-framework-based non-local prior. Furthermore, the proposed algorithm is speeded up by a fast GD matrices construction method, which primarily relies on the selective pixel processing. The extensive experimental results verify the effectiveness of the proposed method compared with several state-of-the-art SR algorithms.

Fast Inter-Mode Decision Algorithm for High-Efficiency Video Coding Based on Textural Features

Due to the problems of the new generation of video coding standard HEVC (High-Efficiency Video Coding), such as high computational complexity and the large computation, This paper proposes a fast inter-mode decision algorithm based on image texture features and by using Sobel operator the edge features are extracted from CU which is partitioned by simulation, and then the final partitioning size of CU is determined by the texture features contained in the current CU block of simulation partitioning. By using this method, both of the traverse layers of CU depth and the times of inter predictive coding are reduced. Thus the computational complexity of coding terminal is lowered effectively. The experimental results showed that, compared with inter-mode decision algorithm in HEVC standard, the time of this method is saved 45.35% on average with little loss of coding efficiency and PSNR