Jun-Jie Huang

Fast Image Interpolation via Random Forests

This paper proposes a two-stage framework for fast image interpolation via random forests (FIRF). The proposed FIRF method gives high accuracy, as well as requires low computation. The underlying idea of this proposed work is to apply random forests to classify the natural image patch space into numerous subspaces and learn a linear regression model for each subspace to map the low-resolution image patch to high-resolution image patch. The FIRF framework consists of two stages. Stage 1 of the framework removes most of the ringing and aliasing artifacts in the initial bicubic interpolated image, while Stage 2 further refines the Stage 1 interpolated image. By varying the number of decision trees in the random forests and the number of stages applied, the proposed FIRF method can realize computationally scalable image interpolation. Extensive experimental results show that the proposed FIRF(3, 2) method achieves more than 0.3 dB improvement in peak signal-to-noise ratio over the state-of-the-art nonlocal autoregressive modeling (NARM) method. Moreover, the proposed FIRF(1, 1) obtains similar or better results as NARM while only takes its 0.3% computational time.

Learning Hierarchical Decision Trees for Single-Image Super-Resolution

Sparse representation has been extensively studied for image super-resolution (SR), and it achieved great improvement. Deep-learning-based SR methods have also emerged in the literature to pursue better SR results. In this paper, we propose to use a set of decision tree strategies for fast and high-quality image SR. Our proposed SR using decision tree (SRDT) method takes the divide-and-conquer strategy, which performs a few simple binary tests to classify an input low-resolution (LR) patch into one of the leaf nodes and directly multiplies this LR patch with the regression model at that leaf node for regression. Both the classification process and the regression process take an extremely small amount of computation. To further boost the SR results, we introduce a SR using hierarchical decision trees (SRHDT) method, which cascades multiple layers of decision trees for SR and progressively refines the estimated high-resolution image. Inspired by the random forests approach, which combines regression models from an ensemble of decision trees, we propose to fuse regression models from relevant leaf nodes within the same decision tree to form a more robust approach. The SRHDT method with fused regression model (SRHDT_f) improves further the SRHDT method by 0.1-dB in PNSR. Our experimental results show that our initial approach, the SRDT method, achieves SR results comparable to those of the sparse-representation-based method and the deep-learning-based method, but our method is much faster. Furthermore, our enhanced version, the SRHDT_f method, achieves more than 0.3-dB higher PSNR than that of the A+ method, which is the state-of-the-art method in SR.

Practical application of random forests for super-resolution imaging

In this paper, a novel learning-based single image super-resolution method using random forest is proposed. Different from example-based super-resolution methods which search for similar image patches from an external database or the input image, and the sparse representation model based methods which rely on the sparse representation, this proposed super-resolution with random forest (SRRF) method takes the divide-and-conquer strategy. Random forest is applied to classify the training LR-HR patch pairs into a number of classes. Within every class, a simple linear regression model is used to model the relationship between the LR image patches and their corresponding HR image patches. Experimental results show that the proposed SRRF method can generate the state-of-the-art super-resolved images with near real-time performance.

Image super-resolution via weighted random forest

This paper proposes a novel learning-based image super-resolution via a weighted random forest model (SWRF). The proposed method uses the LR-HR training data to train a random forest model. The underlying idea of this approach is to use several decision trees to classify the training data based on a simple splitting threshold value at each class. A linear regression model is learnt to map the relationship between LR and HR patches. During the up-sampling process, to obtain a more robust super-resolved HR image, instead of averaging the linear regression models from different trees, a biased weighting vector is learnt to adaptively super-resolve the LR image. Furthermore, we improve this proposed image super-resolution method via a weighted random forest model with rotation (SWRF-f) to further improve the super-resolution quality. Sufficient experimental results prove that the proposed approach can achieve the state-of-the-art super-resolution performance with reduced computation time.

SRHRF+: Self-Example Enhanced Single Image Super-Resolution Using Hierarchical Random Forests

Example-based single image super-resolution (SISR) methods use external training datasets and have recently attracted a lot of interest. Self-example based SISR methods exploit redundant non-local self-similar patterns in natural images and because of that are more able to adapt to the image at hand to generate high quality super-resolved images. In this paper, we propose to combine the advantages of example-based SISR and self-example based SISR. A novel hierarchical random forests based super-resolution (SRHRF) method is proposed to learn statistical priors from external training images. Each layer of random forests reduce the estimation error due to variance by aggregating prediction models from multiple decision trees. The hierarchical structure further boosts the performance by pushing the estimation error due to bias towards zero. In order to further adaptively improve the super-resolved image, a self-example random forests (SERF) is learned from an image pyramid pair constructed from the down-sampled SRHRF generated result. Extensive numerical results show that the SRHRF method enhanced using SERF (SRHRF+) achieves the state-of-the-art performance on natural images and yields substantially superior performance for image with rich self-similar patterns.

Image super-resolution via hybrid NEDI and wavelet-based scheme

This paper proposes to make super-resolution for low-resolution image via a hybrid scheme making use of the wavelet domain processing and the New Edge-Directed Interpolation (NEDI). The proposed method combines the accurate low frequency information obtained from the wavelet transform and phase-free high frequency information predicted from the Shift-Free NEDI (SF-NEDI). The underlying idea of this approach is to study the pixel shift caused by the wavelet transform and to fix this problem when using the SF-NEDI to enlarge image, such that more accurate high frequency information can be extracted from the enlarged image. By using the framework of wavelet transform, the proposed approach uses the original low-resolution image and high frequency information from the SF-NEDI to realize image super-resolution. Extensive experimental results show that the proposed hybrid approach can achieve about 0.7 dB improvement in peak signal-to-noise ratio over the Wavelet Zero-padding and 1.35 dB over the SF-NEDI.

Fast image interpolation with decision tree

This paper proposes a fast image interpolation method using decision tree. This new fast image interpolation with decision tree (FIDT) method can achieve state-of-the-art image interpolation performance and requires only 10% computational time of the soft adaptive interpolation (SAI) method. During training, the proposed method recursively divides the training data at a non-leaf node into two child nodes according to the binary test which can maximize the information gain of a division. At the end, for each of the leaf node, a linear regression model is learned according to the training data at that leaf node. In the image interpolation phase, input image patches are passed into the learned decision tree. According to the stored binary test at each non-leaf node, each input image patch will be classified into its left or right child node until a leaf node is reached. The high-resolution image patch of the input image patch can then be predicted efficiently using the learned linear regression model at the leaf node.

Sparse signal recovery using structured total maximum likelihood

In this paper, we consider the sparse signal recovery problem when the dictionary is a Fourier frame. Based on the annihilation relation, the sparse signal recovery from noisy observations is posed as a structured total maximum likelihood (STML) problem. The recent structured total least squares (STLS) approach for finite rate of innovation signal recovery can be viewed as a particular version of our method. We transform the STML problem which has an additional logdet term into a form similar to the STLS problem. It can be effectively tackled using an iterative quadratic maximum likelihood like algorithm. From simulation results, our proposed STML approach outperforms the STLS based algorithm and the state-of-the-art sparse recovery algorithms.

ProSparse extension: Prony's based sparse pattern recovery with extended dictionaries

ProSparse is a Pronys based method that solves the sparse representation problem of signals in the union of Fourier and canonical bases. By exploiting the structure of the dictionary, ProSparse is able to reconstruct sparse signals beyond the recovery bound of Basis Pursuit. We generalize this framework for a broader class of dictionaries which are still formed from the union of two bases. The proposed algorithm achieves perfect reconstruction over a lower sparsity level than Basis Pursuit in noiseless cases. In the presence of noise, we extend the ProSparse Denoise algorithm to the generalized dictionaries by considering their intrinsic structure. The original ProSparse can be viewed as a special case of our proposed algorithm. From simulation results, our approach outperforms state-of-the-art algorithms.

Hybrid DCT-Wiener-Based interpolation using dual MMSE estimator scheme

Hybrid DCT-Wiener-Based interpolation scheme using the learnt Wiener filter can significantly improve both objective and subjective performance by learning a suitable Wiener filter to fit the hybrid scheme with a good mix of spatial and transform domain process. Using the adaptive k-NN MMSE estimation for each block achieves extraordinary up-sampling results. However, it needs a large database and relatively long processing time. In this paper, we investigate using multiple learnt Wiener filters and combine the information from both the external training images and the original low-resolution image. The proposed dual MMSE estimators adaptively resolve the problem of one general learnt Wiener filter and use less computation time compared with that of the k-NN MMSE estimation. Experimental results show that the proposed dual MMSE estimators achieve around 1dB PSNR improvement compared to the original hybrid DCT-Wiener-Based scheme and provide more natural visual quality.