Min-Chun Yang

A Self-Learning Approach to Single Image Super-Resolution

Learning-based approaches for image super-resolution (SR) have attracted the attention from researchers in the past few years. In this paper, we present a novel self-learning approach for SR. In our proposed framework, we advance support vector regression (SVR) with image sparse representation, which offers excellent generalization in modeling the relationship between images and their associated SR versions. Unlike most prior SR methods, our proposed framework does not require the collection of training low and high-resolution image data in advance, and we do not assume the reoccurrence (or self-similarity) of image patches within an image or across image scales. With theoretical supports of Bayes decision theory, we verify that our SR framework learns and selects the optimal SVR model when producing an SR image, which results in the minimum SR reconstruction error. We evaluate our method on a variety of images, and obtain very promising SR results. In most cases, our method quantitatively and qualitatively outperforms bicubic interpolation and state-of-the-art learning-based SR approaches.

Robust Texture Analysis Using Multi-Resolution Gray-Scale Invariant Features for Breast Sonographic Tumor Diagnosis

Computer-aided diagnosis (CAD) systems in gray-scale breast ultrasound images have the potential to reduce unnecessary biopsy of breast masses. The purpose of our study is to develop a robust CAD system based on the texture analysis. First, gray-scale invariant features are extracted from ultrasound images via multi-resolution ranklet transform. Thus, one can apply linear support vector machines (SVMs) on the resulting gray-level co-occurrence matrix (GLCM)-based texture features for discriminating the benign and malignant masses. To verify the effectiveness and robustness of the proposed texture analysis, breast ultrasound images obtained from three different platforms are evaluated based on cross-platform training/testing and leave-one-out cross-validation (LOO-CV) schemes. We compare our proposed features with those extracted by wavelet transform in terms of receiver operating characteristic (ROC) analysis. The AUC values derived from the area under the curve for the three databases via ranklet transform are 0.918 (95% confidence interval [CI], 0.848 to 0.961), 0.943 (95% CI, 0.906 to 0.968), and 0.934 (95% CI, 0.883 to 0.961), respectively, while those via wavelet transform are 0.847 (95% CI, 0.762 to 0.910), 0.922 (95% CI, 0.878 to 0.958), and 0.867 (95% CI, 0.798 to 0.914), respectively. Experiments with cross-platform training/testing scheme between each database reveal that the diagnostic performance of our texture analysis using ranklet transform is less sensitive to the sonographic ultrasound platforms. Also, we adopt several co-occurrence statistics in terms of quantization levels and orientations (i.e., descriptor settings) for computing the co-occurrence matrices with 0.632+ bootstrap estimators to verify the use of the proposed texture analysis. These experiments suggest that the texture analysis using multi-resolution gray-scale invariant features via ranklet transform is useful for designing a robust CAD system.

Context-Aware Single Image Rain Removal

Rain removal from a single image is one of the challenging image denoising problems. In this paper, we present a learning-based framework for single image rain removal, which focuses on the learning of context information from an input image, and thus the rain patterns present in it can be automatically identified and removed. We approach the single image rain removal problem as the integration of image decomposition and self-learning processes. More precisely, our method first performs context-constrained image segmentation on the input image, and we learn dictionaries for the high-frequency components in different context categories via sparse coding for reconstruction purposes. For image regions with rain streaks, dictionaries of distinct context categories will share common atoms which correspond to the rain patterns. By utilizing PCA and SVM classifiers on the learned dictionaries, our framework aims at automatically identifying the common rain patterns present in them, and thus we can remove rain streaks as particular high-frequency components from the input image. Different from prior works on rain removal from images/videos which require image priors or training image data from multiple frames, our proposed self-learning approach only requires the input image itself, which would save much pre-training effort. Experimental results demonstrate the subjective and objective visual quality improvement with our proposed method.

Domain Adaptive Self-Taught Learning for Heterogeneous Face Recognition

Recognizing image data across different domains has been a challenging task. For biometrics, heterogeneous face recognition (HFR) deals with recognition problems in which training/gallery images are collected in terms of one modality (e.g., photos), while test/probe images are observed in the other (e.g., sketches). In this paper, we present a domain adaptation approach for solving HFR problems. By utilizing external face images (i.e., those collected from the subjects not of interest) from both source and target domains, we propose a novel Domain-independent Component Analysis (DiCA) algorithm for deriving a common subspace for relating and representing cross-domain image data. In order to introduce improved representation ability, we further advance the self-taught learning strategy for learning a domain-independent dictionary in our DiCA subspace, which can be applied to both gallery and probe images of interest to improve representation and recognition. Different from some prior domain-adaptation approaches, we do not require the data correspondences (i.e., data pairs) when collecting external cross-domain image data, nor the label information is needed for learning the common feature space when associating different domains. Thus, our method is practical for real-world cross-domain classification problems. In our experiments, we consider sketch-to-photo and near-infrared (NIR) to visible spectrum (VIS) face recognition problems for evaluating the performance of our proposed approach.

Context-aware single image super-resolution using locality-constrained group sparse representation

We present a novel learning-based method for single image super-resolution (SR). Given a single input low-resolution (LR) image (and its image pyramid), we propose to learn context-specific image sparse representation, which aims at modeling the relationship between low and high-resolution image patch pairs of different context categories in terms of the learned dictionaries. To predict the SR image, we derive the context-specific sparse representation of each image patch in the LR input with additional locality and group sparsity constraints. While the locality constraint searches for the most similar image patches and uses the corresponding highresolution outputs for SR, the group sparsity constraint allows us to utilize the information from most relevant context categories for predicting the final SR output. Experimental results show the proposed method is able to quantitatively and qualitatively achieve state-of-the-art performance.

Query-Adaptive Multiple Instance Learning for Video Instance Retrieval

Given a query image containing the object of interest (OOI), we propose a novel learning framework for retrieving relevant frames from the input video sequence. While techniques based on object matching have been applied to solve this task, their performance would be typically limited due to the lack of capabilities in handling variations in visual appearances of the OOI across video frames. Our proposed framework can be viewed as a weakly supervised approach, which only requires a small number of (randomly selected) relevant and irrelevant frames from the input video for performing satisfactory retrieval performance. By utilizing frame-level label information of such video frames together with the query image, we propose a novel query-adaptive multiple instance learning algorithm, which exploits the visual appearance information of the OOI from the query and that of the aforementioned video frames. As a result, the derived learning model would exhibit additional discriminating abilities while retrieving relevant instances. Experiments on two real-world video data sets would confirm the effectiveness and robustness of our proposed approach.

Recognition at a long distance: Very low resolution face recognition and hallucination

In real-world video surveillance applications, one often needs to recognize face images from a very long distance. Such recognition tasks are very challenging, since such images are typically with very low resolution (VLR). However, if one simply downsamples high-resolution (HR) training images for recognizing the VLR test inputs, or if one directly upsamples the VLR inputs for matching the HR training data, the resulting recognition performance would not be satisfactory. In this paper, we propose a joint face hallucination and recognition approach based on sparse representation. Given a VLR input image, our method is able to synthesize its person-specific HR version with recognition guarantees. In our experiments, we consider two different face image datasets. Empirical results will support the use of our approach for both VLR face recognition. In addition, compared to state-of-the-art super-resolution (SR) methods, we will also show that our method results in improved quality for the recovered HR face images.

Computer-aided diagnosis of breast color elastography

Ultrasound has been an important imaging technique for detecting breast tumors. As opposed to the conventional B-mode image, the ultrasound elastography is a new technique for imaging the elasticity and applied to detect the stiffness of tissues. The red region of color elastography indicates the soft tissue and the blue one indicates the hard tissue, and the harder tissue usually is classified to malignancy. In this paper, we proposed a CAD system on elastography to measure whether this system is effective and accurate to classify the tumor into benign and malignant. According to the features of elasticity, the color elastography was transferred to HSV color space and extracted meaningful features from hue images. Then the neural network was utilized in multiple features to distinguish tumors. In this experiment, there are 180 pathology-proven cases including 113 benign and 67 malignant cases used to examine the classification. The results of the proposed system showed an accuracy of 83.89%, a sensitivity of 85.07% and a specificity of 83.19%. Compared with the physicians diagnosis, an accuracy of 78.33%, a sensitivity of 53.73% and a specificity of 92.92%, the proposed CAD system had better performance. Moreover, the agreement of the proposed CAD system and the physicians diagnosis was calculated by kappa statistics, the kappa 0.54 indicated there is a fair agreement of observers.

Self-Learning of Edge-Preserving Single Image Super-Resolution via Contourlet Transform

We present a self-learning approach for single image super-resolution (SR), with the ability to preserve high frequency components such as edges in resulting high resolution (HR) images. Given a low-resolution (LR) input image, we construct its image pyramid and produce a super pixel dataset. By extracting context information from the super-pixels, we propose to deploy context-specific contour let transform on them in order to model the relationship (via support vector regression) between the input patches and their associated directional high-frequency responses. These learned models are applied to predict the SR output with satisfactory quality. Unlike prior learning-based SR methods, our approach advances a self-learning technique and does not require the self similarity of image patches within or across image scales. More importantly, we do not need to collect training LR/HR image data in advance and only require a single LR input image. Empirical results verify the effectiveness of our approach, which quantitatively and qualitatively outperforms existing interpolation or learning-based SR methods.