Kim-Hui Yap

A Nonlinear Least Square Technique for Simultaneous Image Registration and Super-Resolution

This paper proposes a new algorithm to integrate image registration into image super-resolution (SR). Image SR is a process to reconstruct a high-resolution (HR) image by fusing multiple low-resolution (LR) images. A critical step in image SR is accurate registration of the LR images or, in other words, effective estimation of motion parameters. Conventional SR algorithms assume either the estimated motion parameters by existing registration methods to be error-free or the motion parameters are known a priori. This assumption, however, is impractical in many applications, as most existing registration algorithms still experience various degrees of errors, and the motion parameters among the LR images are generally unknown a priori. In view of this, this paper presents a new framework that performs simultaneous image registration and HR image reconstruction. As opposed to other current methods that treat image registration and HR reconstruction as disjoint processes, the new framework enables image registration and HR reconstruction to be estimated simultaneously and improved progressively. Further, unlike most algorithms that focus on the translational motion model, the proposed method adopts a more generic motion model that includes both translation as well as rotation. An iterative scheme is developed to solve the arising nonlinear least squares problem. Experimental results show that the proposed method is effective in performing image registration and SR for simulated as well as real-life images.

A soft MAP framework for blind super-resolution image reconstruction

This paper proposes a new algorithm to address blind image super-resolution (SR) by fusing multiple low-resolution (LR) blurred images to render a high-resolution (HR) image. Conventional SR image reconstruction algorithms assume the blurring occurred during the image formation process to be either negligible or can be characterized fully a priori. This assumption, however, is impractical as it is often difficult to eliminate the blurring completely in some applications or to know the blurring function completely a priori. In view of this, we present a new soft maximum a posteriori (MAP) estimation framework to perform joint blur identification and HR image reconstruction. The proposed method incorporates a soft blur prior that estimates the relevance of the best-fit parametric blur model, and induces reinforcement learning towards it. An iterative scheme based on alternating minimization is developed to estimate the blur and the HR image progressively. Experimental results show that the new method is effective in performing blind SR image reconstruction where there is limited information about the blurring function.

Fuzzy SVM for content-based image retrieval: a pseudo-label support vector machine framework

Conventional relevance feedback in content-based image retrieval (CBIR) systems uses only the labeled images for learning. Image labeling, however, is a time-consuming task and users are often unwilling to label too many images during the feedback process. This gives rise to the small sample problem where learning from a small number of training samples restricts the retrieval performance. To address this problem, we propose a technique based on the concept of pseudo-labeling in order to enlarge the training data set. As the name implies, a pseudo-labeled image is an image not labeled explicitly by the users, but estimated using a fuzzy rule. Therefore, it contains a certain degree of uncertainty or fuzziness in its class information. Fuzzy support vector machine (FSVM), an extended version of SVM, takes into account the fuzzy nature of some training samples during its training. In order to exploit the advantages of pseudo-labeling, active learning and the structure of FSVM, we develop a unified framework called pseudo-label fuzzy support vector machine (PLFSVM) to perform content-based image retrieval. Experimental results based on a database of 10,000 images demonstrate the effectiveness of the proposed method

A Comparative Study of Mobile-Based Landmark Recognition Techniques

Mobile-based landmark recognition is becoming increasingly appealing due to the proliferation of mobile devices coupled with improving processing techniques, imaging capability, and networking infrastructure. This article provides a general overview of existing mobile-based and nonmobile-based landmark recognition systems and their differences. We discuss content and context analysis and compare landmark classification methods. We also present the experimental results of our own mobile landmark recognition evaluations based on content analysis, context analysis, and integrated content-context analysis.

A soft double regularization approach to parametric blind image deconvolution

This paper proposes a blind image deconvolution scheme based on soft integration of parametric blur structures. Conventional blind image deconvolution methods encounter a difficult dilemma of either imposing stringent and inflexible preconditions on the problem formulation or experiencing poor restoration results due to lack of information. This paper attempts to address this issue by assessing the relevance of parametric blur information, and incorporating the knowledge into the parametric double regularization (PDR) scheme. The PDR method assumes that the actual blur satisfies up to a certain degree of parametric structure, as there are many well-known parametric blurs in practical applications. Further, it can be tailored flexibly to include other blur types if some prior parametric knowledge of the blur is available. A manifold soft parametric modeling technique is proposed to generate the blur manifolds, and estimate the fuzzy blur structure. The PDR scheme involves the development of the meaningful cost function, the estimation of blur support and structure, and the optimization of the cost function. Experimental results show that it is effective in restoring degraded images under different environments.

Intelligent multimedia processing with soft computing

Human-Centered Computing for Image and Video Retrieval.- Vector Color Image Indexing and Retrieval within A Small-World Framework.- A Perceptual Subjectivity Notion in Interactive Content-Based Image Retrieval Systems.- A Scalable Bootstrapping Framework for Auto-Annotation of Large Image Collections.- Moderate Vocabulary Visual Concept Detection for the TRECVID 2002.- Automatic Visual Concept Training Using Imperfect Cross-Modality Information.- Audio-Visual Event Recognition with Application in Sports Video.- Fuzzy Logic Methods for Video Shot Boundary Detection and Classification.- Rate-Distortion Optimal Video Summarization and Coding.- Video Compression by Neural Networks.- Knowledge Extraction in Stereo Video Sequences Using Adaptive Neural Networks.- An Efficient Genetic Algorithm for Small Search Range Problems and Its Applications.- Manifold Learning and Applications in Recognition.- Face Recognition Using Discrete Cosine Transform and RBF Neural Networks.- Probabilistic Reasoning for Closed-Room People Monitoring.- Human-Machine Communication by Audio-Visual Integration.- Probabilistic Fusion of Sorted Score Sequences for Robust Speaker Verification.- Adaptive Noise Cancellation Using Online Self-Enhanced Fuzzy Filters with Applications to Multimedia Processing.- Image Denoising Using Stochastic Chaotic Simulated Annealing.- Soft Computation of Numerical Solutions to Differential Equations in EEG Analysis.- Providing Common Time and Space in Distributed AV-Sensor Networks by Self-Calibration.

A recursive soft-decision approach to blind image deconvolution

This paper presents a new approach to blind image deconvolution based on soft-decision blur identification and hierarchical neural networks. Traditional blind algorithms require a hard-decision on whether the blur satisfies a parametric form before their formulations. As the blurring function is usually unknown a priori, this precondition inhibits the incorporation of parametric blur knowledge domain into the restoration schemes. The new technique addresses this difficulty by providing a continual soft-decision blur adaptation with respect to the best-fit parametric structure throughout deconvolution. The approach integrates the knowledge of well-known blur models without compromising its flexibility in restoring images degraded by nonstandard blurs. An optimization scheme is developed where a new cost function is projected and minimized with respect to the image and blur domains. A nested neural network, called the hierarchical cluster model is employed to provide an adaptive, perception-based restoration. Its sparse synaptic connections are instrumental in reducing the computational cost of restoration. Conjugate gradient optimization is adopted to identify the blur due to its computational efficiency. The approach is shown experimentally to be effective in restoring images degraded by different blurs.

Discriminative Soft Bag-of-Visual Phrase for Mobile Landmark Recognition

This paper proposes a new bag-of-visual phrase (BoP) approach for mobile landmark recognition based on discriminative learning of category-dependent visual phrases. Many previous landmark recognition works adopt a bag-of-words (BoW) method which ignores the co-occurrence relationship between neighboring visual words in an image. Although some works that focus on visual phrase learning have appeared, they mainly construct a generalized phrase dictionary from all categories for recognition, which lacks descriptive capability for a specific category. Another shortcoming of these works is the hard assignment of numerous feature sets to a limited number of phrases, which causes some useful feature sets to be discarded, and yields information loss. In view of this, this paper presents a discriminative soft BoP approach for mobile landmark recognition. The candidate phrases defined as adjacent pairwise codewords are first generated for each category. The important candidates are then selected through a proposed discriminative visual phrase (DVP) selection approach to form the BoP dictionary. Finally, a soft encoding method is developed to quantize each image into a BoP histogram. The context information such as location and direction captured by mobile devices is also integrated with the proposed BoP-based content analysis for landmark recognition. Experimental results on two datasets show that the proposed method is effective in mobile landmark recognition.

A computational reinforced learning scheme to blind image deconvolution

This paper presents a new approach to adaptive blind image deconvolution based on computational reinforced learning in an attractor-embedded solution space. The new technique develops an evolutionary strategy that generates the improved blur and image populations progressively. A dynamic attractor space is constructed by integrating the knowledge domain of the blur structures into the algorithm. The attractors are predicted using a maximum a posteriori estimator and their relevance is evaluated with respect to the computed blurs. We develop a novel reinforced mutation scheme that combines stochastic search and pattern acquisition throughout the blur identification. It enhances the algorithmic convergence and reduces the computational cost significantly. The new technique is robust in alleviating the constraints and difficulties encountered by most conventional methods. Experimental results show that the new algorithm is effective in restoring the degraded images and identifying the blurs.

Discriminative BoW Framework for Mobile Landmark Recognition

This paper proposes a new soft bag-of-words (BoW) method for mobile landmark recognition based on discriminative learning of image patches. Conventional BoW methods often consider the patches/regions in the images as equally important for learning. Amongst the few existing works that consider the discriminative information of the patches, they mainly focus on selecting the representative patches for training, and discard the others. This binary hard selection approach results in underutilization of the information available, as some discarded patches may still contain useful discriminative information. Further, not all the selected patches will contribute equally to the learning process. In view of this, this paper presents a new discriminative soft BoW approach for mobile landmark recognition. The main contribution of the method is that the representative and discriminative information of the landmark is learned at three levels: patches, images, and codewords. The patch discriminative information for each landmark is first learned and incorporated through vector quantization to generate soft BoW histograms. Coupled with the learned representative information of the images and codewords, these histograms are used to train an ensemble of classifiers using fuzzy support vector machine. Experimental results on two different datasets show that the proposed method is effective in mobile landmark recognition.