Gustavo B. Borba

Using visual attention to extract regions of interest in the context of image retrieval

Recent research on computational modeling of visual attention has demonstrated that a bottom-up approach to identifying salient regions within an image can be applied to diverse and practical problems for which conventional machine vision techniques have not succeeded in producing robust solutions. This paper proposes a new method for extracting regions of interest (ROIs) from images using models of visual attention. It is presented in the context of improving content-based image retrieval (CBIR) solutions by implementing a biologically-motivated, unsupervised technique of grouping together images whose salient ROIs are perceptually similar. In this paper we focus on the process of extracting the salient regions of an image. The excellent results obtained with the proposed method have demonstrated that the ROIs of the images can be independently indexed for comparison against other regions on the basis of similarity for use in a CBIR solution.

An attention-driven model for grouping similar images with image retrieval applications

Recent work in the computational modeling of visual attention has demonstrated that a purely bottom-up approach to identifying salient regions within an image can be successfully applied to diverse and practical problems from target recognition to the placement of advertisement. This paper proposes an application of a combination of computational models of visual attention to the image retrieval problem. We demonstrate that certain shortcomings of existing content-based image retrieval solutions can be addressed by implementing a biologically motivated, unsupervised way of grouping together images whose salient regions of interest (ROIs) are perceptually similar regardless of the visual contents of other (less relevant) parts of the image. We propose a model in which only the salient regions of an image are encoded as ROIs whose features are then compared against previously seen ROIs and assigned cluster membership accordingly. Experimental results show that the proposed approach works well for several combinations of feature extraction techniques and clustering algorithms, suggesting a promising avenue for future improvements, such as the addition of a top-down component and the inclusion of a relevance feedback mechanism.

An unsupervised method for clustering images based on their salient regions of interest

We have developed a biologically-motivated, unsupervised way of grouping together images whose salient regions of interest (ROIs) are perceptually similar regardless of the visual contents of other (less relevant) parts of the image. In the implemented model cluster membership is assigned based on feature vectors extracted from salient ROIs. This paper focuses on the experimental evaluation of the proposed approach for several combinations of feature extraction techniques and unsupervised clustering algorithms. The results reported here show that this is a valid approach and encourage further research.

Extraction of Salient Regions of Interest Using Visual Attention Models

This paper presents a novel algorithm for extracting regions of interest (ROIs) from images in an unsupervised way. It relies on the information provided by two computational models of bottom-up visual attention, encoded in the form of the images salient points-of-attention (POAs) and areas-of-attention (AOAs). The proposed method combines these POAs and AOAs to generate binary masks that correspond to the ROIs within the image. First, each AOA is binarized through an adapted relaxation algorithm where the histogram entropy of the AOA measurement is the stop criterion of the iterative process. The AOAs are also smoothed with a Gaussian pyramid followed by interpolation. Next, the binary representation of the AOAs, the smoothed version of the AOAs, and the POAs are converted in a mask that covers the salient ROIs of the image. The proposed ROI extraction algorithm does not impose any constraints on the number or distribution of salient regions in the input image. Qualitative and quantitative results show that the proposed method performs very well in a wide range of images, whether natural or man-made, from simple images of objects against a homogeneous background to complex cluttered scenes.

A Study of the Effect of Illumination Conditions and Color Spaces on Skin Segmentation

This work aims at investigating the influence of luminance information and environment illumination on skin classification. We explore Bayesian approaches to perform automatic classification of human skin pixels on digital images, using color features as input. Two probabilistic skin color models were built on different color spaces (RGB, normalized RG, HSI, HS, YCbCr and CbCr) and tested in a task of automatic pixel classification into skin and non-skin. Analyses of classification performance were done by presenting an illumination controlled image database containing images acquired in four different illumination conditions (shadow, sun, incandescent and fluorescent lights) to these classifiers. Our experiments show that building probabilistic skin color models using the CbCr color space generally improves performance of the classifiers and that best performance is achieved in shadow illumination.

On the Potential of Incorporating Knowledge of Human Visual Attention into Cbir Systems

Content-based image retrieval (CBIR) systems have been actively investigated over the past decade. Several existing CBIR prototypes claim to be designed based on perceptual characteristics of the human visual system, but even those who do are far from recognizing that they could benefit further by incorporating ongoing research in vision science. This paper explores the inclusion of human visual perception knowledge into the design and implementation of CBIR systems. Particularly, it addresses the latest developments in computational modeling of human visual attention. This fresh way of revisiting concepts in CBIR based on the latest findings and open questions in vision science research has the potential to overcome some of the challenges faced by CBIR systems

Fast Saliency Detection Using Sparse Random Color Samples and Joint Upsampling

The human visual system employs a mechanism of visual attention, which selects only part of the incoming information for further processing. Through this mechanism, the brain avoids overloading its limited cognitive capacities. In computer vision, this task is usually accomplished through saliency detection, which outputs the regions of an image that are distinctive with respect to its surroundings. This ability is desirable in many technological applications, such as image compression, video quality assessment and content-based image retrieval. In this paper, a saliency detection method based on color distance with sparse random samples and joint upsampling is presented. This approach computes full-resolution saliency maps with short runtime by leveraging both edge-preserving smoothing and joint upsampling capabilities of the Fast Global Smoother. The proposed method is assessed through precision-recall curves, F-measure and average runtime on the MSRA1K dataset. Results show that the method is competitive with state-of-the-art algorithms in both saliency detection accuracy and runtime.

Combining 3D models with 2D infrared images for medical applications

Infrared images are very useful for providing physiological information, although the representation is two-dimensional. On the other hand, a 3D scanning system is able to generate precise 3D spatial models of the area under study. This paper presents a methodology for combining both imaging modalities into a single representation. The Structure from Motion (SfM) technique is used in order to find the correct infrared cameras positioning and rotations in the space. Then, those 2D infrared images generate a 3D SfM model. Following this stage, the SfM model is replaced by an accurate 3D model from a scanning system, which is wrapped around by the infrared images. The experiments performed with a volunteers face have shown that the proposed methodology successfully reconstruct a unique 3D surface model, which is able to deliver potential clinical applications.

A Forward-Looking User Interface for CBIR and CFIR Systems

This demonstration highlights the benefits that image retrieval systems can enjoy by use of a thoughtful interface. We present a live demonstration of PRISM, a new Web-based application that, through a simple set of natural actions, allows the user the ability to elicit sophisticated responses from easily-formed queries. Additionally, the system was designed with not only content-based, but also content-free image retrieval and semantic annotation in mind. The demonstration system uses an attention-based method for extracting and determining the relevance of salient regions of interest in an unsupervised way, but can be adapted to a variety of implementations

Generation of 3D thermal models for dentistry applications

There are a variety of medical imaging modalities available, although each modality focus into different aspects, for example: anatomical, physiological or geometrical information. This paper presents a new imaging modality (3D THERMO-SCAN) that combines anatomical computer tomography (CT) imaging slices, together with 2D infrared thermography images and 3D scanned shaped models of the area under study. Therefore, it is presented the 3D reconstructions involving a case study of a volunteer with bruxism. Some characteristics of bruxism are the hyperactivity of the chewing muscles, which changes the dynamics of microcirculation, also changing the correspondent skins temperature. The emphasis is to show the corresponding structures, such as jaw/mandibular region that will produce either decrease or increase in temperature, which are related to bruxism and the associated use of an occlusal splint, respectively.