Herman Martins Gomes

A computer vision model for visual-object-based attention and eye movements

This paper presents a new computational framework for modelling visual-object-based attention and attention-driven eye movements within an integrated system in a biologically inspired approach. Attention operates at multiple levels of visual selection by space, feature, object and group depending on the nature of targets and visual tasks. Attentional shifts and gaze shifts are constructed upon their common process circuits and control mechanisms but also separated from their different function roles, working together to fulfil flexible visual selection tasks in complicated visual environments. The framework integrates the important aspects of human visual attention and eye movements resulting in sophisticated performance in complicated natural scenes. The proposed approach aims at exploring a useful visual selection system for computer vision, especially for usage in cluttered natural visual environments.

Cross-modal responses in the primary visual cortex encode complex objects and correlate with tactile discrimination

Cortical areas that directly receive sensory inputs from the thalamus were long thought to be exclusively dedicated to a single modality, originating separate labeled lines. In the past decade, however, several independent lines of research have demonstrated cross-modal responses in primary sensory areas. To investigate whether these responses represent behaviorally relevant information, we carried out neuronal recordings in the primary somatosensory cortex (S1) and primary visual cortex (V1) of rats as they performed whisker-based tasks in the dark. During the free exploration of novel objects, V1 and S1 responses carried comparable amounts of information about object identity. During execution of an aperture tactile discrimination task, tactile recruitment was slower and less robust in V1 than in S1. However, V1 tactile responses correlated significantly with performance across sessions. Altogether, the results support the notion that primary sensory areas have a preference for a given modality but can engage in meaningful cross-modal processing depending on task demand.

Integral Local Binary Patterns: A Novel Approach Suitable for Texture-Based Object Detection Tasks

This work is concerned with the proposition and empirical evaluation of a new feature extraction approach that combines two existing image descriptors, Integral Histograms and Local Binary Patterns (LBP), to achieve a representation that exhibits relevant properties to object detection tasks (such as face detection): fast constant time processing, rotation, and scale invariance. This novel approach is called the Integral Local Binary Patterns (INTLBP), which is based on an existing method for calculating Integral Histograms from LBP images. This paper empirically demonstrates the properties of INTLBP in a scenario of texture extraction for face/non-face classification. Experiments have shown that the new representation added robustness to scale variations in the test images - the proposed approach achieved a mean classification rate 92% higher than the standard Rotation Invariant LBP approach, when testing over images with scales different from the ones used for training. Moreover, the INTLBP dramatically reduced the required processing time when searching patterns in a face detection task.

Learning and extracting primal-sketch features in a log-polar image representation

This paper presents a novel and more successful learning based approach to extracting low level features in a retina-like (log-polar) image representation. The low level features (edges, bars, blobs and ends) are based on Marrs primal sketch hypothesis for the human visual system. The feature extraction process used a neural network that learns examples of the features in a window of receptive fields of the image representation. An architecture designed to encode the features class, position, orientation and contrast has been proposed and tested. Success depended on the incorporation of a function to normalise the features orientation and a PCA pre-processing module to produce better separation in the feature space.

Learning-based versus model-based log-polar feature extraction operators: a comparative study

We compare two distinct primal sketch feature extraction operators: one based on neural network feature learning and the other based on mathematical models of the features. We tested both kinds of operator with a set of known, but previously untrained, synthetic features and, while varying their classification thresholds, measured the operators false acceptance and false rejection errors. Results have shown that the model-based approach is more unstable and unreliable than the learning-based approach, which presented better results with respect to the number of correctly classified features.

Structural Learning from Iconic Representations

This paper addresses the important problem of how to learn geometric relationships from sets of iconic (2-D) models obtained from a sequence of images. It assumes a vision system that operates by foveating at interesting regions in a scene, extracting a number of raw primal sketch-like image descriptions, and matching new regions to previously seen ones. A solution to the structure learning problem is presented in terms of a graph-based representation and algorithm. Vertices represent instances of an image neighbourhood found in the scenes. An edge represents a relationship between two neighbourhoods. Intra and inter model relationships are inferred by means of the cliques found in the graph, which leads to rigid geometric models inferred from the image evidence.

Investigation of local and global features for face detection

This work is concerned with the empirical evaluation of a set of local and global features under the context of frontal (including semi-profile) and full profile face classification. Integral LBP, Integral Histograms, PCA and Optimized Face Ratios features have been evaluated using SVM classifiers. A data set of about 14,000 face and 300,000 non face images has been used in the experiments. Face images were obtained from well known public face research databases, such as BioID, Color Feret, CMU PIE, among others. The PCA-SVM classifier presented best overall results for both frontal and full profile faces whereas the classifier based on Face Ratios presented the lowest classification rates. A weighted combination of all classifiers yielded high True Positive (TPR) and True Negative (TNR) rates: 91.7% and 100%, respectively, for the frontal face experiments; 99.59% and 99.62%, respectively, for the profile face experiments. These results indicate that the evaluated features are very suitable to the problem of face detection and that a simple classifier combination improves individual classifiers performance.

Remote Eye Tracking Systems: Technologies and Applications

Eye tracking is an active multidisciplinary research field, which has shown great progress in the last decades. Eye tracking is the process of monitoring eye movements in order to determine the point of gaze or to analyze motion patterns of an eye relative to the head or the environment. This process provides essential information to tasks such as face detection, usability testing, biometric identification, human behavior studies and human-computer interaction, among others. The term \textit{remote} is used whenever the eye tracker components do not have any physical contact with the users body. In general, highly accurate commercial eye trackers are expensive. However, nowadays it is possible to develop low-cost remote eye tracking systems with off-the-shelf hardware components. By this perspective, an overview of eye tracking technologies and applications is given in this paper, and the state-of-the-art work in the field is discussed as well. Moreover, hardware components and underlying computer vision processes are also presented in this paper.

Combining Multiple Image Features to Guide Automatic Portrait Cropping for Rendering Different Aspect Ratios

Nowadays there exists a large variety of aspect ratios used for both image rendering (e.g. print media, TV, cinema screens etc.) and image acquisition devices (e.g. still and video cameras, scanners etc.). In order to maintain the image’s original aspect ratio when adjusting for a different media, some level of cropping may be required, but the automatic zoom and crop method may not produce satisfactory results regarding image contents. This paper proposes an automatic method that analyses images and estimates the relevant content areas, avoiding distortions and main subject chopping. The analysis is performed by four feature extractors, each producing a grayscale image which indicates relevant image areas. The outputs of these extractors are then combined by means of a Genetic Algorithm (GA) optimization. Experiments involving a subjective evaluation of a set of automatically cropped images have shown that 77% of the 35 human observers considered images adjusted by the proposed approach better than or similar to the outputs of the automatic zoom and crop method.

Neural network classification of photogenic facial expressions based on fiducial points and Gabor features

This work reports a study about the use of Gabor coefficients and coordinates of fiducial (landmark) points to represent facial features and allow the discrimination between photogenic and non-photogenic facial images, using neural networks. Experiments have been performed using 416 images from the Cohn-Kanade AU-Coded Facial Expression Database [1]. In order to extract fiducial points and classify the expressions, a manual processing was performed. The facial expression classifications were obtained with the help of the Action Unit information available in the image database. Various combinations of features were tested and evaluated. The best results were obtained with a weighted sum of a neural network classifier using Gabor coefficients and another using only the fiducial points. These indicated that fiducial points are a very promising feature for the classification performed.