Young-Min Jang

Human intention recognition based on eyeball movement pattern and pupil size variation

To develop an efficient nonverbal human computer interaction system it is important to interpret the users implicit intention, which is vague. According to cognitive visuo-motor theory, the human eye movements are a rich source of information about the human intention and behavior. According to Beattys study, a task-evoked pupillary response is a consistent index of the human cognitive load and attention. In this paper, we propose a novel approach for a humans implicit intention recognition based on the eyeball movement pattern and pupil size variation. Based on the Bernards research, we classify the humans implicit intention during a visual stimulus as informational and navigational intent. In the present study, the navigational intent refers to the humans idea to find some interesting objects in a visual input without a particular goal while the informational intent refers to the humans aspiration to find a particular object of interest. The proposed model utilizes the salient features of the eye such as fixation length, fixation count and pupil size variation as the inputs to classify the humans implicit intention. The experimental results show that the proposed model can achieve plausible recognition performance.

Affective saliency map considering psychological distance

This paper proposes a new affective saliency map (SM) model considering psychological distance as well as the pop-out property based on relative spatial distribution of the primitive visual features such as intensity, edge, color, and orientation. By reflecting congruency between the spatial distance caused by spatial proximity and distal in a visual scene and psychological distance caused by the way people think about visual stimuli, the proposed SM model can produce more human-like visual selective attention than a conventional SM model based on primary visual perception. In the proposed model, a psychological distance caused by a social distance, in which a proximal entity such as friend becomes more attractive when it is located near but a distal entity such as enemy becomes more attractive when it is located far from an observer, is considered. In the experiments, two types of visual stimuli are considered, mono-stimuli and stereo-stimuli. In the case of mono-stimuli, the visual stimuli on a picture with psychological depth cues were considered. Instead, in the case of stereo-stimuli, depth perception is also considered for obtaining real spatial distance of visual target in a visual scene. In order to verify the proposed affective SM model, an eye tracking system was used to measure the visual scan path and fixation time on a specific local area while monitoring the visual scenes by human subjects. Experimental results show that the proposed model can generate plausible visual selective attention properly reflecting both psychological distance and primitive visual stimuli inducing pop-out bottom-up features.

Human implicit intent transition detection based on pupillary analysis

Interpretation of human implicit intention is crucial in the development of an efficient nonverbal human computer interaction system. According to cognitive visuo-motor theory, the human eye movements and pupillary responses are rich source of information about the human intention and behavior. It has been observed that under conditions of constant illumination and accommodation, pupil size varies systematically in relation to a variety of physiological and psychological factors, such as level of mental effort. It is well known that pupillary responses could be used to measure the differences in cognitive load under various tasks. In this paper, we try to detect the transition between the different human implicit intents based on the pupil state analysis. In real-world environment, the pupillary response can be influenced by various external factors like intensity and size of the image. To overcome the influence of the external factors, we develop a robust baseline model. The proposed approach detects the transition of the humans implicit intent from navigational intent to informational intent and vice versa during a visual stimulus. The approach also detects the transition among the different states of the informational intent such as informational intent generation, informational intent maintenance and informational intent disappear.

Recognition of human's implicit intention based on an eyeball movement pattern analysis

We propose a new approach for a humans implicit intention recognition system based on an eyeball movement pattern analysis. In this paper, we present a comprehensive classification of humans implicit intention. Based on Bernards research, we define the Humans implicit intention as informational and navigational intent. The intent for navigational searching is to locate a particular interesting object in an input scene. The intent for informational searching is to locate interesting area concerning a particular topic in order to obtain information from a specific location. In the proposed model, eyeball movement pattern analysis is considered for classifying the two different types of implicit intention. The experimental results show that the proposed model generates plausible recognition performance using a fixation length and counts with a simple nearest neighborhood classifier.

Stereo Saliency Map Considering Affective Factors in a Dynamic Environment

We propose a new integrated saliency map model, which reflects more human-like visual attention mechanism. The proposed model considers not only the binocular stereopsis to construct a final attention area so that the closer attention area can be easily made to pop-out as in human binocular vision, based on the single eye alignment hypothesis, but also both static and dynamic features of an input scene. Moreover, the proposed saliency map model includes an affective computing process to skip an unwanted area and/or to pay attention to a desired area, mimicking the pulvinars function in the human preference and refusal mechanism in subsequent visual search processes. In addition, we show the effectiveness of using the symmetry feature implemented by a neural network and independent component analysis (ICA) filter to construct more object preferable attention model. The experimental results show that the proposed model can generate more plausible scan paths for natural input scenes.

Probing of human implicit intent based on eye movement and pupillary analysis for augmented cognition

The ultimate purpose of augmented cognition is to enhance human cognitive abilities, which are intrinsically limited. To enhance limited human cognitive abilities, we developed a human augmented cognition system that can offer appropriate information or services by actively responding to the users intention. This article mainly describes a framework for probing human implicit intentions for the purpose of augmented cognition. The type of user intention, either task‐free human implicit intention or task‐oriented human implicit intention, can be predicted based on fixation count, fixation length, and pupil size variation induced by eye response. Further, these features are used to detect the transition point between task‐free human implicit intention and task‐oriented human implicit intention. Maximum a Posteriori in Naïve Bayes classification model is used for selecting relevant query keywords to search and retrieve specific information from a personalized knowledge database. The experimental results show that the proposed human intention recognition and probing models are suitable for achieving the goal of augmented cognition.

A real-time personal authentication system based on incremental feature extraction and classification of audiovisual information

We propose a new approach to a real-time personal authentication system based on incrementally updated visual (face) and audio (voice) features of persons. The proposed system consists of real-time face detection, incremental audiovisual feature extraction, and incremental neural classifier model with long-term memory. The face detection part, a biologically motivated face-color preferable selective attention model first localizes face candidate regions in natural scenes, and then the Adaboost-based face detection identifies human faces from the localized face-candidate regions. The mel-frequency cepstral coefficient is used for vocal feature extraction of speakers. Moreover, incremental principal component analysis (IPCA) is used to reduce the dimensions of audiovisual features and to update them incrementally. The features extracted by IPCA is fed to the resource allocating network with long-term memory which learns facial and vocal features incrementally and recognizes faces in real time. Experimental results show that the proposed system can enhance the test performance incrementally without serious forgetting. In addition, a multi-modal (facial and vocal) feature effectively increases the robustness of the personal authentication system in noisy environments.

Biologically Motivated Face Selective Attention Model

In this paper, we propose a face selective attention model, which is based on biologically inspired visual selective attention for human faces. We consider the radial frequency information and skin color filter to localize a candidate region of human face, which is to reflect the roles of the V4 and the infero-temporal (IT) cells. The ellipse matching based on symmetry axis is applied to check whether the candidate region contain a face contour feature. Finally, face detection is conducted by face form perception model implemented by an auto-associative multi-layer perceptron (AAMLP) that mimics the roles of faces selective cells in IT area. Based on both the face-color preferable attention and face-form perception mechanism, the proposed model shows plausible performance for localizing face candidates in real time.

Driver's lane-change intent identification based on pupillary variation

In this paper, we propose a model to identify drivers implicit intent based on eye movement analysis which is suitable for intelligent driver assistance system (IDAS). We use a lane-change intent-prediction system based on the human pupil size variation. Using the eye movement data as the input features, a discriminative classifier is trained to identify the probable lane-change maneuver at a particular point during the driving. In this paper we present the automated detection and recognition of lane-change intent based on drivers pupillary variation. In the proposed method pupil size variation features are extracted using a glass-type eye-tracker.

Probabilistic human intention modeling for cognitive augmentation

The aim of cognitive augmentation is to expand the intrinsically limited humans cognitive abilities caused by cognitive impairment or disability. In order to assist the humans limited cognitive ability, we are trying to develop a human augmented cognition system that aims to provide the appropriate information actively corresponding to what user intents to do. In this paper, we mainly address the probabilistic human intention modeling for cognitive augmentation, and its overall process. The types of implicit intention such as navigational and informational intention can be predicted by using fixation count and length induced by eyeball movement. Also, the gradient of pupil size variation is used to detect the transition point between navigational intent and the informational intent. A Naïve Bayes classifier is used as a tool for the extraction of query keywords to search and retrieve specific information from personalized knowledge database according to the successive series of attended objects according to a specific informational intent in a situation. The experimental results show that the probabilistic human intention model is suitable for achieving the ultimate purpose of the cognitive augmentation.