Janet Hui-wen Hsiao

Two Fixations Suffice in Face Recognition

It is well known that there exist preferred landing positions for eye fixations in visual word recognition. However, the existence of preferred landing positions in face recognition is less well established. It is also unknown how many fixations are required to recognize a face. To investigate these questions, we recorded eye movements during face recognition. During an otherwise standard face-recognition task, subjects were allowed a variable number of fixations before the stimulus was masked. We found that optimal recognition performance is achieved with two fixations; performance does not improve with additional fixations. The distribution of the first fixation is just to the left of the center of the nose, and that of the second fixation is around the center of the nose. Thus, these appear to be the preferred landing positions for face recognition. Furthermore, the fixations made during face learning differ in location from those made during face recognition and are also more variable in duration; this suggests that different strategies are used for face learning and face recognition.

Not All Visual Expertise Is Holistic, but It May Be Leftist The Case of Chinese Character Recognition

We examined whether two purportedly face-specific effects, holistic processing and the left-side bias, can also be observed in expert-level processing of Chinese characters, which are logographic and share many properties with faces. Non-Chinese readers (novices) perceived these characters more holistically than Chinese readers (experts). Chinese readers had a better awareness of the components of characters, which were not clearly separable to novices. This finding suggests that holistic processing is not a marker of general visual expertise; rather, holistic processing depends on the features of the stimuli and the tasks typically performed on them. In contrast, results for the left-side bias were similar to those obtained in studies of face perception. Chinese readers exhibited a left-side bias in the perception of mirror-symmetric characters, whereas novices did not; this effect was also reflected in eye fixations. Thus, the left-side bias may be a marker of visual expertise.

Convergence of the visual field split: Hemispheric modeling of face and object recognition

Anatomical evidence shows that our visual field is initially split along the vertical midline and contralaterally projected to different hemispheres. It remains unclear at which processing stage the split information converges. In the current study, we applied the Double Filtering by Frequency (DFF) theory (Ivry & Robertson, 1998) to modeling the visual field split; the theory assumes a right-hemisphere/low-frequency bias. We compared three cognitive architectures with different timings of convergence and examined their cognitive plausibility to account for the left-side bias effect in face perception observed in human data. We show that the early convergence model failed to show the left-side bias effect. The modeling, hence, suggests that the convergence may take place at an intermediate or late stage, at least after information has been extracted/encoded separately in the two hemispheres, a fact that is often overlooked in computational modeling of cognitive processes. Comparative anatomical data suggest that this separate encoding process that results in differential frequency biases in the two hemispheres may be engaged from V1 up to the level of area V3a and V4v, and converge at least after the lateral occipital region. The left-side bias effect in our model was also observed in Greeble recognition; the modeling, hence, also provides testable predictions about whether the left-side bias effect may also be observed in (expertise-level) object recognition.

A TMS examination of semantic radical combinability effects in Chinese character recognition

The proposal of human foveal splitting assumes a vertical meridian split in the foveal representation and the consequent contralateral projection of information in the two hemifields to the two hemispheres and has been shown to have important implications for visual word recognition. According to this assumption, in Chinese character recognition, the two halves of a centrally fixated character may be initially projected to and processed in different hemispheres. Here, we describe a repetitive transcranial magnetic stimulation (rTMS) investigation of hemispheric processing in Chinese character recognition, through examining semantic radical combinability effects in a character semantic judgment task. The materials used were a dominant type of Chinese character which consists of a semantic radical on the left and a phonetic radical on the right. Thus, according to the split fovea assumption, the semantic and phonetic radicals are initially projected to and processed in the right hemisphere and the left hemisphere, respectively. We show that rTMS over the left occipital cortex impaired the facilitation of semantic radicals with large combinability, whereas right occipital rTMS did not. This interaction between stimulation site and radical combinability reveals a flexible division of labor between the hemispheres in Chinese character recognition, with each hemisphere responding optimally to the information in the contralateral visual hemifield to which it has direct access. The results are also consistent with the split fovea claim, suggesting functional foveal splitting as a universal processing constraint in reading.

Predicting an observer's task using multi-fixation pattern analysis

Since Yarbuss seminal work in 1965, vision scientists have argued that peoples eye movement patterns differ depending upon their task. This suggests that we may be able to infer a persons task (or mental state) from their eye movements alone. Recently, this was attempted by Greene et al. [2012] in a Yarbus-like replication study; however, they were unable to successfully predict the task given to their observer. We reanalyze their data, and show that by using more powerful algorithms it is possible to predict the observers task. We also used our algorithms to infer the image being viewed by an observer and their identity. More generally, we show how off-the-shelf algorithms from machine learning can be used to make inferences from an observers eye movements, using an approach we call Multi-Fixation Pattern Analysis (MFPA).

Hemispheric asymmetry in perception: A differential encoding account

Hemispheric asymmetry in the processing of local and global features has been argued to originate from differences in frequency filtering in the two hemispheres, with little neurophysiological support. Here we test the hypothesis that this asymmetry takes place at an encoding stage beyond the sensory level, due to asymmetries in anatomical connections within each hemisphere. We use two simple encoding networks with differential connection structures as models of differential encoding in the two hemispheres based on a hypothesized generalization of neuroanatomical evidence from the auditory modality to the visual modality: The connection structure between columns is more distal in the language areas of the left hemisphere and more local in the homotopic regions in the right hemisphere. We show that both processing differences and differential frequency filtering can arise naturally in this neurocomputational model with neuroanatomically inspired differences in connection structures within the two model hemispheres, suggesting that hemispheric asymmetry in the processing of local and global features may be due to hemispheric asymmetry in connection structure rather than in frequency tuning.

NIMBLE: A kernel density model of saccade-based visual memory

We present a Bayesian version of J. Lacroix, J. Murre, and E. Postmas (2006) Natural Input Memory (NIM) model of saccadic visual memory. Our model, which we call NIMBLE (NIM with Bayesian Likelihood Estimation), uses a cognitively plausible image sampling technique that provides a foveated representation of image patches. We conceive of these memorized image fragments as samples from image class distributions and model the memory of these fragments using kernel density estimation. Using these models, we derive class-conditional probabilities of new image fragments and combine individual fragment probabilities to classify images. Our Bayesian formulation of the model extends easily to handle multi-class problems. We validate our model by demonstrating human levels of performance on a face recognition memory task and high accuracy on multi-category face and object identification. We also use NIMBLE to examine the change in beliefs as more fixations are taken from an image. Using fixation data collected from human subjects, we directly compare the performance of NIMBLEs memory component to human performance, demonstrating that using human fixation locations allows NIMBLE to recognize familiar faces with only a single fixation.

An examination of semantic radical combinability effects with lateralized cues in Chinese character recognition

Auclair and Siéroff (2002) examined lateralized cuing effects in the identification of centrally presented letter strings and reported no cuing effects for short word stimuli. They argued for a redistribution of attention over the entire word for short familiar words. We explored cuing effects with Chinese phonetic compounds, which can be considered extreme examples of short words, in a character-level semantic judgment task. When the semantic radical position was placed on the left of the characters, strong radical combinability and semantic transparency effects were observed. There was also a significant interaction between cue position (left vs. right) and radical combinability: A left cue facilitated semantic judgment of characters with small radical combinability more than did a right cue. This behavior reflects the information profile of Chinese phonetic compounds. Semantic radicals with small combinability are more informative than those with large combinability in determining the meaning of the whole character; they therefore benefit more from a left than a right cue. A mechanism redistributing attention over the whole of the character was not in evidence at the level of semantic processing.

Perceptual Expertise Can Sensorimotor Experience Change Holistic Processing and Left-Side Bias?

Holistic processing and left-side bias are both behavioral markers of expert face recognition. By contrast, expert recognition of characters in Chinese orthography involves left-side bias but reduced holistic processing, although faces and Chinese characters share many visual properties. Here, we examined whether this reduction in holistic processing of Chinese characters can be better explained by writing experience than by reading experience. Compared with Chinese nonreaders, Chinese readers who had limited writing experience showed increased holistic processing, whereas Chinese readers who could write characters fluently showed reduced holistic processing. This result suggests that writing and sensorimotor experience can modulate holistic-processing effects and that the reduced holistic processing observed in expert Chinese readers may depend mostly on writing experience. However, both expert writers and writers with limited experience showed similarly stronger left-side bias than novices did in processing mirror-symmetric Chinese characters; left-side bias may therefore be a robust expertise marker for object recognition that is uninfluenced by sensorimotor experience.

The modulation of visual and task characteristics of a writing system on hemispheric lateralization in visual word recognition-a computational exploration.

Through computational modeling, here we examine whether visual and task characteristics of writing systems alone can account for lateralization differences in visual word recognition between different languages without assuming influence from left hemisphere (LH) lateralized language processes. We apply a hemispheric processing model of face recognition to visual word recognition; the model implements a theory of hemispheric asymmetry in perception that posits low spatial frequency biases in the right hemisphere and high spatial frequency (HSF) biases in the LH. We show two factors that can influence lateralization: (a) Visual similarity among words: The more similar the words in the lexicon look visually, the more HSF/LH processing is required to distinguish them, and (b) Requirement to decompose words into graphemes for grapheme-phoneme mapping: Alphabetic reading (involving grapheme-phoneme conversion) requires more HSF/LH processing than logographic reading (no grapheme-phoneme mapping). These factors may explain the difference in lateralization between English and Chinese orthographic processing.