Umesh Rajashekar

DOVES: a database of visual eye movements.

DOVES, a database of visual eye movements, is a set of eye movements collected from 29 human observers as they viewed 101 natural calibrated images. Recorded using a high-precision dual-Purkinje eye tracker, the database consists of around 30 000 fixation points, and is believed to be the first large-scale database of eye movements to be made available to the vision research community. The database, along with MATLAB functions for its use, may be downloaded freely from http://live.ece.utexas.edu/research/doves, and used without restriction for educational and research purposes, providing that this paper is cited in any published work. This paper documents the acquisition procedure, summarises common eye movement statistics, and highlights numerous research topics for which DOVES may be used.

The SIVA Demonstration Gallery for signal, image, and video processing education

The techniques of digital signal processing (DSP) and digital image processing (DIP) have found a myriad of applications in diverse fields of scientific, commercial, and technical endeavor. DSP and DIP education needs to cater to a wide spectrum of people from different educational backgrounds. This paper describes tools and techniques that facilitate a gentle introduction to fascinating concepts in signal and image processing. Novel LabVIEW- and MATLAB-based demonstrations are presented, which, when supplemented with Web-based class lectures, help to illustrate the power and beauty of signal and image-processing algorithms. Equipped with informative visualizations and a user-friendly interface, these modules are currently being used effectively in a classroom environment for teaching DSP and DIP at the University of Texas at Austin (UT-Austin). Most demonstrations use audio and image signals to give students a flavor of real-world applications of signal and image processing. This paper is also intended to provide a library of more than 50 visualization modules that accentuate the intuitive aspects of DSP algorithms as a free didactic tool to the broad signal and image-processing community.

Quantifying color image distortions based on adaptive spatio-chromatic signal decompositions

We describe a framework for quantifying color image distortion based on an adaptive signal decomposition. Specifically, local blocks of the image error are decomposed using a set of spatio-chromatic basis functions that are adapted to the spatial and color structure of the original image. The adaptive functions are chosen to isolate specific distortions such as luminance, hue, and saturation changes. These adaptive basis functions are used to augment a generic orthonormal basis, and the overall distortion is computed from the weighted sum of the coefficients of the resulting overcomplete decomposition, with smaller weights chosen for the adaptive terms. A set of preliminary experiments show that the proposed distortion measure is consistent with human perception of color images subjected to a variety of different common distortions. The framework may be easily extended to include any form of continuous spatio-chromatic distortion.

Multiscale Denoising of Photographic Images

Publisher Summary Image noise can be quite noticeable, as in images captured by inexpensive cameras built into cellular telephones, or imperceptible, as in images captured by professional digital cameras. Stated simply, the goal of image denoising is to recover the true signal (or its best approximation) from these noisy acquired observations. All such methods rely on understanding and exploiting the differences between the properties of signal and noise. Formally, solutions to the denoising problem rely on three fundamental components: a signal model, a noise model, and finally a measure of signal fidelity (commonly known as the objective function) that is to be minimized. A simple strategy for denoising an image is to separate it into smooth and nonsmooth parts, or equivalently, low-frequency and high-frequency components. This decomposition can then be applied recursively to the lowpass component to generate a multiscale representation. The lower frequency subbands are smoother, and thus can be subsampled to allow a more efficient representation, generally known as a multiscale pyramid. The resulting collection of frequency subbands contains the exact same information as the input image, but as one shall see, it has been separated in such a way that it is more easily distinguished from noise.

Visual Memory for Fixated Regions of Natural Images Dissociates Attraction and Recognition

Recognition memory for fixated regions from briefly viewed full-screen natural images is examined. Low-level image statistics reveal that observers fixated, on average (pooled across images and observers), image regions that possessed greater visual saliency than non-fixated regions, a finding that is robust across multiple fixation indices. Recognition-memory performance indicates that, of the fixation loci tested, observers were adept at recognising those with a particular profile of image statistics; visual saliency was found to be attenuated for unrecognised loci, despite that all regions were freely fixated. Furthermore, although elevated luminance was the local image statistic found to discriminate least between human and random image locations, it was the greatest predictor of recognition-memory performance, demonstrating a dissociation between image features that draw fixations and those that support visual memory. An analysis of corresponding eye movements indicates that image regions fixated via short-distance saccades enjoyed better recognition-memory performance, alluding to a focal rather than ambient mode of processing. Recognised image regions were more likely to have originated from areas evaluated (a posteriori) to have higher fixation density, a numerical metric of local interest. Surprisingly, memory for image regions fixated later in the viewing period exhibited no recency advantage, despite (typically) also being longer in duration, a finding for which a number of explanations are posited.

Foveated Analysis and Selection of Visual Fixations in Natural Scenes

The ability to automatically detect visually interesting regions in images has practical applications in the design of active machine vision systems. Analysis of the statistics of image features at observers gaze can provide insights into the mechanisms of fixation selection in humans. Using a novel foveated analysis framework, in which features were analyzed at the spatial resolution at which they were perceived, we studied the statistics of four low-level local image features: luminance, contrast, center-surround outputs of luminance and contrast, and discovered that the image patches around human fixations had, on average, higher values of each of these features than the image patches selected at random. Center-surround contrast showed the greatest difference between human and random fixations, followed by contrast, center-surround luminance, and luminance. Using these measurements, we present a new algorithm that selects image regions as likely candidates for fixation. These regions are shown to correlate well with fixations recorded from observers.

A study of human recognition rates for foveola-sized image patches selected from initial and final fixations on calibrated natural images

Recent years have seen a resurgent interest in eye movements during natural scene viewing. Aspects of eye movements that are driven by low-level image properties are of particular interest due to their applicability to biologically motivated artificial vision and surveillance systems. In this paper, we report an experiment in which we recorded observers’ eye movements while they viewed calibrated greyscale images of natural scenes. Immediately after viewing each image, observers were shown a test patch and asked to indicate if they thought it was part of the image they had just seen. The test patch was either randomly selected from a different image from the same database or, unbeknownst to the observer, selected from either the first or last location fixated on the image just viewed. We find that several low-level image properties differed significantly relative to the observers’ ability to successfully designate each patch. We also find that the differences between patch statistics for first and last fixations are small compared to the differences between hit and miss responses. The goal of the paper was to, in a non-cognitive natural setting, measure the image properties that facilitate visual memory, additionally observing the role that temporal location (first or last fixation) of the test patch played. We propose that a memorability map of a complex natural scene may be constructed to represent the low-level memorability of local regions in a similar fashion to the familiar saliency map, which records bottom-up fixation attractors.

Statistical analysis and selection of visual fixations

xii

Performance evaluation of mail-scanning cameras

Letter-scanning cameras (LSCs) form the front- end im- aging systems for virtually all mail-scanning systems that are cur- rently used to automatically sort mail products. As with any vision- dependent technology, the quality of the images generated by the camera is fundamental to the overall performance of the system. We present novel techniques for objective evaluation of LSCs using comparative imaging—a technique that involves measuring the fi- delity of target images produced by a camera with reference to an image of the same target captured at very high quality. Such a framework provides a unique opportunity to directly quantify the cameras ability to capture real-world targets, such as handwritten and printed text. Noncomparative techniques were also used to measure properties such as the cameras modulation transfer func- tion, dynamic range, and signal-to-noise ratio. To simulate real- world imaging conditions, application-specific test samples were de- signed using actual mail product materials.

Contextually adaptive signal representation using conditional principal component analysis

The conventional method of generating a basis that is optimally adapted (in MSE) for representation of an ensemble of signals is principal component analysis (PCA). A more ambitious modern goal is the construction of bases that are adapted to individual signal instances. Here we develop a new framework for instance-adaptive signal representation by exploiting the fact that many real-world signals exhibit local self-similarity. Specifically, we decompose the signal into multiscale subbands, and then represent local blocks of each subband using basis functions that are linearly derived from the surrounding context. The linear mappings that generate these basis functions are learned sequentially, with each one optimized to account for as much variance as possible in the local blocks. We apply this methodology to learning a coarse-to-fine representation of images within a multi-scale basis, demonstrating that the adaptive basis can account for significantly more variance than a PCA basis of the same dimensionality.