Richard W. Harvey

Extraction of visual features for lipreading

The multimodal nature of speech is often ignored in human-computer interaction, but lip deformations and other body motion, such as those of the head, convey additional information. We integrate speech cues from many sources and this improves intelligibility, especially when the acoustic signal is degraded. The paper shows how this additional, often complementary, visual speech information can be used for speech recognition. Three methods for parameterizing lip image sequences for recognition using hidden Markov models are compared. Two of these are top-down approaches that fit a model of the inner and outer lip contours and derive lipreading features from a principal component analysis of shape or shape and appearance, respectively. The third, bottom-up, method uses a nonlinear scale-space analysis to form features directly from the pixel intensity. All methods are compared on a multitalker visual speech recognition task of isolated letters.

Non-retrieval: Blocking Pornographic Images

We extend earlier work on detecting pornographic images. Our focus is on the classification stage and we give new results for a variety of classical and modern classifiers. We find the artificial neural network offers a statistically significant improvement. In all cases the error rate is too high unless deployed sensitively so we show how such a system may be built into a commercial environment.

Morphological scale-space preserving transforms in many dimensions

The theory of an image decomposition that we refer to as a sieve is developed for images defined in any finite number of dimensions. The decomposition has many desirable properties in- cluding the preservation of scale-space causality and the localiza- tion of sharp-edged objects in the transformation domain. The de- composition has the additional properties of manipulability, which means that it is easy to construct pattern recognition systems, and scale-calibration, which means that it may be used for accurate measurement.

Scale-space from nonlinear filters

Decomposition by extrema is put into the context of linear vision systems and scale-space. It is proved that discrete one-dimensional, M- and N-sieves neither introduce new edges as the scale increases nor create new extrema. They share this property with diffusion based filters. They are robust and preserve edges of large scale features.

Accommodating color blind computer users

Important visual information often disappears when color documents are viewed by color blind people. The algorithm introduced here maps colors using the World Wide Web Consortium evaluation criteria so that detail is preserved for color blind viewers, especially dichromats. The algorithm has four parts: 1) select a representative set of colors from the source document; 2) compute target color distances using color and brightness differences; 3) solve an optimization step that preserves the target distances for a particular class of color blind viewer; and 4) interpolate the mapped colors across the remaining colors in the document. We demonstrate the efficacy of our method using simulations and critique our method in the context of earlier work.

An interface to support color blind computer users

A new method for adapting digital images so that they are suitable for color blind viewers is presented. In contrast to earlier automatic methods which formulate the problem of adapting images for color blind observers as one of optimization, we demonstrate how it is possible to allow a user to compute a very wide range of adaptations in reasonable time under the control of a single variable. We demonstrate how the algorithm can be delivered as an adaptive technology via a simple interface, and evaluate the efficacy of our method using psychovisual experiments with simulated color blind users and a standard color vision test.

Theoretical and experimental comparison of different approaches for color texture classification

Color texture classification has been an area of intensive research activity. From the very onset, approaches to combining color and texture have been the subject of much discussion, and in particular, whether they should be considered joint or separately. We present a comprehensive comparison of the most prominent approaches both from a theoretical and experimental standpoint. The main contributions of our work are: (i) the establishment of a generic and extensible framework to classify methods for color texture classification on a mathematical basis, and (ii) a theoretical and experimental comparison of the most salient existing methods. Starting from an extensive set of experiments based on the Outex dataset, we highlight those texture descriptors that provide good accuracy along with low dimensionality. The results suggest that separate color and texture processing is the best practice when one seeks for optimal compromise between accuracy and limited number of features. We believe that our work may serve as a guide for those who need to choose the appropriate method for a specific application, as well as a basis for the development of new methods.

Robust facial feature tracking using selected multi-resolution linear predictors

This paper proposes a learnt data-driven approach for accurate, real-time tracking of facial features using only intensity information. Constraints such as a-priori shape models or temporal models for dynamics are not required or used. Tracking facial features simply becomes the independent tracking of a set of points on the face. This allows us to cope with facial configurations not present in the training data. Tracking is achieved via linear predictors which provide a fast and effective method for mapping pixel-level information to tracked feature position displacements. To improve on this, a novel and robust biased linear predictor is proposed in this paper. Multiple linear predictors are grouped into a rigid flock to increase robustness. To further improve tracking accuracy, a novel probabilistic selection method is used to identify relevant visual areas for tracking a feature point. These selected flocks are then combined into a hierarchical multi-resolution LP model. Experimental results also show that this method performs more robustly and accurately than AAMs, without any a priori shape information and with minimal training examples.

The Segmentation of Images via Scale-Space Trees

A useful representation of an image would be an object tree in which nodes represent objects, or parts of objects, and which includes at least one node that, together with its children, represents each object: a grandmothernode. It is shown that scale-trees, obtained from greyscale images, approximate such a tree. It is then shown how they may be modified using other attributes to more closely become object trees. The result is a data structure that provides “handles” for every element of the image that can be used for manipulating the image. This segmentation has potential for object recognition.

A Comparison of Active Shape Model and Scale Decomposition Based Features for Visual Speech Recognition

Two quite different strategies for characterising mouth shapes for visual speech recognition (lipreading) are compared. The first strategy extracts the parameters required to fit an active shape model (ASM) to the outline of the lips. The second uses a feature derived from a one-dimensional multiscale spatial analysis (MSA) of the mouth region using a new processor derived from mathematical morphology and median filtering. With multispeaker trials, using image data only, the accuracy is 45% using MSA and 19% using ASM on a letters database. A digits database is simpler with accuracies of 77% and 77% respectively. These scores are significant since separate work has demonstrated that even quite low recognition accuracies in the vision channel can be combined with the audio system to give improved composite performance [16].