Michael E. Farmer

A wrapper-based approach to image segmentation and classification

The traditional processing flow of segmentation followed by classification in computer vision assumes that the segmentation is able to successfully extract the object of interest from the background image. It is extremely difficult to obtain a reliable segmentation without any prior knowledge about the object that is being extracted from the scene. This is further complicated by the lack of any clearly defined metrics for evaluating the quality of segmentation or for comparing segmentation algorithms. We propose a method of segmentation that addresses both of these issues, by using the object classification subsystem as an integral part of the segmentation. This will provide contextual information regarding the objects to be segmented, as well as allow us to use the probability of correct classification as a metric to determine the quality of the segmentation. We view traditional segmentation as a filter operating on the image that is independent of the classifier, much like the filter methods for feature selection. We propose a new paradigm for segmentation and classification that follows the wrapper methods of feature selection. Our method wraps the segmentation and classification together, and uses the classification accuracy as the metric to determine the best segmentation. By using shape as the classification feature, we are able to develop a segmentation algorithm that relaxes the requirement that the object of interest to be segmented must be homogeneous in some low-level image parameter, such as texture, color, or grayscale. This represents an improvement over other segmentation methods that have used classification information only to modify the segmenter parameters, since these algorithms still require an underlying homogeneity in some parameter space. Rather than considering our method as, yet, another segmentation algorithm, we propose that our wrapper method can be considered as an image segmentation framework, within which existing image segmentation algorithms may be executed. We show the performance of our proposed wrapper-based segmenter on real-world and complex images of automotive vehicle occupants for the purpose of recognizing infants on the passenger seat and disabling the vehicle airbag. This is an interesting application for testing the robustness of our approach, due to the complexity of the images, and, consequently, we believe the algorithm will be suitable for many other real-world applications.

Interacting multiple model (IMM) Kalman filters for robust high speed human motion tracking

Accurate and robust tracking of humans is of growing interest in the image processing and computer vision communities. The ability of a vision system to track the subjects and accurately, predict their future locations is critical to many surveillance and camera control applications. Further, an inference of the type of motion as well as to rapidly detect and switch between motion models is critical since in some applications the switching time between motion models can be extremely small. The interacting multiple model (IMM) Kalman filter provides a powerful framework for performing the tracking of both the motion as well as the shape of these subjects. The tracking system utilizes a simple geometric shape primitive such as an ellipse to define a bounding extent of the subject. The utility of the IMM paradigm for rapid model switching and behaviour detection is shown for a passenger airbag suppression system in an automobile. The simplicity, of the methods and the robustness of the underlying IMM filtering make the framework well suited for low-cost embedded real-time motion sequence analysis systems.

Large scale feature selection using modified random mutation hill climbing

Feature selection is a critical component of many pattern recognition applications. There are two distinct mechanisms for feature selection, namely the wrapper methods and the filter methods. The filter methods are generally considered inferior to the wrapper method, however wrapper methods are computationally more demanding than filter methods. One of the popular methods for wrapper-based feature selection is random mutation hill climbing. It performs a random search over the feature space to derive the optimal set of features. We would describe two enhancements to this algorithm, one that would improve its convergence time, and the other that would allow us to bias the results towards either higher accuracy or lower final feature space dimensionality. We would apply the algorithm to a real-world massive-scale feature selection problem involving the image classification problem associated with suppressing automobile airbags for children. We would provide classification results on an image database of nearly 4,000 images that indicate the advantages of the proposed method.

Application of Genetic Algorithms for Wrapper-based Image Segmentation and Classification

The traditional processing flow of segmentation followed by classification in computer vision assumes that the segmentation is able to successfully extract the object of interest. This is challenging without any prior knowledge about the object that is being extracted from the scene. We previously proposed a method of segmentation that uses the classification subsystem as an integral part of the segmentation, which provides contextual information regarding the objects to be segmented. Our approach integrated segmentation and classification in a manner analogous to wrapper methods in feature selection. We initially perform low-level segmentation to label the image as a set of non-overlapping blobs. We then use the wrapper framework to select the blobs that comprise the final segmentation based on the classification performance of the wrapper. In this paper, the process of combining the blobs and then evaluating these combinations is performed with a genetic algorithm. We show the performance of the genetic algorithm based wrapper segmentation on real-world complex images of automotive vehicle occupants, where our overall classification accuracy is roughly 88% and the resultant segmentations are extremely accurate.

Smart Automotive Airbags: Occupant Classification and Tracking

The introduction of airbags into automobiles has significantly improved the safety of the occupants. Unfortunately, airbags can also cause fatal injuries if the occupant is a child smaller (in weight) than a typical six-year-old. Between 1986 and 2001, 19 infants and 85 children were killed by airbags during relatively minor vehicle collisions. In addition to these infant and child deaths, there have also been seven adults killed by airbags due to their proximity to the airbag during deployment. In response to these deaths, the National Highway Transportation and Safety Administration has mandated that, starting in the 2006 model year, all automobiles be equipped with automatic airbag suppression. The suppression of the airbag based on the type of occupant can be framed as a two-class classification problem, while the suppression of the airbag based on the location of the occupant relative to the airbag can be framed as an occupant-tracking problem. This paper describes an integrated real-time vision-based occupant classification and tracking system using a single grayscale camera with commercially available processing hardware. The classification system has achieved a classification accuracy of approximately 98%. Likewise, the tracking system has demonstrated the ability to detect a dangerous proximity of the occupant relative to the airbag within only 7 ms

A Chaos Theoretic Analysis of Motion and Illumination in Video Sequences

Accurate and robust image motion detection has been of substantial interest in the image processing and computer vision communities. Unfortunately, no single motion detection algorithm has been universally superior, yet biological vision systems are adept at motion detection. Recent research in neural signals have shown biological neural systems are highly responsive to signals which appear to be chaotic in nature. In this paper, we exploit these biological results and hypothesize that motion in images may produce changes in pixel amplitudes that are reminiscent of chaotic dynamical systems. In particular, we demonstrate that the trajectories of pixel amplitudes in phase space due to motion result in chaos-like behavior. We likewise demonstrate that the effects of spatio-temporally varying illumination produces phase space trajectories of the pixel amplitudes which are clearly non-chaotic. We review the research tying chaotic behavior to the fractal characteristics of phase space trajectories, and we investigate multi-fractal measures which can be used to classify the pixels in an image stream based on their fractal behavior in phase space. We finally apply these measures to the task of motion detection and segmentation and show they are effective in identifying moving objects while ignoring spatio-temporally varying illumination changes.

The Effects of Motion and Spatio-temporal Non-uniform Illumination on Image-pair Joint Scattergrams

Accurate and robust image change detection and motion segmentation has been of substantial interest in the image processing and computer vision communities. Spatio-temporal illumination effects such as complex lighting effects involving moving light bands or moving shadow bands can confuse existing motion segmentation algorithms, which are based on analysing the greyscale image directly. While efforts have been made to improve the robustness of motion segmentation algorithms under varying illumination, the results to date are still not completely satisfactory. We propose using the joint scattergram between two images as the representational space for analyzing motion. These scattergrams have properties which can distinguish motion from complex illumination changes. We verify these properties of the scattergram on image datasets captured in the laboratory as well as outdoors and show its utility as a robust representational domain

Illumination invariant intensity-based image registration using chaos theory

Accurate and robust registration of image pairs is of interest in many fields that use computer vision such as surveillance and medical diagnostics. In each of these fields the area-based (or voxel-based) approach to image registration is popular, however it is known that these methods are sensitive to illumination change where incorrect results are common. Past work in applying chaos theory to computer vision has demonstrated that the underlying physics of illumination change versus contextual change result in very different behavior when analyzed in phase space. Illumination is deterministic and results in non-fractal phase space behavior, while contextual change is chaos-like and results in complex fractal regions in phase space. A chaos-theoretic approach to image registration is presented with favorable results compared to the traditional and very popular Mutual Information measure.

Chaotic Phenomena from Motion in Image Sequences

Recent research in neural signals have shown biological neural systems are highly responsive to chaotic signals. In this paper, we analyze image sequences using tools from chaos theory associated with the phase plot of time series signals. We demonstrate that the changes in pixel amplitudes due to the motion of objects in an image sequence result in interesting behavior, visible as fractal patterns in the phase space, reminiscent of chaotic strange attractors. Likewise we demonstrate that changes in image amplitude due to spatio-temporally varying illumination are non-chaotic and well structured in phase space. We use the Lambertian reflectance model to explain the source of the non-linearities responsible for this chaotic behavior.

A comparison of a chaos-theoretic method for pre-attentive vision with traditional grayscale-based methods

Accurate and robust attention direction has been of substantial interest in the computer vision community, particularly for industrial surveillance systems that initiate recording at the onset of motion or an interesting contextual event. One key issue is minimizing false alarms to limit video record bandwidth and capacity. One issue that these systems face is high false alarm rates under sudden illumination change. In this paper we propose a system which applies measures from chaos theory and fractal analysis to provide a robust pre-attentive processing engine for motion detection. Results compare quite favorably in terms of probability of detection versus false detection rate against traditional methods for low-level change detection, namely Sum of Absolute Differences, and Gaussian Mixture Models. The proposed chaos-based method is shown to have superior performance. Additionally the proposed approach has an intuitive justification based on creation and flow of information between image frames, and consequently a very intuitive and problem-based threshold determination.