Yali Amit

Shape quantization and recognition with randomized trees

We explore a new approach to shape recognition based on a virtually infinite family of binary features (queries) of the image data, designed to accommodate prior information about shape invariance and regularity. Each query corresponds to a spatial arrangement of several local topographic codes (or tags), which are in themselves too primitive and common to be informative about shape. All the discriminating power derives from relative angles and distances among the tags. The important attributes of the queries are a natural partial ordering corresponding to increasing structure and complexity; semi-invariance, meaning that most shapes of a given class will answer the same way to two queries that are successive in the ordering; and stability, since the queries are not based on distinguished points and substructures. No classifier based on the full feature set can be evaluated, and it is impossible to determine a priori which arrangements are informative. Our approach is to select informative features and build tree classifiers at the same time by inductive learning. In effect, each tree provides an approximation to the full posterior where the features chosen depend on the branch that is traversed. Due to the number and nature of the queries, standard decision tree construction based on a fixed-length feature vector is not feasible. Instead we entertain only a small random sample of queries at each node, constrain their complexity to increase with tree depth, and grow multiple trees. The terminal nodes are labeled by estimates of the corresponding posterior distribution over shape classes. An image is classified by sending it down every tree and aggregating the resulting distributions. The method is applied to classifying handwritten digits and synthetic linear and nonlinear deformations of three hundred symbols. State-of-the-art error rates are achieved on the National Institute of Standards and Technology database of digits. The principal goal of the experiments on symbols is to analyze invariance, generalization error and related issues, and a comparison with artificial neural networks methods is presented in this context. Figure 1: LATEX Symbol

A computational model for visual selection

We propose a computational model for detecting and localizing instances from an object class in static gray-level images. We divide detection into visual selection and final classification, concentrating on the former: drastically reducing the number of candidate regions that require further, usually more intensive, processing, but with a minimum of computation and missed detections. Bottom-up processing is based on local groupings of edge fragments constrained by loose geometrical relationships. They have no a priori semantic or geometric interpretation. The role of training is to select special groupings that are moderately likely at certain places on the object but rare in the background. We show that the statistics in both populations are stable. The candidate regions are those that contain global arrangements of several local groupings. Whereas our model was not conceived to explain brain functions, it does cohere with evidence about the functions of neurons in V1 and V2, such as responses to coarse or incomplete patterns (e.g., illusory contours) and to scale and translation invariance in IT. Finally, the algorithm is applied to face and symbol detection.

Joint induction of shape features and tree classifiers

We introduce a very large family of binary features for two-dimensional shapes. The salient ones for separating particular shapes are determined by inductive learning during the construction of classification trees. There is a feature for every possible geometric arrangement of local topographic codes. The arrangements express coarse constraints on relative angles and distances among the code locations and are nearly invariant to substantial affine and nonlinear deformations. They are also partially ordered, which makes it possible to narrow the search for informative ones at each node of the tree. Different trees correspond to different aspects of shape. They are statistically and weakly dependent due to randomization and are aggregated in a simple way. Adapting the algorithm to a shape family is then fully automatic once training samples are provided. As an illustration, we classified handwritten digits from the NIST database; the error rate was 0.7 percent.

Single-Unit Stability Using Chronically Implanted Multielectrode Arrays

The use of chronic intracortical multielectrode arrays has become increasingly prevalent in neurophysiological experiments. However, it is not obvious whether neuronal signals obtained over multiple recording sessions come from the same or different neurons. Here, we develop a criterion to assess single-unit stability by measuring the similarity of 1) average spike waveforms and 2) interspike interval histograms (ISIHs). Neuronal activity was recorded from four Utah arrays implanted in primary motor and premotor cortices in three rhesus macaque monkeys during 10 recording sessions over a 15- to 17-day period. A unit was defined as stable through a given day if the stability criterion was satisfied on all recordings leading up to that day. We found that 57% of the original units were stable through 7 days, 43% were stable through 10 days, and 39% were stable through 15 days. Moreover, stable units were more likely to remain stable in subsequent recording sessions (i.e., 89% of the neurons that were stable through four sessions remained stable on the fifth). Using both waveform and ISIH data instead of just waveforms improved performance by reducing the number of false positives. We also demonstrate that this method can be used to track neurons across days, even during adaptation to a visuomotor rotation. Identifying a stable subset of neurons should allow the study of long-term learning effects across days and has practical implications for pooling of behavioral data across days and for increasing the effectiveness of brain-machine interfaces.

Graphical templates for model registration

A new method of model registration is proposed using graphical templates. A graph of landmarks is chosen in the template image. All possible candidates for these landmarks are found in the data image using local operators. A dynamic programming algorithm on decomposable subgraphs of the template graph finds the optimal match to a subset of the candidate points in polynomial time. This combination of local operators to describe points of interest/landmarks and a graph to describe their geometric orientation in the plane, yields fast and precise matches of the model to the data, with no initialization required.

Encoding of Movement Fragments in the Motor Cortex

Previous studies have suggested that complex movements can be elicited by electrical stimulation of the motor cortex. Most recording studies in the motor cortex, however, have investigated the encoding of time-independent features of movement such as direction, velocity, position, or force. Here, we show that single motor cortical neurons encode temporally evolving movement trajectories and not simply instantaneous movement parameters. We explicitly characterize the preferred trajectories of individual neurons using a simple exponential encoding model and demonstrate that temporally extended trajectories not only capture the tuning of motor cortical neurons more accurately, but can be used to decode the instantaneous movement direction with less error. These findings suggest that single motor cortical neurons encode whole movement fragments, which are temporally extensive and can be quite complex.

POP: Patchwork of Parts Models for Object Recognition

We formulate a deformable template model for objects with an efficient mechanism for computation and parameter estimation. The data consists of binary oriented edge features, robust to photometric variation and small local deformations. The template is defined in terms of probability arrays for each edge type. A primary contribution of this paper is the definition of the instantiation of an object in terms of shifts of a moderate number local submodels—parts—which are subsequently recombined using a patchwork operation, to define a coherent statistical model of the data. Object classes are modeled as mixtures of patchwork of parts POP models that are discovered sequentially as more class data is observed. We define the notion of the support associated to an instantiation, and use this to formulate statistical models for multi-object configurations including possible occlusions. All decisions on the labeling of the objects in the image are based on comparing likelihoods. The combination of a deformable model with an efficient estimation procedure yields competitive results in a variety of applications with very small training sets, without need to train decision boundaries—only data from the class being trained is used. Experiments are presented on the MNIST database, reading zipcodes, and face detection.

An integrated network for invariant visual detection and recognition

We describe an architecture for invariant visual detection and recognition. Learning is performed in a single central module. The architecture makes use of a replica module consisting of copies of retinotopic layers of local features, with a particular design of inputs and outputs, that allows them to be primed either to attend to a particular location, or to attend to a particular object representation. In the former case the data at a selected location can be classified in the central module. In the latter case all instances of the selected object are detected in the field of view. The architecture is used to explain a number of psychophysical and physiological observations: object based attention, the different response time slopes of target detection among distractors, and observed attentional modulation of neuronal responses. We hypothesize that the organization of visual cortex in columns of neurons responding to the same feature at the same location may provide the copying architecture needed for translation invariance.

Efficient Focusing and Face Detection

We present an algorithm for shape detection and apply it to frontal views of faces in still grey level images with arbitrary backgrounds. Detection is done in two stages: (i) “focusing,” during which a relatively small number of regions-of-interest are identified, minimizing computation and false negatives at the (temporary) expense of false positives; and (ii) “intensive classification,” during which a selected region-of-interest is labeled face or background based on multiple decision trees and normalized data. In contrast to most detection algorithms, the processing is then very highly concentrated in the regions near faces and near false positives.

Graphical shape templates for automatic anatomy detection with applications to MRI brain scans

A new method of model registration is proposed using graphical templates. A decomposable graph of landmarks is chosen in the template image. All possible candidates for these landmarks are found in the data image using robust relational local operators. A dynamic programming algorithm on the template graph finds the optimal match to a subset of the candidate points in polynomial time. This combination-local operators to describe points of interest/landmarks and a graph to describe their geometric arrangement in the plane-yields fast and precise matches of the model to the data with no initialization required. In addition, it provides a generic tool box for modeling shape in a variety of applications. This methodology is applied in the context of T2-weighted magnetic resonance (MR) axial and sagittal images of the brain to identify specific anatomies.