Thomas B. Sebastian

Recognition of shapes by editing their shock graphs

This paper presents a novel framework for the recognition of objects based on their silhouettes. The main idea is to measure the distance between two shapes as the minimum extent of deformation necessary for one shape to match the other. Since the space of deformations is very high-dimensional, three steps are taken to make the search practical: 1) define an equivalence class for shapes based on shock-graph topology, 2) define an equivalence class for deformation paths based on shock-graph transitions, and 3) avoid complexity-increasing deformation paths by moving toward shock-graph degeneracy. Despite these steps, which tremendously reduce the search requirement, there still remain numerous deformation paths to consider. To that end, we employ an edit-distance algorithm for shock graphs that finds the optimal deformation path in polynomial time. The proposed approach gives intuitive correspondences for a variety of shapes and is robust in the presence of a wide range of visual transformations. The recognition rates on two distinct databases of 99 and 216 shapes each indicate highly successful within category matches (100 percent in top three matches), which render the framework potentially usable in a range of shape-based recognition applications.

On aligning curves

We present a novel approach to finding a correspondence (alignment) between two curves. The correspondence is based on a notion of an alignment curve which treats both curves symmetrically. We then define a similarity metric based on the alignment curve using two intrinsic properties of the curve, namely, length and curvature. The optimal correspondence is found by an efficient dynamic-programming method both for aligning pairs of curve segments and pairs of closed curves, and is effective in the presence of a variety of transformations of the curve. Finally, the correspondence is shown in application to handwritten character recognition, prototype formation, and object recognition, and is potentially useful in other applications such as registration and tracking.

Person Reidentification Using Spatiotemporal Appearance

In many surveillance applications it is desirable to determine if a given individual has been previously observed over a network of cameras. This is the person reidentification problem. This paper focuses on reidentification algorithms that use the overall appearance of an individual as opposed to passive biometrics such as face and gait. Person reidentification approaches have two aspects: (i) establish correspondence between parts, and (ii) generate signatures that are invariant to variations in illumination, pose, and the dynamic appearance of clothing. A novel spatiotemporal segmentation algorithm is employed to generate salient edgels that are robust to changes in appearance of clothing. The invariant signatures are generated by combining normalized color and salient edgel histograms. Two approaches are proposed to generate correspondences: (i) a model based approach that fits an articulated model to each individual to establish a correspondence map, and (ii) an interest point operator approach that nominates a large number of potential correspondences which are evaluated using a region growing scheme. Finally, the approaches are evaluated on a 44 person database across 3 disparate views.

Shape and Appearance Context Modeling

In this work we develop appearance models for computing the similarity between image regions containing deformable objects of a given class in realtime. We introduce the concept of shape and appearance context. The main idea is to model the spatial distribution of the appearance relative to each of the object parts. Estimating the model entails computing occurrence matrices. We introduce a generalization of the integral image and integral histogram frameworks, and prove that it can be used to dramatically speed up occurrence computation. We demonstrate the ability of this framework to recognize an individual walking across a network of cameras. Finally, we show that the proposed approach outperforms several other methods.

Appearance-based person reidentification in camera networks: problem overview and current approaches

Recent advances in visual tracking methods allow following a given object or individual in presence of significant clutter or partial occlusions in a single or a set of overlapping camera views. The question of when person detections in different views or at different time instants can be linked to the same individual is of fundamental importance to the video analysis in large-scale network of cameras. This is the person reidentification problem. The paper focuses on algorithms that use the overall appearance of an individual as opposed to passive biometrics such as face and gait. Methods that effectively address the challenges associated with changes in illumination, pose, and clothing appearance variation are discussed. More specifically, the development of a set of models that capture the overall appearance of an individual and can effectively be used for information retrieval are reviewed. Some of them provide a holistic description of a person, and some others require an intermediate step where specific body parts need to be identified. Some are designed to extract appearance features over time, and some others can operate reliably also on single images. The paper discusses algorithms for speeding up the computation of signatures. In particular it describes very fast procedures for computing co-occurrence matrices by leveraging a generalization of the integral representation of images. The algorithms are deployed and tested in a camera network comprising of three cameras with non-overlapping field of views, where a multi-camera multi-target tracker links the tracks in different cameras by reidentifying the same people appearing in different views.

Segmentation of carpal bones from CT images using skeletally coupled deformable models.

The in vivo investigation of joint kinematics in normal and injured wrist requires the segmentation of carpal bones from 3D (CT) images, and their registration over time. The non-uniformity of bone tissue, ranging from dense cortical bone to textured spongy bone, the irregular shape of closely packed carpal bones, small inter-bone spaces compared to the resolution of CT images, along with the presence of blood vessels, and the inherent blurring of CT imaging render the segmentation of carpal bones a challenging task. We review the performance of statistical classification, deformable models (active contours), region growing, region competition, and morphological operations for this application. We then propose a model which combines several of these approaches in a unified framework. Specifically, our approach is to use a curve evolution implementation of region growing from initialized seeds, where growth is modulated by a skeletally-mediated competition between neighboring regions. The inter-seed skeleton, which we interpret as the predicted boundary of collision between two regions, is used to couple the growth of seeds and to mediate long-range competition between them. The implementation requires subpixel representations of each growing region as well as the inter-region skeleton. This method combines the advantages of active contour models, region growing, and both local and global region competition methods. We demonstrate the effectiveness of this approach for our application where many of the difficulties presented above are overcome as illustrated by synthetic and real examples. Since this segmentation method does not rely on domain-specific knowledge, it should be applicable to a range of other medical imaging segmentation tasks.

Shock-Based Indexing into Large Shape Databases

This paper examines issues arising in applying a previously developed edit-distance shock graph matching technique to indexing into large shape databases. This approach compares the shock graph topology and attributes to produce a similarity metric, and results in 100% recognition rate in querying a database of approximately 200 shapes. However, indexing into a significantly larger database is faced with both the lack of a suitable database, and more significantly with the expense related to computing the metric. We have thus (i) gathered shapes from a variety of sources to create a database of over 1000 shapes from forty categories as a stage towards developing an approach for indexing into a much larger database; (ii) developed a coarse-scale approximate similarly measure which relies on the shock graph topology and a very coarse sampling of link attributes. We show that this is a good first-order approximation of the similarly metric and is two orders of magnitude more efficient to compute. An interesting outcome of using this efficient but approximate similarity measure is that the approximation naturally demands a notion of categories to give high precision; (iii) developed an exemplar-based indexing scheme which discards a large number of non-matching shapes solely based on distance to exemplars, coarse scale representatives of each category. The use of a coarse-scale matching measure in conjunction with a coarse-scale sampling of the database leads to a significant reduction in the computational effort without discarding correct matches, thus paving the way for indexing into databases of tens of thousands of shapes.

Curves vs skeletons in object recognition

The type of representation used in describing shape can have a significant impact on the effectiveness of a recognition strategy. Shape has been represented by its bounding curve as well as by the medial axis representation which captures the regional interaction of the boundaries. Shape matching with the former representation is achieved by curve matching, while the latter is achieved by matching skeletal graphs. We compare the effectiveness of these two methods using approaches which we have developed recently for each. The results indicate that skeletal matching involves a higher degree of computational complexity, but is better than curve matching in the presence of articulation or rearrangement of parts. However, when these variations are not present, curve matching is a better strategy due to its lower complexity and roughly equivalent recognition rate.

Curves vs. skeletons in object recognition

The type of representation used in describing shape can have a significant impact on the effectiveness and efficiency of a recognition strategy. Shape has been represented by a point set, outline curve and shock-graph (medial axis). The curve-based representation can be viewed as point-based representation with additional organization, namely, order along a contour; shock-based representation, in turn, Can be viewed as curve-based representation with additional organization, namely, pairing of contours. This additional complexity in organization leads to greater computational effort in deriving and matching these representations. However, it leads to an increase in robustness in the presence of variations. In This paper, we examine the tradeoff between robustness and computational complexity for curve-and shock-based representations. Our results indicate that the additional computational effort required in shock-graph matching is worthwhile in the presence of large amount variations, in particular those involving the presence of articulation or rearrangement of parts. However, when the space of variations is smaller, curve matching is a better strategy due to its lower complexity and roughly equivalent recognition rate.

Metric-based shape retrieval in large databases

This paper examines the problem of database organization and retrieval based on computing metric pairwise distances. A low-dimensional Euclidean approximation of a high-dimensional metric space is not efficient, while search in a high-dimensional Euclidean space suffers from the curse of dimensionality. Thus, techniques designed for searching metric spaces must be used. We evaluate several such existing exact metric-based indexing techniques, and show that they require extensive computational effort. This motivates the development of an approximate nearest neighbor search technique where the k nearest neighbors are used to approximate the local neighborhood of a point. The resulting kNN graph is searched in a best-first fashion producing excellent indexing efficiency.