Amitabha Mukerjee

Non-linear Dimensionality Reduction by Locally Linear Isomaps

Algorithms for nonlinear dimensionality reduction (NLDR) find meaningful hidden low-dimensional structures in a high-dimensional space. Current algorithms for NLDR are Isomaps, Local Linear Embedding and Laplacian Eigenmaps. Isomaps are able to reliably recover low-dimensional nonlinear structures in high-dimensional data sets, but suffer from the problem of short-circuiting, which occurs when the neighborhood distance is larger than the distance between the folds in the manifolds. We propose a new variant of Isomap algorithm based on local linear properties of manifolds to increase its robustness to short-circuiting. We demonstrate that the proposed algorithm works better than Isomap algorithm for normal, noisy and sparse data sets.

Conceptual description of visual scenes from linguistic models

Abstract As model-based vision moves towards handling imprecise descriptions like a long bench is in front of the tree, it has to confront questions involving widely variable shapes in unclear positions. Such descriptions may be said to be “conceptual” in the sense that they provide a loose set of constraints permitting a range of instantiations for the scene. One of the validations of a computational systems ability to handle such descriptions is provided by immediate visualization, which tells the user whether the bench is of the right shape and has been positioned correctly. Such a visualization must handle impreciseness in Shape and Spatial Pose, and, for dynamic vision, Object Articulation and Motion Parameters as well. The visualization task is a concretization which consists of generating an “instance” of the scene/action being described. The principal requirement for concretizing the conceptual model is a large visual database of objects and actions, along with a set of constraints corresponding to default dependencies in the domain. In our work, the resulting set of constraints is combined using multi-dimensional fuzzy functions called continuum fields (potentials). A set of experiments was conducted to determine the parameters of these continuum fields. An instance is generated by identifying minima in the continuum fields involved is generated by identifying minima in the continuum fields involved in generating the shape, position and motion. These are then used to create default instantiations of the objects described. The resulting image/animation may be considered to be the “most likely” visualization, and if this matches the linguistic description, the continuum fields selected are a good model for the conceptual content in the linguistic model of the scene. We present examples of scene reconstruction from conceptual descriptions of urban parks.

Detecting Complex Predicates in Hindi using POS Projection across Parallel Corpora

Complex Predicates or CPs are multiword complexes functioning as single verbal units. CPs are particularly pervasive in Hindi and other Indo-Aryan languages, but an usage account driven by corpus-based identification of these constructs has not been possible since single-language systems based on rules and statistical approaches require reliable tools (POS taggers, parsers, etc.) that are unavailable for Hindi. This paper highlights the development of first such database based on the simple idea of projecting POS tags across an English-Hindi parallel corpus. The CP types considered include adjective-verb (AV), noun-verb (NV), adverb-verb (Adv-V), and verb-verb (VV) composites. CPs are hypothesized where a verb in English is projected onto a multi-word sequence in Hindi. While this process misses some CPs, those that are detected appear to be more reliable (83% precision, 46% recall). The resulting database lists usage instances of 1439 CPs in 4400 sentences.

Acquiring linguistic argument structure from multimodal input using attentive focus

This work is premised on three assumptions: that the semantics of certain actions may be learned prior to language, that objects in attentive focus are likely to indicate the arguments participating in that action, and that knowing such arguments helps align linguistic attention on the relevant predicate (verb). Using a computational model of dynamic attention, we present an algorithm that clusters visual events into action classes in an unsupervised manner using the Merge Neural Gas algorithm. With few clusters, the model correlates to coarse concepts such as come-closer, but with a finer granularity, it reveals hierarchical substructure such as come-closer-one-object-static and come-closer-both-moving. That the argument ordering is non-commutative is discovered for actions such as chase or come-closer-one-object-static. Knowing the arguments, and given that noun-referent mappings that are easily learned, language learning can now be constrained by considering only linguistic expressions and actions that refer to the objects in perceptual focus. We learn action schemas for linguistic units like ldquomoving towardsrdquo or ldquochaserdquo, and validate our results by producing output commentaries for 3D video.

Confidence Based updation of Motion Conspicuity in Dynamic Scenes

Computational models of visual attention result in considerable data compression by eliminating processing on regions likely to be devoid of meaningful content. While saliency maps in static images is indexed on image region (pixels), psychovisual data indicates that in dynamic scenes human attention is object driven and localized motion is a significant determiner of object conspicuity. We have introduced a confidence map, which indicates the uncertainty in the position of the moving objects incorporating the exponential loss of information as we move away from the fovea. We improve the model further using a computational model of visual attention based on perceptual grouping of objects with motion and computation of a motion saliency map based on localized motion conspicuity of the objects. Behaviors exhibited in the system include attentive focus on moving wholes, shifting focus in multiple object motion, focus on objects moving contrary to the majority motion. We also present experimental data contrasting the model with human gaze tracking in a simple visual task.

Formulation, detection and application of occlusion states (Oc-7) in the context of multiple object tracking

Occlusion is often thought of as a challenge for visual algorithms, specially tracking. Existing literature, however, has identified a number of occlusion categories in the context of tracking in ad hoc manner. We propose a systematic approach to formulate a set of occlusion cases by considering the spatial relations among object support(s) (projections on the image plane) with the detected foreground blob(s), to show that only 7 occlusion states are possible. We designate the resulting qualitative formalism as Oc-7, and show how these occlusion states can be detected and used effectively for the task of multi-object tracking under occlusion of various types. The object support is decomposed into overlapping patches which are tracked independently on the occurrence of occlusions. As a demonstration of the application of these occlusion states, we propose a reasoning scheme for selective tracker execution and object feature updates to track multiple objects in complex environments.

A multiscale co-linearity statistic based approach to robust background modeling

Background subtraction is an essential task in several static camera based computer vision systems. Background modeling is often challenged by spatio-temporal changes occurring due to local motion and/or variations in illumination conditions. The background model is learned from an image sequence in a number of stages, viz. preprocessing, pixel/region feature extraction and statistical modeling of feature distribution. A number of algorithms, mainly focusing on feature extraction and statistical modeling have been proposed to handle the problems and comparatively little exploration has occurred at the preprocessing stage. Motivated by the fact that disturbances caused by local motions disappear at lower resolutions, we propose to represent the images at multiple scales in the preprocessing stage to learn a pyramid of background models at different resolutions. During operation, foreground pixels are detected first only at the lowest resolution, and only these pixels are further analyzed at higher resolutions to obtain a precise silhouette of the entire foreground blob. Such a scheme is also found to yield a significant reduction in computation. The second contribution in this paper involves the use of the co-linearity statistic (introduced by Mester et al. for the purpose of illumination independent change detection in consecutive frames) as a pixel neighborhood feature by assuming a linear model with a signal modulation factor and additive noise. The use of co-linearity statistic as a feature has shown significant performance improvement over intensity or combined intensity-gradient features. Experimental results and performance comparisons (ROC curves) for the proposed approach with other algorithms show significant improvements for several test sequences.

DynaTracker: Target tracking in active video surveillance systems

Active video surveillance systems provide challenging research issues in the interface of computer vision, pattern recognition and control system analysis. A significant part of such systems is devoted toward active camera control for efficient target tracking. DynaTracker is a pan-tilt device based active camera system for maintaining continuous track of the moving target, while keeping the same at a pre-specified region (typically, the center) of the image. The significant contributions in this work are the use of mean-shift algorithm for visual tracking and the derivation of the error dynamics for a proportional-integral control action. The stability analysis and optimal controller gain selections are performed from the simulation studies of the derived error dynamics. Simulation predictions are also validated from the results of practical experimentations. The present implementation of DynaTracker performs on a standard Pentium IV PC at an average speed of 10 frames per second while operating on color images of 320times240 resolution

Anomaly Localization in Topic-Based Analysis of Surveillance Videos

Topic-models for video analysis have been used for unsupervised identification of normal activity in videos, thereby enabling the detection of anomalous actions. However, while intervals containing anomalies are detected, it has not been possible to localize the anomalous activities in such models. This is a challenging problem as the abnormal content is usually a small fraction of the entire video data and hence distinctions in terms of likelihood are unlikely. Here we propose a methodology to extend the topic based analysis with rich local descriptors incorporating quantized spatio-temporal gradient descriptors with image location and size information. The visual clips over this vocabulary are then represented in latent topic space using models like pLSA. Further, we introduce an algorithm to quantify the anomalous content in a video clip by projecting the learned topic space information. Using the algorithm, we detect whether the video clip is abnormal and if positive, localize the anomaly in spatio-temporal domain. We also contribute one real world surveillance video dataset for comprehensive evaluation of the proposed algorithm. Experiments are presented on the proposed and two other standard surveillance datasets.

Discovering implicit constraints in design

Abstract Designers who are experts in a given design domain are well known to be able to Immediately focus on “good designs,” suggesting that they may have learned additional constraints while exploring the design space based on some functional aspects. These constraints, which are often implicit, result in a redefinition of the design space, and may be crucial for discovering chunks or interrelations among the design variables. Here we propose a machine-learning approach for discovering such constraints in supervised design tasks. We develop models for specifying design function in situations where the design has a given structure or embodiment, in terms of a set of performance metrics that evaluate a given design. The functionally feasible regions, which are those parts of the design space that demonstrate high levels of performance, can now be learned using any general purpose function approximator. We demonstrate this process using examples from the design of simple locking mechanisms, and as in human experience, we show that the quality of the constraints learned improves with greater exposure in the design space. Next, we consider changing the embodiment and suggest that similar embodiments may have similar abstractions. To explore convergence, we also investigate the variability in time and error rates where the experiential patterns are significantly different. In the process, we also consider the situation where certain functionally feasible regions may encode lower dimensional manifolds and how this may relate to cognitive chunking.