Ajay J. Joshi

Multi-class active learning for image classification

One of the principal bottlenecks in applying learning techniques to classification problems is the large amount of labeled training data required. Especially for images and video, providing training data is very expensive in terms of human time and effort. In this paper we propose an active learning approach to tackle the problem. Instead of passively accepting random training examples, the active learning algorithm iteratively selects unlabeled examples for the user to label, so that human effort is focused on labeling the most “useful” examples. Our method relies on the idea of uncertainty sampling, in which the algorithm selects unlabeled examples that it finds hardest to classify. Specifically, we propose an uncertainty measure that generalizes margin-based uncertainty to the multi-class case and is easy to compute, so that active learning can handle a large number of classes and large data sizes efficiently. We demonstrate results for letter and digit recognition on datasets from the UCI repository, object recognition results on the Caltech-101 dataset, and scene categorization results on a dataset of 13 natural scene categories. The proposed method gives large reductions in the number of training examples required over random selection to achieve similar classification accuracy, with little computational overhead.

Estimating pedestrian counts in groups

The goal of this work is to provide a system which can aid in monitoring crowded urban environments, which often contain tight groups of people. In this paper, we consider the problem of counting the number of people in the scene and also tracking them reliably. We propose a novel method for detecting and estimating the count of people in groups, dense or otherwise, as well as tracking them. Using prior knowledge obtained from the scene and accurate camera calibration, the system learns the parameters required for estimation. This information can then be used to estimate the count of people in the scene, in real-time. Groups are tracked in the same manner as individuals, using Kalman filtering techniques. Favorable results are shown for groups of various sizes moving in an unconstrained fashion.

Learning to Detect Moving Shadows in Dynamic Environments

We propose a novel adaptive technique for detecting moving shadows and distinguishing them from moving objects in video sequences. Most methods for detecting shadows work in a static setting with significant human input. To remove these limitations, we propose a more general semi-supervised learning technique to tackle the problem. First, we exploit characteristic differences in color and edges in the video frames to come up with a set of features useful for classification. Second, we use a learning technique that employs Support Vector Machines and the Co-training algorithm, that relies on a small set of human-labeled data. We observe a surprising phenomenon that Co-training can counter the effects of changing underlying probability distributions in the feature space. From the standpoint of detecting shadows, once deployed, the proposed method can dynamically adapt to varying conditions without any manual intervention, and performs better classification than previous methods on static and dynamic environments alike. The strengths of the proposed technique are the small quantity of human labeled data required, and the ability to adapt automatically to changing scene conditions.

Scalable Active Learning for Multiclass Image Classification

Machine learning techniques for computer vision applications like object recognition, scene classification, etc., require a large number of training samples for satisfactory performance. Especially when classification is to be performed over many categories, providing enough training samples for each category is infeasible. This paper describes new ideas in multiclass active learning to deal with the training bottleneck, making it easier to train large multiclass image classification systems. First, we propose a new interaction modality for training which requires only yes-no type binary feedback instead of a precise category label. The modality is especially powerful in the presence of hundreds of categories. For the proposed modality, we develop a Value-of-Information (VOI) algorithm that chooses informative queries while also considering user annotation cost. Second, we propose an active selection measure that works with many categories and is extremely fast to compute. This measure is employed to perform a fast seed search before computing VOI, resulting in an algorithm that scales linearly with dataset size. Third, we use locality sensitive hashing to provide a very fast approximation to active learning, which gives sublinear time scaling, allowing application to very large datasets. The approximation provides up to two orders of magnitude speedups with little loss in accuracy. Thorough empirical evaluation of classification accuracy, noise sensitivity, imbalanced data, and computational performance on a diverse set of image datasets demonstrates the strengths of the proposed algorithms.

Moving Shadow Detection with Low- and Mid-Level Reasoning

In this paper, we propose a multi-level shadow identification scheme which is generally applicable without restrictions on the number of light sources, illumination conditions, surface orientations, and object sizes. In the first level, we use a background segmentation technique to identify foreground regions which include moving shadows. In the second step, pixel-based decisions are made by comparing the current frame with the background model to distinguish between shadows and actual foreground. In the third step, this result is improved using blob-level reasoning which works on geometric constraints of identified shadow and foreground blobs. Results on various indoor and outdoor sequences under different illumination conditions show the success of the proposed approach.

Breaking the interactive bottleneck in multi-class classification with active selection and binary feedback

Multi-class classification schemes typically require human input in the form of precise category names or numbers for each example to be annotated – providing this can be impractical for the user when a large (and possibly unknown) number of categories are present. In this paper, we propose a multi-class active learning model that requires only binary (yes/no type) feedback from the user. For instance, given two images the user only has to say whether they belong to the same class or not. We first show the interactive benefits of such a scheme with user experiments. We then propose a Value of Information (VOI)-based active selection algorithm in the binary feedback model. The algorithm iteratively selects image pairs for annotation so as to maximize accuracy, while also minimizing user annotation effort. To our knowledge, this is the first multi-class active learning approach that requires only yes/no inputs. Experiments show that the proposed method can substantially minimize user supervision compared to the traditional training model, on problems with as many as 100 classes. We also demonstrate that the system is robust to real-world issues such as class population imbalance and labeling noise.

Scene-Adaptive Human Detection with Incremental Active Learning

In many computer vision tasks, scene changes hinder the generalization ability of trained classifiers. For instance, a human detector trained with one set of images is unlikely to perform well in different scene conditions. In this paper, we propose an incremental learning method for human detection that can take generic training data and build a new classifier adapted to the new deployment scene. Two operation modes are proposed: i) a completely autonomous mode wherein first few empty frames of video are used for adaptation, and ii) an active learning approach with user in the loop, for more challenging scenarios including situations where empty initialization frames may not exist. Results show the strength of the proposed methods for quick adaptation.

Learning of moving cast shadows for dynamic environments

We propose a novel online framework for detecting moving shadows in video sequences using statistical learning techniques. In this framework, support vector machines are applied to obtain a classifier that can differentiate between moving shadows and other foreground objects. The co-training algorithm of Blum and Mitchell is then used in an online setting to improve accuracy with the help of unlabeled data. We evaluate the concept of co-training and show its viability even when explicit assumptions made by the algorithm are not satisfied. Thus, given a small random set of labeled examples (in our application domain, shadow and foreground), the system gives encouraging generalization performance using a semi-supervised approach. In dynamic environments such as those induced by robot motion, the view changes significantly and traditional algorithms do not work well. Our method can handle such changing conditions by adapting online using a semi-supervised approach.

Coverage optimized active learning for k - NN classifiers

Fast image recognition and classification is extremely important in various robotics applications such as exploration, rescue, localization, etc. k-nearest neighbor (kNN) classifiers are popular tools used in classification since they involve no explicit training phase, and are simple to implement. However, they often require large amounts of training data to work well in practice. In this paper, we propose a batch-mode active learning algorithm for efficient training of kNN classifiers, that substantially reduces the amount of training required. As opposed to much previous work on iterative single-sample active selection, the proposed system selects samples in batches. We propose a coverage formulation that enforces selected samples to be distributed such that all data points have labeled samples at a bounded maximum distance, given the training budget, so that there are labeled neighbors in a small neighborhood of each point. Using submodular function optimization, the proposed algorithm presents a near-optimal selection strategy for an otherwise intractable problem. Further we employ uncertainty sampling along with coverage to incorporate model information and improve classification. Finally, we use locality sensitive hashing for fast retrieval of nearest neighbors during active selection as well as classification, which provides 1-2 orders of magnitude speedups thus allowing real-time classification with large datasets.