Claire Monteleoni

Analysis of Perceptron-Based Active Learning

We start by showing that in an active learning setting, the Perceptron algorithm needs Ω(1/e2) labels to learn linear separators within generalization error e. We then present a simple active learning algorithm for this problem, which combines a modification of the Perceptron update with an adaptive filtering rule for deciding which points to query. For data distributed uniformly over the unit sphere, we show that our algorithm reaches generalization error e after asking for just O(d log 1/e) labels. This exponential improvement over the usual sample complexity of supervised learning had previously been demonstrated only for the computationally more complex query-by-committee algorithm.

Practical Online Active Learning for Classification

We compare the practical performance of several recently proposed algorithms for active learning in the online classification setting. We consider two active learning algorithms (and their combined variants) that are strongly online, in that they access the data sequentially and do not store any previously labeled examples, and for which formal guarantees have recently been proven under various assumptions. We motivate an optical character recognition (OCR) application that we argue to be appropriately served by online active learning. We compare the practical efficacy, for this application, of the algorithm variants, and show significant reductions in label-complexity over random sampling.

Fast spectral clustering via the Nystrom method

We propose and analyze a fast spectral clustering algorithm with computational complexity linear in the number of data points that is directly applicable to large-scale datasets. The algorithm combines two powerful techniques in machine learning: spectral clustering algorithms and Nystrom methods commonly used to obtain good quality low rank approximations of large matrices. The proposed algorithm applies the Nystrom approximation to the graph Laplacian to perform clustering. We provide theoretical analysis of the performance of the algorithm and show the error bound it achieves and we discuss the conditions under which the algorithm performance is comparable to spectral clustering with the original graph Laplacian. We also present empirical results.

Environment selection and hierarchical place recognition

As robots continue to create long-term maps, the amount of information that they need to handle increases over time. In terms of place recognition, this implies that the number of images being considered may increase until exceeding the computational resources of the robot. In this paper we consider a scenario where, given multiple independent large maps, possibly from different cities or locations, a robot must effectively and in real time decide whether it can localize itself in one of those known maps. Since the number of images to be handled by such a system is likely to be extremely large, we find that it is beneficial to decompose the set of images into independent groups or environments. This raises a new question: Given a query image, how do we select the best environment? This paper proposes a similarity criterion that can be used to solve this problem. It is based on the observation that, if each environment is described in terms of its co-occurrent features, similarity between environments can be established by comparing their co-occurrence matrices. We show that this leads to a novel place recognition algorithm that divides the collection of images into environments and arranges them in a hierarchy of inverted indices. By selecting first the relevant environment for the operating robot, we can reduce the number of images to perform the actual loop detection, reducing the execution time while preserving the accuracy. The practicality of this approach is shown through experimental results on several large datasets covering a combined distance of more than 750Km.

A Semi-Supervised Learning Approach to Differential Privacy

Motivated by the semi-supervised model in the data mining literature, we propose a model for differentially-private learning in which private data is augmented by public data to achieve better accuracy. Our main result is a differentially private classifier with significantly improved accuracy compared to previous work. We experimentally demonstrate that such a classifier produces good prediction accuracies even in those situations where the amount of private data is fairly limited. This expands the range of useful applications of differential privacy since typical results in the differential privacy model require large private data sets to obtain good accuracy.

Climate Informatics: Accelerating Discovering in Climate Science with Machine Learning

Given the impact of climate change, understanding the climate system is an international priority. The goal of climate informatics is to inspire collaboration between climate scientists and data scientists, in order to develop tools to analyze complex and ever-growing amounts of observed and simulated climate data, and thereby bridge the gap between data and understanding. Here, recent climate informatics work is presented, along with details of some of the remaining challenges.

Differentially-Private Learning of Low Dimensional Manifolds

In this paper, we study the problem of differentially-private learning of low dimensional manifolds embedded in high dimensional spaces. The problems one faces in learning in high dimensional spaces are compounded in differentially-private learning. We achieve the dual goals of learning the manifold while maintaining the privacy of the dataset by constructing a differentially-private data structure that adapts to the doubling dimension of the dataset. Our differentially-private manifold learning algorithm extends random projection trees of Dasgupta and Freund. A naive construction of differentially-private random projection trees could involve queries with high global sensitivity that would affect the usefulness of the trees. Instead, we present an alternate way of constructing differentially-private random projection trees that uses low sensitivity queries that are precise enough for learning the low dimensional manifolds. We prove that the size of the tree depends only on the doubling dimension of the dataset and not its extrinsic dimension.

Efficient algorithms for general active learning

Selective sampling, a realistic active learning model, has received recent attention in the learning theory literature. While the analysis of selective sampling is still in its infancy, we focus here on one of the (seemingly) simplest problems that. remain open. Given a pool of unlabeled examples, drawn i.i.d. from an arbitrary input distribution known to the learner, and oracle access to their labels, the objective is to achieve a target error-rate with minimum label-complexity, via an efficient algorithm. No prior distribution is assumed over the concept class, however the problem remains open even under the realizability assumption: there exists a target hypothesis in the concept class that perfectly classifies all examples, and the labeling oracle is noiseless. 1 As a precise variant of the problem, we consider the case of learning homogeneous half-spaces in the realizable setting: unlabeled examples, x t , are drawn i.i.d. from a known distribution D over the surface of the unit ball in R d and labels y t are either -1 or +1. The target function is a half-space u.x > 0 represented by a unit vector u ∈ R d such that y t (u.x t ) > 0 for all t. We denote a hypothesis vs prediction as v(x) = SGN(υ .x). Problem: Provide an algorithm for active learning of half-spaces, such that (with high probability with respect to D and any internal randomness): 1. After L label queries, algorithms hypothesis υ obeys P x∼D [υ(x)¬= u(x)] < e. 2. L is at most the PAC sample complexity of the supervised problem, O(d elog 1 e), and for a general class of input distributions, L is significantly lower. 2 3. Total running time is at most poly(d,1 e).

Resource allocation using sequential auctions

Market-based mechanisms such as auctions are being studied as an appropriate means for resource allocation in distributed and multiagent decision problems. When agents value resources in combination rather than in isolation, one generally relies on combinatorial auctions where agents bid for resource bundles, or simultaneous auctions for all resources. We develop a different model, where agents bid for required resources sequentially. This model has the advantage that it can be applied in settings where combinatorial and simultaneous models are infeasible (e.g., when resources are made available at different points in time by different parties), as well as certain benefits in settings where combinatorial models are applicable. We develop a dynamic programming model for agents to compute bidding policies based on estimated distributions over prices. We also describe how these distributions are updated to provide a learning model for bidding behavior.

Technological Innovation, Data Analytics, and Environmental Enforcement

Technical innovation is ubiquitous in contemporary society and contributes to its extraordinarily dynamic character. Sometimes these innovations have significant effects on the state of the environment or on human health and they have stimulated efforts to develop second order technologies to ameliorate those effects. The development of the automobile and its impact on life in the United States and throughout the world is an example. The story of modern environmental regulation more generally includes chapters filled with examples of similar efforts to respond to an enormous array of technological advances. This Article uses a different lens to consider the role of technological innovation. In particular, it considers how technological advances have the potential to shape governance efforts in the compliance realm. The Article demonstrates that such technological advances – especially new and improved monitoring capacity, advances in information dissemination through e-reporting and other techniques, and improved capacity to analyze information – have significant potential to transform governance efforts to promote compliance. Such transformation is likely to affect not only the “how” of compliance promotion, but also the “who.” Technological innovation is likely to contribute to new thinking about the roles key actors can and should play in promoting compliance with legal norms. The Article discusses some of the potential benefits of these types of technological innovation in the context of the Environmental Protection Agency (EPA)’s ongoing efforts to improve its compliance efforts by taking advantage of emerging technologies. We also identify some of the pitfalls or challenges that agencies such as EPA need to be aware of in opening this emerging bundle of new tools and making use of them to address real-world environmental needs.