XuanLong Nguyen

Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization

We develop and analyze M-estimation methods for divergence functionals and the likelihood ratios of two probability distributions. Our method is based on a nonasymptotic variational characterization of f -divergences, which allows the problem of estimating divergences to be tackled via convex empirical risk optimization. The resulting estimators are simple to implement, requiring only the solution of standard convex programs. We present an analysis of consistency and convergence for these estimators. Given conditions only on the ratios of densities, we show that our estimators can achieve optimal minimax rates for the likelihood ratio and the divergence functionals in certain regimes. We derive an efficient optimization algorithm for computing our estimates, and illustrate their convergence behavior and practical viability by simulations.

A kernel-based learning approach to ad hoc sensor network localization

We show that the coarse-grained and fine-grained localization problems for ad hoc sensor networks can be posed and solved as a pattern recognition problem using kernel methods from statistical learning theory. This stems from an observation that the kernel function, which is a similarity measure critical to the effectiveness of a kernel-based learning algorithm, can be naturally defined in terms of the matrix of signal strengths received by the sensors. Thus we work in the natural coordinate system provided by the physical devices. This not only allows us to sidestep the difficult ranging procedure required by many existing localization algorithms in the literature, but also enables us to derive a simple and effective localization algorithm. The algorithm is particularly suitable for networks with densely distributed sensors, most of whose locations are unknown. The computations are initially performed at the base sensors, and the computation cost depends only on the number of base sensors. The localization step for each sensor of unknown location is then performed locally in linear time. We present an analysis of the localization error bounds, and provide an evaluation of our algorithm on both simulated and real sensor networks.

Communication-Efficient Online Detection of Network-Wide Anomalies

There has been growing interest in building large-scale distributed monitoring systems for sensor, enterprise, and ISP networks. Recent work has proposed using principal component analysis (PCA) over global traffic matrix statistics to effectively isolate network-wide anomalies. To allow such a PCA-based anomaly detection scheme to scale, we propose a novel approximation scheme that dramatically reduces the burden on the production network. Our scheme avoids the expensive step of centralizing all the data by performing intelligent filtering at the distributed monitors. This filtering reduces monitoring bandwidth overheads, but can result in the anomaly detector making incorrect decisions based on a perturbed view of the global data set. We employ stochastic matrix perturbation theory to bound such errors. Our algorithm selects the filtering parameters at local monitors such that the errors made by the detector are guaranteed to lie below a user-specified upper bound. Our algorithm thus allows network operators to explicitly balance the tradeoff between detection accuracy and the amount of data communicated over the network. In addition, our approach enables real-time detection because we exploit continuous monitoring at the distributed monitors. Experiments with traffic data from Abilene backbone network demonstrate that our methods yield significant communication benefits while simultaneously achieving high detection accuracy.

Planning graph as the basis for deriving heuristics for plan synthesis by state space and CSP search

Most recent strides in scaling up planning have centered around two competing themesdisjunctive planners, exemplified by Graphplan, and heuristic state search planners, exemplified by UNPOP, HSP and HSP-r. In this paper, we present a novel approach for successfully harnessing the advantages of the two competing paradigms to develop planners that are significantly more powerful than either of the approaches. Specifically, we show that the polynomial-time planning graph structure that the Graphplan builds provides a rich substrate for deriving a family of highly effective heuristics for guiding state space search as well as CSP style search. The main leverage provided by the planning graph structure is a systematic and graded way to take subgoal interactions into account in designing state space heuristics. For state space search, we develop several families of heuristics, some aimed at search speed and others at optimality of solutions, and analyze many approaches for improving the cost-quality tradeoffs offered by these heuristics. Our normalized empirical comparisons show that our heuristics handily outperform the existing state space heuristics. For CSP style search, we describe a novel way of using the planning graph structure to derive highly effective variable and value ordering heuristics. We show that these heuristics can be used to improve Graphplans own backward search significantly. To demonstrate the effectiveness of our approach vis a vis the state-of-the-art in plan synthesis, we present AltAlt, a planner literally cobbled together from the implementations of Graphplan and state search style planners using our theory. We evaluate AltAlt on the suite of problems used in the recent AIPS-2000 planning competition. The results place AltAlt in the top tier of the competition plannersoutperforming both Graphplan based and heuristic search based planners.

Nonparametric decentralized detection using kernel methods

We consider the problem of decentralized detection under constraints on the number of bits that can be transmitted by each sensor. In contrast to most previous work, in which the joint distribution of sensor observations is assumed to be known, we address the problem when only a set of empirical samples is available. We propose a novel algorithm using the framework of empirical risk minimization and marginalized kernels and analyze its computational and statistical properties both theoretically and empirically. We provide an efficient implementation of the algorithm and demonstrate its performance on both simulated and real data sets.

ON surrogate loss functions and f-divergences

The goal of binary classification is to estimate a discriminant function y from observations of covariate vectors and corresponding binary labels. We consider an elaboration of this problem in which the covariates are not available directly but are transformed by a dimensionality-reducing quantizer Q. We present conditions on loss functions such that empirical risk minimization yields Bayes consistency when both the discriminant function and the quantizer are estimated. These conditions are stated in terms of a general correspondence between loss functions and a class of functionals known as Ali-Silvey or /-divergence functionals. Whereas this correspondence was established by Blackwell [Proc. 2nd Berkeley Symp. Probab. Statist. 1 (1951) 93-102. Univ. California Press, Berkeley] for the 0-1 loss, we extend the correspondence to the broader class of surrogate loss functions that play a key role in the general theory of Bayes consistency for binary classification. Our result makes it possible to pick out the (strict) subset of surrogate loss functions that yield Bayes consistency for joint estimation of the discriminant function and the quantizer.

Nonlinear identification of a gasoline HCCI engine using neural networks coupled with principal component analysis

Homogeneous charge compression ignition (HCCI) is a futuristic combustion technology that operates with high efficiency and reduced emissions. HCCI combustion is characterized by complex nonlinear dynamics which necessitates the use of a predictive model in controller design. Developing a physics based model for HCCI involves significant development times and associated costs arising from developing simulation models and calibration. In this paper, a neural networks (NN) based methodology is reported where black box type models are developed to predict HCCI combustion behavior during transient operation. The NN based approach can be considered a low cost and quick alternative to the traditional physics based modeling. A multi-input single-output model was developed each for indicated net mean effective pressure, combustion phasing, maximum in-cylinder pressure rise rate and equivalent air-fuel ratio. The two popular architectures namely multi-layer perceptron (MLP) and radial basis network (RBN) models were compared with respect to design, prediction performance and overall applicability to the transient HCCI modeling problem. A principal component analysis (PCA) is done as a pre-processing step to reduce input dimension thereby reducing memory requirements of the models. Also, PCA reduces the cross-validation time required to identify optimal model hyper-parameters. On comparing the model predictions with the experimental data, it was shown that neural networks can be a powerful approach for non-linear identification of a complex combustion system like the HCCI engine.

Distributed Online Simultaneous Fault Detection for Multiple Sensors

Monitoring its health by detecting its failed sensors is essential to the reliable functioning of any sensor network. This paper presents a distributed, online, sequential algorithm for detecting multiple faults in a sensor network. The algorithm works by detecting change points in the correlation statistics of neighboring sensors, requiring only neighbors to exchange information. The algorithm provides guarantees on detection delay and false alarm probability. This appears to be the first work to offer such guarantees for a multiple sensor network. Based on the performance guarantees, we compute a tradeoff between sensor node density, detection delay and energy consumption. We also address synchronization, finite storage and data quantization. We validate our approach with some example applications.

Nonparametric estimation of the likelihood ratio and divergence functionals

We develop and analyze a nonparametric method for estimating the class of f-divergence functionals, and the density ratio of two probability distributions. Our method is based on a non-asymptotic variational characterization of the f-divergence, which allows us to cast the problem of estimating divergences in terms of risk minimization. We thus obtain an M-estimator for divergences, based on a convex and differentiable optimization problem that can be solved efficiently. We analyze the consistency and convergence rates for this M-estimator given conditions only on the ratio of densities.

An ELM based predictive control method for HCCI engines

We formulate and develop a control method for homogeneous charge compression ignition (HCCI) engines using model predictive control (MPC) and models learned from operational data. An HCCI engine is a highly efficient but complex combustion system that operates with a high fuel efficiency and reduced emissions compared to the present technology. HCCI control is a nonlinear, multi-input multi-output problem with state and actuator constraints which makes controller design a challenging task. In this paper, we propose an MPC approach where the constraints are elegantly included in the control problem along with optimality in control. We develop the engine models using experimental data so that the complexity and time involved in the modeling process can be reduced. An Extreme Learning Machine (ELM) is used to capture the engine dynamic behavior and is used by the MPC controller to evaluate control actions. We also used a simplified quadratic programming making use of the convexity of the MPC problem so that the algorithm can be implemented on the engine control unit that is limited in memory. The working and effectiveness of the proposed MPC methodology has been analyzed in simulation using a nonlinear HCCI engine model. The controller tracks several reference signals taking into account the constraints defined by HCCI states, actuators and operational limits.