Eric A. Wan

The unscented Kalman filter for nonlinear estimation

This paper points out the flaws in using the extended Kalman filter (EKE) and introduces an improvement, the unscented Kalman filter (UKF), proposed by Julier and Uhlman (1997). A central and vital operation performed in the Kalman filter is the propagation of a Gaussian random variable (GRV) through the system dynamics. In the EKF the state distribution is approximated by a GRV, which is then propagated analytically through the first-order linearization of the nonlinear system. This can introduce large errors in the true posterior mean and covariance of the transformed GRV, which may lead to sub-optimal performance and sometimes divergence of the filter. The UKF addresses this problem by using a deterministic sampling approach. The state distribution is again approximated by a GRV, but is now represented using a minimal set of carefully chosen sample points. These sample points completely capture the true mean and covariance of the GRV, and when propagated through the true nonlinear system, captures the posterior mean and covariance accurately to the 3rd order (Taylor series expansion) for any nonlinearity. The EKF in contrast, only achieves first-order accuracy. Remarkably, the computational complexity of the UKF is the same order as that of the EKF. Julier and Uhlman demonstrated the substantial performance gains of the UKF in the context of state-estimation for nonlinear control. Machine learning problems were not considered. We extend the use of the UKF to a broader class of nonlinear estimation problems, including nonlinear system identification, training of neural networks, and dual estimation problems. In this paper, the algorithms are further developed and illustrated with a number of additional examples.

The square-root unscented Kalman filter for state and parameter-estimation

Over the last 20-30 years, the extended Kalman filter (EKF) has become the algorithm of choice in numerous nonlinear estimation and machine learning applications. These include estimating the state of a nonlinear dynamic system as well estimating parameters for nonlinear system identification (eg, learning the weights of a neural network). The EKF applies the standard linear Kalman filter methodology to a linearization of the true nonlinear system. This approach is sub-optimal, and can easily lead to divergence. Julier et al. (1997), proposed the unscented Kalman filter (UKF) as a derivative-free alternative to the extended Kalman filter in the framework of state estimation. This was extended to parameter estimation by Wan and Van der Merwe et al., (2000). The UKF consistently outperforms the EKF in terms of prediction and estimation error, at an equal computational complexity of (OL/sup 3/)/sup l/ for general state-space problems. When the EKF is applied to parameter estimation, the special form of the state-space equations allows for an O(L/sup 2/) implementation. This paper introduces the square-root unscented Kalman filter (SR-UKF) which is also O(L/sup 3/) for general state estimation and O(L/sup 2/) for parameter estimation (note the original formulation of the UKF for parameter-estimation was O(L/sup 3/)). In addition, the square-root forms have the added benefit of numerical stability and guaranteed positive semi-definiteness of the state covariances.

Neural network classification: a Bayesian interpretation

The relationship between minimizing a mean squared error and finding the optimal Bayesian classifier is reviewed. This provides a theoretical interpretation for the process by which neural networks are used in classification. A number of confidence measures are proposed to evaluate the performance of the neural network classifier within a statistical framework.

RSSI-Based Indoor Localization and Tracking Using Sigma-Point Kalman Smoothers

Solutions for indoor tracking and localization have become more critical with recent advancement in context and location-aware technologies. The accuracy of explicit positioning sensors such as global positioning system (GPS) is often limited for indoor environments. In this paper, we evaluate the feasibility of building an indoor location tracking system that is cost effective for large scale deployments, can operate over existing Wi-Fi networks, and can provide flexibility to accommodate new sensor observations as they become available. This paper proposes a sigma-point Kalman smoother (SPKS)-based location and tracking algorithm as a superior alternative for indoor positioning. The proposed SPKS fuses a dynamic model of human walking with a number of low-cost sensor observations to track 2-D position and velocity. Available sensors include Wi-Fi received signal strength indication (RSSI), binary infra-red (IR) motion sensors, and binary foot-switches. Wi-Fi signal strength is measured using a receiver tag developed by Ekahau, Inc. The performance of the proposed algorithm is compared with a commercially available positioning engine, also developed by Ekahau, Inc. The superior accuracy of our approach over a number of trials is demonstrated.

Temporal backpropagation for FIR neural networks

The traditional feedforward neural network is a static structure which simply maps input to output. To better reflect the dynamics in a biological system, a network structure which models each synapse by a finite-impulse response (FIR) linear filter is proposed. An efficient-gradient descent algorithm which is shown to be a temporal generalization of the familiar backpropagation algorithm is derived. By modeling each synapse as a linear filter, the neural network as a whole may be thought of as an adaptive system with its own internal dynamics. Equivalently, one may think of the network as a complex nonlinear filter. Applications should thus include areas of pattern recognition where there is an inherent temporal quality to the data, such as in speech recognition. The networks should also find a natural use in areas of nonlinear control, and other adaptive signal processing and filtering applications such as noise cancellation or equalization

Neural dual extended Kalman filtering: applications in speech enhancement and monaural blind signal separation

The removal of noise from speech signals has applications ranging from speech enhancement for cellular communications, to front ends for speech recognition systems. A nonlinear time-domain method called dual extended Kalman filtering (DEKF) is presented for removing nonstationary and colored noise from speech. We further generalize the algorithm to perform the blind separation of two speech signals from a single recording.

Adjoint LMS: an efficient alternative to the filtered-x LMS and multiple error LMS algorithms

The Filtered-x LMS algorithm is currently the most popular method for adapting a filter when there exists a transfer function in the error path. Such instances arise, for example, in active control of sound and vibration. For multiple-input-multiple-output systems the Multiple Error LMS Algorithm is a generalization of Filtered-x LMS. The derivation of both algorithms rely on several assumptions, including linearity of the adaptive filter and error channel. Furthermore, in the Multiple Error LMS Algorithm the desirable order N computational complexity of LMS is lost, resulting in a prohibitive cost in certain DSP implementations. In this paper, we describe a new algorithm termed adjoint LMS which provides a simple alternative to the previously mentioned algorithms. In adjoint LMS, the error (rather than the input) is filtered through an adjoint filter of the error channel. Performance regarding convergence and misadjustment are equivalent. However, linearity is not assumed in the derivation of the algorithm. Furthermore, equations for single-input-single-output (SISO) and multiple-input-multiple-output (MIMO) are identical and both remain order N.

Diagrammatic derivation of gradient algorithms for neural networks

Deriving gradient algorithms for time-dependent neural network structures typically requires numerous chain rule expansions, diligent bookkeeping, and careful manipulation of terms. In this paper, we show how to derive such algorithms via a set of simple block diagram manipulation rules. The approach provides a common framework to derive popular algorithms including backpropagation and backpropagation-through-time without a single chain rule expansion. Additional examples are provided for a variety of complicated architectures to illustrate both the generality and the simplicity of the approach.

Speech enhancement based on temporal processing

Finite impulse response (FIR) Wiener-like filters are applied to time trajectories of the cubic-root compressed short-term power spectrum of noisy speech recorded over cellular telephone communications. Informal listenings indicate that the technique brings a noticeable improvement to the quality of processed noisy speech while not causing any significant degradation to clean speech. Alternative filter structures are being investigated as well as other potential applications in cellular channel compensation and narrowband to wideband speech mapping.

State-Dependent Riccati Equation Control for Small Autonomous Helicopters

DOI: 10.2514/1.21910 This paper presents a flight control approach based on a state-dependent Riccati equation and its application to autonomous helicopters. For our experiments, we used two different platforms: an XCell-90 small hobby helicopter and a larger vehicle based on the Yamaha R-Max. The control design uses a six-degree-of-freedom nonlinear dynamicmodelthatismanipulated into apseudolinearformwheresystemmatricesare givenexplicitly asafunction of the current state. A standard Riccati equation is then solved numerically at each step of a 50 Hz control loop to design the nonlinear state feedback control law online. In addition, the state-dependent Riccati equation control is augmented with a nonlinear compensator that addresses issues with the mismatch between the original nonlinear dynamics and its pseudolinear transformation.