Henrik Ohlsson

On the estimation of transfer functions, regularizations and Gaussian processes-Revisited

Intrigued by some recent results on impulse response estimation by kernel and nonparametric techniques, we revisit the old problem of transfer function estimation from input-output measurements. We formulate a classical regularization approach, focused on finite impulse response (FIR) models, and find that regularization is necessary to cope with the high variance problem. This basic, regularized least squares approach is then a focal point for interpreting other techniques, like Bayesian inference and Gaussian process regression. The main issue is how to determine a suitable regularization matrix (Bayesian prior or kernel). Several regularization matrices are provided and numerically evaluated on a data bank of test systems and data sets. Our findings based on the data bank are as follows. The classical regularization approach with carefully chosen regularization matrices shows slightly better accuracy and clearly better robustness in estimating the impulse response than the standard approach-the prediction error method/maximum likelihood (PEM/ML) approach. If the goal is to estimate a model of given order as well as possible, a low order model is often better estimated by the PEM/ML approach, and a higher order model is often better estimated by model reduction on a high order regularized FIR model estimated with careful regularization. Moreover, an optimal regularization matrix that minimizes the mean square error matrix is derived and studied. The importance of this result lies in that it gives the theoretical upper bound on the accuracy that can be achieved for this classical regularization approach.

Compressive Phase Retrieval From Squared Output Measurements Via Semidefinite Programming

Given a linear system in a real or complex domain, linear regression aims to recover the model parameters from a set of observations. Recent studies in compressive sensing have successfully shown that under certain conditions, a linear program, namely, l1-minimization, guarantees recovery of sparse parameter signals even when the system is underdetermined. In this paper, we consider a more challenging problem: when the phase of the output measurements from a linear system is omitted. Using a lifting technique, we show that even though the phase information is missing, the sparse signal can be recovered exactly by solving a semidefinite program when the sampling rate is sufficiently high. This is an interesting finding since the exact solutions to both sparse signal recovery and phase retrieval are combinatorial. The results extend the type of applications that compressive sensing can be applied to those where only output magnitudes can be observed. We demonstrate the accuracy of the algorithms through extensive simulation and a practical experiment.

Improved multiple constant multiplication using a minimum spanning tree

Recently, a novel technique for the multiple constant multiplication (MCM) problem using minimum spanning trees (MSTs) has been proposed. The approach works by finding simple differences between the coefficients to realize and then applying the same method to the differences (which is an MCM problem as well). Each iteration is divided into two steps. First, finding a minimum spanning tree in the graph describing the differences between the coefficients. Second, as each edge in the graph may correspond to more than one difference, one difference is selected for each edge in the MST. Generally, both these stages have multiple solutions. The aim of this work is to more closely study how the MST and the differences should be selected to give better total results. It is also discussed how the two stages in each iteration may be joined into one problem.

On conditions for uniqueness in sparse phase retrieval

The phase retrieval problem has a long history and is an important problem in many areas of optics. Theoretical understanding of phase retrieval is still limited and fundamental questions such as uniqueness and stability of the recovered solution are not yet fully understood. This paper provides several additions to the theoretical understanding of sparse phase retrieval. In particular we show that if the measurement ensemble can be chosen freely, as few as 4k - 1 phaseless measurements suffice to guarantee uniqueness of a k-sparse M-dimensional real solution. We also prove that 2(k2 - k + 1) Fourier magnitude measurements are sufficient under rather general conditions.

Clustering using sum-of-norms regularization: With application to particle filter output computation

We present a novel clustering method, formulated as a convex optimization problem. The method is based on over-parameterization and uses a sum-of-norms (SON) regularization to control the tradeoff between the model fit and the number of clusters. Hence, the number of clusters can be automatically adapted to best describe the data, and need not to be specified a priori. We apply SON clustering to cluster the particles in a particle filter, an application where the number of clusters is often unknown and time varying, making SON clustering an attractive alternative.

Kernel selection in linear system identification part II: A classical perspective

In this companion paper, the choice of kernels for estimating the impulse response of linear stable systems is considered from a classical, “frequentist”, point of view. The kernel determines the regularization matrix in a regularized least squares estimate of an FIR model. The quality is assessed from a mean square error (MSE) perspective, and measures and algorithms for optimizing the MSE are discussed. The ideas are tested on the same data bank as used in Part I of the companion papers. The resulting findings and conclusions in the two papers are very similar despite the different perspectives.

On the Estimation of Transfer Functions, Regularizations and Gaussian Processes - Revisited

Intrigued by some recent results on impulse response estimation by kernel and nonparametric techniques, we revisit the old problem of transfer function estimation from input-output measurements.We ...

Minimum-adder integer multipliers using carry-save adders

In this paper we investigate graph-based minimum-adder integer multipliers using carry-save adders. The previously proposed approaches use carry-propagation adders with two inputs and one output and are not suitable for carry-save adder implementation when we have a single input and a carry-save output of the multiplier. Using carry-save adders avoids carry propagation and results in a higher throughput. We find that mapping from carry-propagation adders to carry-save adders is suboptimal and the multipliers should be designed for carry-save adders directly. Multiplier graphs of up to five adders are presented. Exhaustive search finds that for carry-save adders savings are possible for coefficients with wordlength larger than nine bits. For 19 bits an average saving of over 10% is obtained.

Fundamental limits of nonintrusive load monitoring

Provided an arbitrary nonintrusive load monitoring (NILM) algorithm, we seek bounds on the probability of distinguishing between scenarios, given an aggregate power consumption signal. We introduce a framework for studying a general NILM algorithm, and analyze the theory in the general case. Then, we specialize to the case where the error is Gaussian. In both cases, we are able to derive upper bounds on the probability of distinguishing scenarios. Finally, we apply the results to real data to derive bounds on the probability of distinguishing between scenarios as a function of the measurement noise, the sampling rate, and the device usage.

Trajectory generation using sum-of-norms regularization

Many tracking problems are split into two sub-problems, first a smooth reference trajectory is generated that meet the control design objectives, and then a closed loop control system is designed to follow this reference trajectory as well as possible. Applications of this kind include (autonomous) vehicle navigation systems and robotics. Typically, a spline model is used for trajectory generation and another physical and dynamical model is used for the control design. Here we propose a direct approach where the dynamical model is used to generate a control signal that takes the state trajectory through the waypoints specified in the design goals. The strength of the proposed formulation is the methodology to obtain a control signal with compact representation and that changes only when needed, something often wanted in tracking. The formulation takes the shape of a constrained least-squares problem with sum-of-norms regularization, a generalization of the ℓ1-regularization. The formulation also gives a tool to, e.g. in model predictive control, prevent chatter in the input signal, and also select the most suitable instances for applying the control inputs.