Efthymios Tsakonas

Sparse Conjoint Analysis Through Maximum Likelihood Estimation

Conjoint analysis (CA) is a classical tool used in preference assessment, where the objective is to estimate the utility function of an individual, or a group of individuals, based on expressed preference data. An example is choice-based CA for consumer profiling, i.e., unveiling consumer utility functions based solely on choices between products. A statistical model for choice-based CA is investigated in this paper. Unlike recent classification-based approaches, a sparsity-aware Gaussian maximum likelihood (ML) formulation is proposed to estimate the model parameters. Drawing from related robust parsimonious modeling approaches, the model uses sparsity constraints to account for outliers and to detect the salient features that influence decisions. Contributions include conditions for statistical identifiability, derivation of the pertinent Cramér-Rao Lower Bound (CRLB), and ML consistency conditions for the proposed sparse nonlinear model. The proposed ML approach lends itself naturally to ℓ1-type convex relaxations which are well-suited for distributed implementation, based on the alternating direction method of multipliers (ADMM). A particular decomposition is advocated which bypasses the apparent need for outlier communication, thus maintaining scalability. The performance of the proposed ML approach is demonstrated by comparing against the associated CRLB and prior state-of-the-art using both synthetic and real data sets.

Connections between sparse estimation and robust statistical learning

Recent literature on robust statistical inference suggests that promising outlier rejection schemes can be based on accounting explicitly for sparse gross errors in the modeling, and then relying on compressed sensing ideas to perform the outlier detection. In this paper, we consider two models for recovering a sparse signal from noisy measurements, possibly also contaminated with outliers. The models considered here are a linear regression model, and its natural one-bit counterpart where measurements are additionally quantized to a single bit. Our contributions can be summarized as follows: We start by providing conditions for identification and the Cramér-Rao Lower Bounds (CRLBs) for these two models. Then, focusing on the one-bit model, we derive conditions for consistency of the associated Maximum Likelihood estimator, and show the performance of relevant ℓ1-based relaxation strategies by comparing against the theoretical CRLB.

Maximum likelihood based sparse and distributed conjoint analysis

A new statistical model for choice-based conjoint analysis is proposed. The model uses auxiliary variables to account for outliers and to detect the salient features that influence decisions. Unlike recent classification-based approaches to choice-based conjoint analysis, a sparsity-aware maximum likelihood (ML) formulation is proposed to estimate the model parameters. The proposed approach is conceptually appealing, mathematically tractable, and is also well-suited for distributed implementation. Its performance is tested and compared to the prior state-of-art using synthetic as well as real data coming from a conjoint choice experiment for coffee makers, with very promising results.

Signal Processing and Optimization Tools for Conference Review and Session Assignment

Anyone who has served as a technical program committee (TPC) chair for a conference (or program manager for a funding agency) understands that paper (or proposal panel) review assignment is a demanding job that takes a lot of time, and reviewers are rarely satisfied with the end results. This article presents signal processing tools for two critical ?mass assignment? tasks: assigning papers (or proposals) to reviewers in a way that matches reviewing expertise to scientific content while respecting the reviewers? capacity constraints and splitting accepted papers (or submitted proposals) to sessions (panels) while adhering to session (panel) capacity constraints. The basic idea is to use feature vectors to represent papers and reviewers. Features can be key words or phrases (e.g., optimization or sensor networks) or other types of attributes (e.g., timeliness). This viewpoint enables optimal assignment problem formulations that make sense from a scientific and practical point of view. While optimal solutions are hard to compute for a large number of papers and reviewers, high-quality approximate solutions of moderate complexity are developed here using familiar signal processing and optimization tools. These algorithmic solutions easily outperform days of expert manual work as demonstrated in experiments with real conference data.

Line spectrum estimation from broadband power detection bits

Line spectrum estimation from analog signal samples is a classic problem with numerous applications. However, sending analog or finely quantized signal sample streams to a fusion center is a burden in distributed sensing scenarios. Instead, it is appealing to estimate the frequency lines from a few randomly filtered broadband power measurement bits taken using a network of cheap sensors. This leads to a new problem: line spectrum estimation from inequalities. Three different techniques are proposed for this estimation task. In the first two, the autocorrelation function is first estimated nonparametrically, then a parametric method is used to estimate the sought frequencies. The third is a direct maximum likelihood (ML) parameter estimation approach that uses coordinate descent. Simulations show that the underlying frequencies can be accurately estimated using the proposed techniques, even from relatively few bits; and that the ML estimates obtained with the third technique can meet the Cramer-Rao lower bound (also derived here), when the number of sensors is sufficiently large.

Convergence of the Huber Regression M-Estimate in the Presence of Dense Outliers

We consider the problem of estimating a deterministic unknown vector which depends linearly on n noisy measurements, additionally contaminated with (possibly unbounded) additive outliers. The measurement matrix of the model (i.e., the matrix involved in the linear transformation of the sought vector) is assumed known, and comprised of standard Gaussian i.i.d. entries. The outlier variables are assumed independent of the measurement matrix, deterministic or random with possibly unknown distribution. Under these assumptions we provide a simple proof that the minimizer of the Huber penalty function of the residuals converges to the true parameter vector with a √n-rate, even when outliers are dense, in the sense that there is a constant linear fraction of contaminated measurements which can be arbitrarily close to one. The constants influencing the rate of convergence are shown to explicitly depend on the outlier contamination level.

Robust binary least squares: Relaxations and algorithms

Finding the least squares (LS) solution s to a system of linear equations Hs = y where H, y are given and s is a vector of binary variables, is a well known NP-hard problem. In this paper, we consider binary LS problems under the assumption that the coefficient matrix H is also unknown, and lies in a given uncertainty ellipsoid. We show that the corresponding worst-case robust optimization problem, although NP-hard, is still amenable to semidefinite relaxation (SDR)-based approximations. However, the relaxation step is not obvious, and requires a certain problem reformulation to be efficient. The proposed relaxation is motivated using Lagrangian duality and simulations suggest that it performs well, offering a robust alternative over the traditional SDR approaches for binary LS problems.

Time-Frequency Analysis using Particle Filtering: Closed-Form Optimal Importance Function and Sampling Procedure for a Single Time-Varying Harmonic

We consider the problem of tracking the frequency and complex amplitude of a time-varying (TV) harmonic signal using particle filtering (PF) tools. Similar to previous PF approaches to TV spectral analysis, we assume that the frequency and complex amplitude evolve according to a Gaussian AR(1) model; but we concentrate on the important special case of a single TV harmonic. For this case, we show that the optimal importance function (that minimizes the variance of the particle weights) can be computed in closed form. We also develop a suitable procedure to sample from the optimal importance function. The end result is a custom PF solution that is more efficient than generic ones, and can be used in a broad range of important applications that postulate a single TV harmonic component, e.g., TV Doppler estimation in communications and radar.