Sahand Negahban

A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers

High-dimensional statistical inference deals with models in which the the number of parameters p is comparable to or larger than the sample size n. Since it is usually impossible to obtain consistent procedures unless p/n → 0, a line of recent work has studied models with various types of structure (e.g., sparse vectors; block-structured matrices; low-rank matrices; Markov assumptions). In such settings, a general approach to estimation is to solve a regularized convex program (known as a regularized M-estimator) which combines a loss function (measuring how well the model fits the data) with some regularization function that encourages the assumed structure. The goal of this paper is to provide a unified framework for establishing consistency and convergence rates for such regularized M-estimators under high-dimensional scaling. We state one main theorem and show how it can be used to re-derive several existing results, and also to obtain several new results on consistency and convergence rates. Our analysis also identifies two key properties of loss and regularization functions, referred to as restricted strong convexity and decomposability, that ensure the corresponding regularized M-estimators have fast convergence rates.

Fast global convergence of gradient methods for high-dimensional statistical recovery

Many statistical

Simultaneous Support Recovery in High Dimensions: Benefits and Perils of Block

M

\ell _{1}/\ell _{\infty}

-estimators are based on convex optimization problems formed by the combination of a data-dependent loss function with a norm-based regularizer. We analyze the convergence rates of projected gradient and composite gradient methods for solving such problems, working within a high-dimensional framework that allows the data dimension

-Regularization

\pdim

Using machine learning for discovery in synoptic survey imaging data

to grow with (and possibly exceed) the sample size

Understanding adversarial training: Increasing local stability of supervised models through robust optimization

\numobs

Rank centrality: Ranking from pairwise comparisons

. This high-dimensional structure precludes the usual global assumptions---namely, strong convexity and smoothness conditions---that underlie much of classical optimization analysis. We define appropriately restricted versions of these conditions, and show that they are satisfied with high probability for various statistical models. Under these conditions, our theory guarantees that projected gradient descent has a globally geometric rate of convergence up to the \emph{statistical precision} of the model, meaning the typical distance between the true unknown parameter

Analysis of Machine Learning Techniques for Heart Failure Readmissions

\theta^*

Individualized rank aggregation using nuclear norm regularization

and an optimal solution