Sivaraman Balakrishnan

Learning generative models for protein fold families

We introduce a new approach to learning statistical models from multiple sequence alignments (MSA) of proteins. Our method, called GREMLIN (Generative REgularized ModeLs of proteINs), learns an undirected probabilistic graphical model of the amino acid composition within the MSA. The resulting model encodes both the position‐specific conservation statistics and the correlated mutation statistics between sequential and long‐range pairs of residues. Existing techniques for learning graphical models from MSA either make strong, and often inappropriate assumptions about the conditional independencies within the MSA (e.g., Hidden Markov Models), or else use suboptimal algorithms to learn the parameters of the model. In contrast, GREMLIN makes no a priori assumptions about the conditional independencies within the MSA. We formulate and solve a convex optimization problem, thus guaranteeing that we find a globally optimal model at convergence. The resulting model is also generative, allowing for the design of new protein sequences that have the same statistical properties as those in the MSA. We perform a detailed analysis of covariation statistics on the extensively studied WW and PDZ domains and show that our method out‐performs an existing algorithm for learning undirected probabilistic graphical models from MSA. We then apply our approach to 71 additional families from the PFAM database and demonstrate that the resulting models significantly out‐perform Hidden Markov Models in terms of predictive accuracy. Proteins 2011;

Statistical guarantees for the EM algorithm: From population to sample-based analysis

We develop a general framework for proving rigorous guarantees on the performance of the EM algorithm and a variant known as gradient EM. Our analysis is divided into two parts: a treatment of these algorithms at the population level (in the limit of infinite data), followed by results that apply to updates based on a finite set of samples. First, we characterize the domain of attraction of any global maximizer of the population likelihood. This characterization is based on a novel view of the EM updates as a perturbed form of likelihood ascent, or in parallel, of the gradient EM updates as a perturbed form of standard gradient ascent. Leveraging this characterization, we then provide non-asymptotic guarantees on the EM and gradient EM algorithms when applied to a finite set of samples. We develop consequences of our general theory for three canonical examples of incomplete-data problems: mixture of Gaussians, mixture of regressions, and linear regression with covariates missing completely at random. In each case, our theory guarantees that with a suitable initialization, a relatively small number of EM (or gradient EM) steps will yield (with high probability) an estimate that is within statistical error of the MLE. We provide simulations to confirm this theoretically predicted behavior.

Stochastically Transitive Models for Pairwise Comparisons: Statistical and Computational Issues

There are various parametric models for analyzing pairwise comparison data, including the Bradley–Terry–Luce (BTL) and Thurstone models, but their reliance on strong parametric assumptions is limiting. In this paper, we study a flexible model for pairwise comparisons, under which the probabilities of outcomes are required only to satisfy a natural form of stochastic transitivity. This class includes parametric models, including the BTL and Thurstone models as special cases, but is considerably more general. We provide various examples of models in this broader stochastically transitive class for which classical parametric models provide poor fits. Despite this greater flexibility, we show that the matrix of probabilities can be estimated at the same rate as in standard parametric models up to logarithmic terms. On the other hand, unlike in the BTL and Thurstone models, computing the minimax-optimal estimator in the stochastically transitive model is non-trivial, and we explore various computationally tractable alternatives. We show that a simple singular value thresholding algorithm is statistically consistent but does not achieve the minimax rate. We then propose and study algorithms that achieve the minimax rate over interesting sub-classes of the full stochastically transitive class. We complement our theoretical results with thorough numerical simulations.

Alternative paths in HIV-1 targeted human signal transduction pathways

BackgroundHuman immunodeficiency virus-1 (HIV-1) has a minimal genome of only 9 genes, which encode 15 proteins. HIV-1 thus depends on the human host for virtually every aspect of its life cycle. The universal language of communication in biological systems, including between pathogen and host, is via signal transduction pathways. The fundamental units of these pathways are protein protein interactions. Understanding the functional significance of HIV-1, human interactions requires viewing them in the context of human signal transduction pathways.ResultsIntegration of HIV-1, human interactions with known signal transduction pathways indicates that the majority of known human pathways have the potential to be effected through at least one interaction with an HIV-1 protein at some point during the HIV-1 life cycle. For each pathway, we define simple paths between start points (i.e. no edges going into a node) and end points (i.e. no edges leaving a node). We then identify the paths that pass through human proteins that interact with HIV-1 proteins. We supplement the combined map with functional information, including which proteins are known drug targets and which proteins contribute significantly to HIV-1 function as revealed by recent siRNA screens. We find that there are often alternative paths starting and ending at the same proteins but circumventing the intermediate steps disrupted by HIV-1.ConclusionA mapping of HIV-1, human interactions to human signal transduction pathways is presented here to link interactions with functions. We proposed a new way of analyzing the virus host interactions by identifying HIV-1 targets as well as alternative paths bypassing the HIV-1 targeted steps. This approach yields numerous experimentally testable hypotheses on how HIV-1 function may be compromised and human cellular function restored by pharmacological approaches. We are making the full set of pathway analysis results available to the community.

Recovering block-structured activations using compressive measurements

We consider the problems of detection and support recovery of a contiguous block of weak activation in a large matrix, from a small number of noisy, possibly adaptive, compressive (linear) measurements. We characterize the tradeoffs between the various problem dimensions, the signal strength and the number of measurements required to reliably detect and recover the support of the signal. In each case sufficient conditions are complemented with information theoretic lower bounds. This is closely related to the problem of (noisy) compressed sensing, where the analogous task is to detect or recover the support of a sparse vector using a small number of linear measurements. In compressed sensing, it has been shown that, at least in a minimax sense, for both detection and support recovery, adaptivity and contiguous structure only reduce signal strength requirements by logarithmic factors. On the contrary, we show that while for detection neither adaptivity nor structure reduce the signal strength requirement, for support recovery the signal strength requirement is strongly influenced by both structure and the ability to choose measurements adaptively.

Feeling the bern: Adaptive estimators for Bernoulli probabilities of pairwise comparisons

We study methods for aggregating pairwise comparison data in order to estimate outcome probabilities for future comparisons. We investigate this problem under a flexible class of models satisfying the strong stochastic transitivity (SST) condition. Prior works have studied the minimax risk for estimation of the pairwise comparison probabilities under the SST model. The minimax risk, however, is a measure of the worst-case risk of an estimator over a large parameter space, and in general provides only a rudimentary understanding of an estimator in problems where the intrinsic difficulty of estimation varies considerably over the parameter space. In this paper, we introduce an adaptivity index, in order to benchmark the performance of an estimator against an oracle estimator. The adaptivity index, in addition to measuring the worst-case risk of an estimator, also captures the extent to which the estimator adapts to the instance-specific difficulty of the underlying problem, relative to an oracle estimator. In the context of this adaptivity index we provide two main results. We propose a three-step, Count-Randomize-Least squares (CRL) estimator, and derive upper bounds on the adaptivity index of this estimator. We complement this result with a complexity-theoretic result, that shows that conditional on the planted clique hardness conjecture, no computationally efficient estimator can achieve a substantially smaller adaptivity index.

Completion of high-rank ultrametric matrices using selective entries

Ultrametric matrices are hierarchically structured matrices that arise naturally in many scenarios, e.g. delay covariance of packets sent from a source to a set of clients in a computer network, interactions between multi-scale communities in a social network, and genome sequence alignment scores in phylogenetic tree reconstruction problems. In this work, we show that it is possible to complete n × n ultrametric matrices using only n log n entries. Since ultrametric matrices are high-rank matrices, our results extend recent work on completion of n×n low-rank matrices that requires n log n randomly sampled entries. In the ultrametric setting, a random sampling of entries does not suffice, and we require selective sampling of entries using feedback obtained from entries observed at a previous stage.

Statistical and computational guarantees for the Baum-Welch algorithm

The Hidden Markov Model (HMM) is one of the main-stays of statistical modeling of discrete time series and is widely used in many applications. Estimating an HMM from its observation process is often addressed via the Baum-Welch algorithm, which performs well empirically when initialized reasonably close to the truth. This behavior could not be explained by existing theory which predicts susceptibility to bad local optima. In this paper we aim at closing the gap and provide a framework to characterize a sufficient basin of attraction for any global optimum in which Baum-Welch is guaranteed to converge linearly to an “optimally” small ball around the global optimum. The framework is then used to determine the linear rate of convergence and a sufficient initialization region for Baum-Welch applied on a two component isotropic hidden Markov mixture of Gaussians.

Communication interception of human signal transduction pathways by Human Immunodeficiency Virus-1

Signal transduction pathways are central to most biological processes. Diversion of such pathways is postulated to be central to the mechanism by which the Human Immunodeficiency Virus-1 (HIV) takes over the human cellular machinery. In this paper, we present an analysis of the interactions between HIV and human signal transduction pathways. We find that the majority of known human pathways are targeted through at least one HIV, human protein interaction (277 of the 453 pathways we considered). There are some pathways in which HIV interacts with disproportionately many proteins, targeting a single pathway at multiple positions. These numerous interactions are not just a function of the size of the pathways; other large pathways are not necessarily targeted to the same extent. Based on this analysis, we propose a novel rational drug design strategy as one of identifying possible “alternate” pathways. Activating or suppressing them may bypass HIV targeted pathways, thus exploiting redundancies in the human protein interaction network.

Biases in Data-Driven Networking, and What to Do About Them

Recent efforts highlight the promise of data-driven approaches to optimize network decisions. Many such efforts use trace-driven evaluation; i.e., running offline analysis on network traces to estimate the potential benefits of different policies before running them in practice. Unfortunately, such frameworks can have fundamental pitfalls (e.g., skews due to previous policies that were used in the data collection phase and insufficient data for specific subpopulations) that could lead to misleading estimates and ultimately suboptimal decisions. In this paper, we shed light on such pitfalls and identify a promising roadmap to address these pitfalls by leveraging parallels in causal inference, namely the Doubly Robust estimator.