Madeleine Udell

Generalized Low Rank Models

Principal components analysis (PCA) is a well-known technique for approximating a tabular data set by a low rank matrix. Here, we extend the idea of PCA to handle arbitrary data sets consisting of numerical, Boolean, categorical, ordinal, and other data types. This framework encompasses many well-known techniques in data analysis, such as nonnegative matrix factorization, matrix completion, sparse and robust PCA, k-means, k-SVD, and maximum margin matrix factorization. The method handles heterogeneous data sets, and leads to coherent schemes for compressing, denoising, and imputing missing entries across all data types simultaneously. It also admits a number of interesting interpretations of the low rank factors, which allow clustering of examples or of features. We propose several parallel algorithms for fitting generalized low rank models, and describe implementations and numerical results.

Convex optimization in Julia

This paper describes Convex1, a convex optimization modeling framework in Julia. Convex translates problems from a user-friendly functional language into an abstract syntax tree describing the problem. This concise representation of the global structure of the problem allows Convex to infer whether the problem complies with the rules of disciplined convex programming (DCP), and to pass the problem to a suitable solver. These operations are carried out in Julia using multiple dispatch, which dramatically reduces the time required to verify DCP compliance and to parse a problem into conic form. Convex then automatically chooses an appropriate backend solver to solve the conic form problem.

Incorporation of flexible objectives and time-linked simulation with flux balance analysis.

We present two modifications of the flux balance analysis (FBA) metabolic modeling framework which relax implicit assumptions of the biomass reaction. Our flexible flux balance analysis (flexFBA) objective removes the fixed proportion between reactants, and can therefore produce a subset of biomass reactants. Our time-linked flux balance analysis (tFBA) simulation removes the fixed proportion between reactants and byproducts, and can therefore describe transitions between metabolic steady states. Used together, flexFBA and tFBA model a time scale shorter than the regulatory and growth steady state encoded by the biomass reaction. This combined short-time FBA method is intended for integrated modeling applications to enable detailed and dynamic depictions of microbial physiology such as whole-cell modeling. For example, when modeling Escherichia coli, it avoids artifacts caused by low-copy-number enzymes in single-cell models with kinetic bounds. Even outside integrated modeling contexts, the detailed predictions of flexFBA and tFBA complement existing FBA techniques. We show detailed metabolite production of in silico knockouts used to identify when correct essentiality predictions are made for the wrong reason.

Factorization for analog-to-digital matrix multiplication

We present matrix factorization as an enabling technique for analog-to-digital matrix multiplication (AD-MM). We show that factorization in the analog domain increases the total precision of AD-MM in precision-limited analog multiplication, reduces the number of analog-to-digital (A/D) conversions needed for overcomplete matrices, and avoids unneeded computations in the digital domain. Finally, we present a factorization algorithm using alternating convex relaxation.

Revealed Preference at Scale: Learning Personalized Preferences from Assortment Choices

We consider the problem of learning the preferences of a heterogeneous population by observing choices from an assortment of products, ads, or other offerings. Our observation model takes a form common in assortment planning applications: each arriving customer is offered an assortment consisting of a subset of all possible offerings; we observe only the assortment and the customers single choice. In this paper we propose a mixture choice model with a natural underlying low-dimensional structure, and show how to estimate its parameters. In our model, the preferences of each customer or segment follow a separate parametric choice model, but the underlying structure of these parameters over all the models has low dimension. We show that a nuclear-norm regularized maximum likelihood estimator can learn the preferences of all customers using a number of observations much smaller than the number of item-customer combinations. This result shows the potential for structural assumptions to speed up learning and improve revenues in assortment planning and customization. We provide a specialized factored gradient descent algorithm and study the success of the approach empirically.

Disciplined multi-convex programming

A multi-convex optimization problem is one in which the variables can be partitioned into sets over which the problem is convex when the other variables are fixed. Multi-convex problems are generally solved approximately using variations on alternating or cyclic minimization. Multi-convex problems arise in many applications, such as nonnegative matrix factorization, generalized low rank models, and structured control synthesis, to name just a few. In most applications to date the multi-convexity is simple to verify by hand. In this paper we study the automatic detection and verification of multi-convexity using the ideas of disciplined convex programming. We describe an implementation of our proposed method that detects and verifies multi-convexity, and then invokes one of the general solution methods.

Practical Sketching Algorithms for Low-Rank Matrix Approximation

This paper describes a suite of algorithms for constructing low-rank approximations of an input matrix from a random linear image, or sketch, of the matrix. These methods can preserve structural properties of the input matrix, such as positive-semidefiniteness, and they can produce approximations with a user-specified rank. The algorithms are simple, accurate, numerically stable, and provably correct. Moreover, each method is accompanied by an informative error bound that allows users to select parameters a priori to achieve a given approximation quality. These claims are supported by numerical experiments with real and synthetic data.

DISCOVERING PATIENT PHENOTYPES USING GENERALIZED LOW RANK MODELS.

The practice of medicine is predicated on discovering commonalities or distinguishing characteristics among patients to inform corresponding treatment. Given a patient grouping (hereafter referred to as a phenotype), clinicians can implement a treatment pathway accounting for the underlying cause of disease in that phenotype. Traditionally, phenotypes have been discovered by intuition, experience in practice, and advancements in basic science, but these approaches are often heuristic, labor intensive, and can take decades to produce actionable knowledge. Although our understanding of disease has progressed substantially in the past century, there are still important domains in which our phenotypes are murky, such as in behavioral health or in hospital settings. To accelerate phenotype discovery, researchers have used machine learning to find patterns in electronic health records, but have often been thwarted by missing data, sparsity, and data heterogeneity. In this study, we use a flexible framework called Generalized Low Rank Modeling (GLRM) to overcome these barriers and discover phenotypes in two sources of patient data. First, we analyze data from the 2010 Healthcare Cost and Utilization Project National Inpatient Sample (NIS), which contains upwards of 8 million hospitalization records consisting of administrative codes and demographic information. Second, we analyze a small (N=1746), local dataset documenting the clinical progression of autism spectrum disorder patients using granular features from the electronic health record, including text from physician notes. We demonstrate that low rank modeling successfully captures known and putative phenotypes in these vastly different datasets.

Graph-Regularized Generalized Low-Rank Models

Image data is frequently extremely large and oftentimes pixel values are occluded or observed with noise. Additionally, images can be related to each other, as in images of a particular individual. This method augments the recently proposed Generalized Low Rank Model (GLRM) framework with graph regularization, which flexibly models relationships between images. For example, relationships might include images that change slowly over time (as in video or surveillance data), images of the same individual, or diagnostic images which picture the same medical condition. This paper proposes a fast optimization method to solve these graph-regularized GLRMs, which we have released as an open-source software library. We demonstrate that the method outperforms competing methods on a variety of data sets, and show how to use this method to classify and group images.