Lawrence Carin

Sparse multinomial logistic regression: fast algorithms and generalization bounds

Recently developed methods for learning sparse classifiers are among the state-of-the-art in supervised learning. These methods learn classifiers that incorporate weighted sums of basis functions with sparsity-promoting priors encouraging the weight estimates to be either significantly large or exactly zero. From a learning-theoretic perspective, these methods control the capacity of the learned classifier by minimizing the number of basis functions used, resulting in better generalization. This paper presents three contributions related to learning sparse classifiers. First, we introduce a true multiclass formulation based on multinomial logistic regression. Second, by combining a bound optimization approach with a component-wise update procedure, we derive fast exact algorithms for learning sparse multiclass classifiers that scale favorably in both the number of training samples and the feature dimensionality, making them applicable even to large data sets in high-dimensional feature spaces. To the best of our knowledge, these are the first algorithms to perform exact multinomial logistic regression with a sparsity-promoting prior. Third, we show how nontrivial generalization bounds can be derived for our classifier in the binary case. Experimental results on standard benchmark data sets attest to the accuracy, sparsity, and efficiency of the proposed methods.

Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing

Bayesian compressive sensing (CS) is considered for signals and images that are sparse in a wavelet basis. The statistical structure of the wavelet coefficients is exploited explicitly in the proposed model, and, therefore, this framework goes beyond simply assuming that the data are compressible in a wavelet basis. The structure exploited within the wavelet coefficients is consistent with that used in wavelet-based compression algorithms. A hierarchical Bayesian model is constituted, with efficient inference via Markov chain Monte Carlo (MCMC) sampling. The algorithm is fully developed and demonstrated using several natural images, with performance comparisons to many state-of-the-art compressive-sensing inversion algorithms.

Multitask Compressive Sensing

Compressive sensing (CS) is a framework whereby one performs N nonadaptive measurements to constitute a vector v isin RN used to recover an approximation u isin RM desired signal u isin RM with N << M this is performed under the assumption that u is sparse in the basis represented by the matrix Psi RMtimesM. It has been demonstrated that with appropriate design of the compressive measurements used to define v, the decompressive mapping vrarru may be performed with error ||u-u||2 2 having asymptotic properties analogous to those of the best adaptive transform-coding algorithm applied in the basis Psi. The mapping vrarru constitutes an inverse problem, often solved using l1 regularization or related techniques. In most previous research, if L > 1 sets of compressive measurements {vi}i=1,L are performed, each of the associated {ui}i=1,Lare recovered one at a time, independently. In many applications the L ldquotasksrdquo defined by the mappings virarrui are not statistically independent, and it may be possible to improve the performance of the inversion if statistical interrelationships are exploited. In this paper, we address this problem within a multitask learning setting, wherein the mapping vrarru for each task corresponds to inferring the parameters (here, wavelet coefficients) associated with the desired signal vi, and a shared prior is placed across all of the L tasks. Under this hierarchical Bayesian modeling, data from all L tasks contribute toward inferring a posterior on the hyperparameters, and once the shared prior is thereby inferred, the data from each of the L individual tasks is then employed to estimate the task-dependent wavelet coefficients. An empirical Bayesian procedure for the estimation of hyperparameters is considered; two fast inference algorithms extending the relevance vector machine (RVM) are developed. Example results on several data sets demonstrate the effectiveness and robustness of the proposed algorithms.

Nonparametric Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images

Nonparametric Bayesian methods are considered for recovery of imagery based upon compressive, incomplete, and/or noisy measurements. A truncated beta-Bernoulli process is employed to infer an appropriate dictionary for the data under test and also for image recovery. In the context of compressive sensing, significant improvements in image recovery are manifested using learned dictionaries, relative to using standard orthonormal image expansions. The compressive-measurement projections are also optimized for the learned dictionary. Additionally, we consider simpler (incomplete) measurements, defined by measuring a subset of image pixels, uniformly selected at random. Spatial interrelationships within imagery are exploited through use of the Dirichlet and probit stick-breaking processes. Several example results are presented, with comparisons to other methods in the literature.

Nonparametric factor analysis with beta process priors

We propose a nonparametric extension to the factor analysis problem using a beta process prior. This beta process factor analysis (BP-FA) model allows for a dataset to be decomposed into a linear combination of a sparse set of factors, providing information on the underlying structure of the observations. As with the Dirichlet process, the beta process is a fully Bayesian conjugate prior, which allows for analytical posterior calculation and straightforward inference. We derive a varia-tional Bayes inference algorithm and demonstrate the model on the MNIST digits and HGDP-CEPH cell line panel datasets.

Ultra-Wideband, Short-Pulse Electromagnetics 2

PULSE GENERATION AND DETECTION: Semiconductor Switching: The Time Evolution of Photonic Crystal Bandgaps (K. Agi et al.). General: Ultrawideband Pulser Technology (D.M. Parkes). ANTENNAS: Impulse Radiating Antennas: Transient Fields of Rectangular Aperture Antennas (S.P. Sulkin). Reflector Impulse Radiating Antennas: Temporal and Spectral Radiation on Boresight of a Reflector Type of Impulse Radiating Antenna (IRA) (D.V. Giri, C.E. Baum). Lens Impulse Radiating Antennas and TEM Horns: Design of the Low-Frequency Compensation of an Extreme-Bandwidth TEM Horn and Lens IRA (M.H. Vogel). Arrays: Transient Arrays (C.E> Baum). General: Some Basic Properties of Antennas Associated with Ultra-wideband Radiation (S.N. Samaddar, E.L. Mokole). PULSE PROPOGATION AND GUIDEANCE: Transient Dielectric Coefficient and Conductance in Dielectric Media in Nonstationary Fields (A. Gutman). SCATTERING THEORY, COMPUTATION, AND MEASUREMENTS: Early Time Signature Analysis of Dielectric Targets Using UWB Radar (S. Cloude et al.). SIGNAL PROCESSING: Time-Frequency Analysis: Feature Extraction from Electromagnetic Backscattered Data Using Joint Time-Frequency Processing (L.C. Trintinalia, H. Ling). Spectral Techniques: The E-Pulse Technique for Dispersive Scatterers (S. Primak et al.). Probabilistic Considerations: Robust Target Identification Using a Generalized Likelihood Ratio Test (J.E. Mooney et al.). General: Error Correction in Transient Electromagnetic Field Measurements Using Deconvolution Techniques (J-Z. Bao et al.). BROADBAND ELECTRONIC SYSTEMS AND COMPONENTS: Systems and Components: Ultra-wideband Sources and Antennas: Present Technology, Future Challenges (W.D. Prather et al.). Ultra-Wideband Radars: Dense Media Penetrating Radar (K. Min, M. Willis, Jr.). Polarimetric Ultra-wideband Radars: Polarimetry in Ultra-wideband Interferometric SEnsing and Imaging (W-M. Boerner, J.S. Verdi). BURIED TARGETS: Analytic Methods for Pulsed Signal Interaction with Layered, Lossy Soil Environments and Buried Objects (L.B. Felsen). 41 additional articles. Index.

Bayesian Robust Principal Component Analysis

A hierarchical Bayesian model is considered for decomposing a matrix into low-rank and sparse components, assuming the observed matrix is a superposition of the two. The matrix is assumed noisy, with unknown and possibly non-stationary noise statistics. The Bayesian framework infers an approximate representation for the noise statistics while simultaneously inferring the low-rank and sparse-outlier contributions; the model is robust to a broad range of noise levels, without having to change model hyperparameter settings. In addition, the Bayesian framework allows exploitation of additional structure in the matrix. For example, in video applications each row (or column) corresponds to a video frame, and we introduce a Markov dependency between consecutive rows in the matrix (corresponding to consecutive frames in the video). The properties of this Markov process are also inferred based on the observed matrix, while simultaneously denoising and recovering the low-rank and sparse components. We compare the Bayesian model to a state-of-the-art optimization-based implementation of robust PCA; considering several examples, we demonstrate competitive performance of the proposed model.

Sepsis: An integrated clinico-metabolomic model improves prediction of death in sepsis

A molecular signature, derived from integrated analysis of clinical data, the metabolome, and the proteome in prospective human studies, improved the prediction of death in patients with sepsis, potentially identifying a subset of patients who merit intensive treatment. Understanding Survival of the Fittest in Sepsis Differentiating mild infections from life-threatening ones is a complex decision that is made millions of times a year in U.S. emergency rooms. Should a patient be sent home with antibiotics and chicken soup? Or should he or she be hospitalized for intensive treatment? Sepsis—a serious infection that is associated with a generalized inflammatory response—is one of the leading causes of death. In two prospective clinical studies reported by Langley et al., patients arriving at four urban emergency departments with symptoms of sepsis were evaluated clinically and by analysis of their plasma proteome and metabolome. Survivors and nonsurvivors at 28 days were compared, and a molecular signature was detected that appeared to differentiate these outcomes—even as early as the time of hospital arrival. The signature was part of a large set of differences between these groups, showing that better energy-producing fatty acid catabolism was associated with survival of the fittest in sepsis. A test developed from the signature was able to predict sepsis survival and nonsurvival reproducibly and better than current methods. This test could help to make all important decisions in the emergency room more accurate. Sepsis is a common cause of death, but outcomes in individual patients are difficult to predict. Elucidating the molecular processes that differ between sepsis patients who survive and those who die may permit more appropriate treatments to be deployed. We examined the clinical features and the plasma metabolome and proteome of patients with and without community-acquired sepsis, upon their arrival at hospital emergency departments and 24 hours later. The metabolomes and proteomes of patients at hospital admittance who would ultimately die differed markedly from those of patients who would survive. The different profiles of proteins and metabolites clustered into the following groups: fatty acid transport and β-oxidation, gluconeogenesis, and the citric acid cycle. They differed consistently among several sets of patients, and diverged more as death approached. In contrast, the metabolomes and proteomes of surviving patients with mild sepsis did not differ from survivors with severe sepsis or septic shock. An algorithm derived from clinical features together with measurements of five metabolites predicted patient survival. This algorithm may help to guide the treatment of individual patients with sepsis.

Compressive Sensing on Manifolds Using a Nonparametric Mixture of Factor Analyzers: Algorithm and Performance Bounds

Nonparametric Bayesian methods are employed to constitute a mixture of low-rank Gaussians, for data x ∈ RN that are of high dimension N but are constrained to reside in a low-dimensional subregion of RN. The number of mixture components and their rank are inferred automatically from the data. The resulting algorithm can be used for learning manifolds and for reconstructing signals from manifolds, based on compressive sensing (CS) projection measurements. The statistical CS inversion is performed analytically. We derive the required number of CS random measurements needed for successful reconstruction, based on easily-computed quantities, drawing on block-sparsity properties. The proposed methodology is validated on several synthetic and real datasets.

Temporal Dynamics of Host Molecular Responses Differentiate Symptomatic and Asymptomatic Influenza A Infection

Exposure to influenza viruses is necessary, but not sufficient, for healthy human hosts to develop symptomatic illness. The host response is an important determinant of disease progression. In order to delineate host molecular responses that differentiate symptomatic and asymptomatic Influenza A infection, we inoculated 17 healthy adults with live influenza (H3N2/Wisconsin) and examined changes in host peripheral blood gene expression at 16 timepoints over 132 hours. Here we present distinct transcriptional dynamics of host responses unique to asymptomatic and symptomatic infections. We show that symptomatic hosts invoke, simultaneously, multiple pattern recognition receptors-mediated antiviral and inflammatory responses that may relate to virus-induced oxidative stress. In contrast, asymptomatic subjects tightly regulate these responses and exhibit elevated expression of genes that function in antioxidant responses and cell-mediated responses. We reveal an ab initio molecular signature that strongly correlates to symptomatic clinical disease and biomarkers whose expression patterns best discriminate early from late phases of infection. Our results establish a temporal pattern of host molecular responses that differentiates symptomatic from asymptomatic infections and reveals an asymptomatic host-unique non-passive response signature, suggesting novel putative molecular targets for both prognostic assessment and ameliorative therapeutic intervention in seasonal and pandemic influenza.