Michalis K. Titsias

Bayesian feature and model selection for Gaussian mixture models

We present a Bayesian method for mixture model training that simultaneously treats the feature selection and the model selection problem. The method is based on the integration of a mixture model formulation that takes into account the saliency of the features and a Bayesian approach to mixture learning that can be used to estimate the number of mixture components. The proposed learning algorithm follows the variational framework and can simultaneously optimize over the number of components, the saliency of the features, and the parameters of the mixture model. Experimental results using high-dimensional artificial and real data illustrate the effectiveness of the method.

Unsupervised learning of multiple aspects of moving objects from video

A popular framework for the interpretation of image sequences is based on the layered model; see e.g. Wang and Adelson [8], Irani et al. [2]. Jojic and Frey [3] provide a generative probabilistic model framework for this task. However, this layered models do not explicitly account for variation due to changes in the pose and self occlusion. In this paper we show that if the motion of the object is large so that different aspects (or views) of the object are visible at different times in the sequence, we can learn appearance models of the different aspects using a mixture modelling approach.

SAMHD1 is mutated recurrently in chronic lymphocytic leukemia and is involved in response to DNA damage

SAMHD1 is a deoxynucleoside triphosphate triphosphohydrolase and a nuclease that restricts HIV-1 in noncycling cells. Germ-line mutations in SAMHD1 have been described in patients with Aicardi-Goutières syndrome (AGS), a congenital autoimmune disease. In a previous longitudinal whole genome sequencing study of chronic lymphocytic leukemia (CLL), we revealed a SAMHD1 mutation as a potential founding event. Here, we describe an AGS patient carrying a pathogenic germ-line SAMHD1 mutation who developed CLL at 24 years of age. Using clinical trial samples, we show that acquired SAMHD1 mutations are associated with high variant allele frequency and reduced SAMHD1 expression and occur in 11% of relapsed/refractory CLL patients. We provide evidence that SAMHD1 regulates cell proliferation and survival and engages in specific protein interactions in response to DNA damage. We propose that SAMHD1 may have a function in DNA repair and that the presence of SAMHD1 mutations in CLL promotes leukemia development.

Greedy learning of multiple objects in images using robust statistics and factorial learning

We consider data that are images containing views of multiple objects. Our task is to learn about each of the objects present in the images. This task can be approached as a factorial learning problem, where each image must be explained by instantiating a model for each of the objects present with the correct instantiation parameters. A major problem with learning a factorial model is that as the number of objects increases, there is a combinatorial explosion of the number of configurations that need to be considered. We develop a method to extract object models sequentially from the data by making use of a robust statistical method, thus avoiding the combinatorial explosion, and present results showing successful extraction of objects from real images.

Shared kernel models for class conditional density estimation

We present probabilistic models which are suitable for class conditional density estimation and can be regarded as shared kernel models where sharing means that each kernel may contribute to the estimation of the conditional densities of an classes. We first propose a model that constitutes an adaptation of the classical radial basis function (RBF) network (with full sharing of kernels among classes) where the outputs represent class conditional densities. In the opposite direction is the approach of separate mixtures model where the density of each class is estimated using a separate mixture density (no sharing of kernels among classes). We present a general model that allows for the expression of intermediate cases where the degree of kernel sharing can be specified through an extra model parameter. This general model encompasses both the above mentioned models as special cases. In all proposed models the training process is treated as a maximum likelihood problem and expectation-maximization algorithms have been derived for adjusting the model parameters.

Retrieval of Biophysical Parameters With Heteroscedastic Gaussian Processes

An accurate estimation of biophysical variables is the key to monitor our Planet. Leaf chlorophyll content helps in interpreting the chlorophyll fluorescence signal from space, whereas oceanic chlorophyll concentration allows us to quantify the healthiness of the oceans. Recently, the family of Bayesian nonparametric methods has provided excellent results in these situations. A particularly useful method in this framework is the Gaussian process regression (GPR). However, standard GPR assumes that the variance of the noise process is independent of the signal, which does not hold in most of the problems. In this letter, we propose a nonstandard variational approximation that allows accurate inference in signal-dependent noise scenarios. We show that the so-called variational heteroscedastic GPR (VHGPR) is an excellent alternative to standard GPR in two relevant Earth observation examples, namely, Chl vegetation retrieval from hyperspectral images and oceanic Chl concentration estimation from in situ measured reflectances. The proposed VHGPR outperforms the tested empirical approaches, as well as statistical linear regression (both least squares and least absolute shrinkage and selection operator), neural nets, and kernel ridge regression, and the homoscedastic GPR, in terms of accuracy and bias, and proves more robust when a low number of examples is available.

Mixture of experts classification using a hierarchical mixture model

A three-level hierarchical mixture model for classification is presented that models the following data generation process: (1) the data are generated by a finite number of sources (clusters), and (2) the generation mechanism of each source assumes the existence of individual internal class-labeled sources (subclusters of the external cluster). The model estimates the posterior probability of class membership similar to a mixture of experts classifier. In order to learn the parameters of the model, we have developed a general training approach based on maximum likelihood that results in two efficient training algorithms. Compared to other classification mixture models, the proposed hierarchical model exhibits several advantages and provides improved classification performance as indicated by the experimental results.

Identifying targets of multiple co-regulating transcription factors from expression time-series by Bayesian model comparison

BackgroundComplete transcriptional regulatory network inference is a huge challenge because of the complexity of the network and sparsity of available data. One approach to make it more manageable is to focus on the inference of context-specific networks involving a few interacting transcription factors (TFs) and all of their target genes.ResultsWe present a computational framework for Bayesian statistical inference of target genes of multiple interacting TFs from high-throughput gene expression time-series data. We use ordinary differential equation models that describe transcription of target genes taking into account combinatorial regulation. The method consists of a training and a prediction phase. During the training phase we infer the unobserved TF protein concentrations on a subnetwork of approximately known regulatory structure. During the prediction phase we apply Bayesian model selection on a genome-wide scale and score all alternative regulatory structures for each target gene. We use our methodology to identify targets of five TFs regulating Drosophila melanogaster mesoderm development. We find that confident predicted links between TFs and targets are significantly enriched for supporting ChIP-chip binding events and annotated TF-gene interations. Our method statistically significantly outperforms existing alternatives.ConclusionsOur results show that it is possible to infer regulatory links between multiple interacting TFs and their target genes even from a single relatively short time series and in presence of unmodelled confounders and unreliable prior knowledge on training network connectivity. Introducing data from several different experimental perturbations significantly increases the accuracy.

A probabilistic RBF network for classification

We present a probabilistic neural network model which is suitable for classification problems. This model constitutes an adaptation of the classical RBF network where the outputs represent the class conditional distributions. Since the network outputs correspond to probability density functions, training process is treated as maximum likelihood problem and an expectation-maximization (EM) algorithm is proposed for adjusting the network parameters. Experimental results show that proposed architecture exhibits superior classification performance compared to the classical RBF network.

Class conditional density estimation using mixtures with constrained component sharing

We propose a generative mixture model classifier that allows for the class conditional densities to be represented by mixtures having certain subsets of their components shared or common among classes. We argue that, when the total number of mixture components is kept fixed, the most efficient classification model is obtained by appropriately determining the sharing of components among class conditional densities. In order to discover such an efficient model, a training method is derived based on the EM algorithm that automatically adjusts component sharing. We provide experimental results with good classification performance.