Marcelo Pereyra

Estimating the Granularity Coefficient of a Potts-Markov Random Field Within a Markov Chain Monte Carlo Algorithm

This paper addresses the problem of estimating the Potts parameter β jointly with the unknown parameters of a Bayesian model within a Markov chain Monte Carlo (MCMC) algorithm. Standard MCMC methods cannot be applied to this problem because performing inference on β requires computing the intractable normalizing constant of the Potts model. In the proposed MCMC method, the estimation of β is conducted using a likelihood-free Metropolis-Hastings algorithm. Experimental results obtained for synthetic data show that estimating β jointly with the other unknown parameters leads to estimation results that are as good as those obtained with the actual value of β. On the other hand, choosing an incorrect value of β can degrade estimation performance significantly. To illustrate the interest of this method, the proposed algorithm is successfully applied to real bidimensional SAR and tridimensional ultrasound images.

Bayesian computation: a summary of the current state, and samples backwards and forwards

Recent decades have seen enormous improvements in computational inference for statistical models; there have been competitive continual enhancements in a wide range of computational tools. In Bayesian inference, first and foremost, MCMC techniques have continued to evolve, moving from random walk proposals to Langevin drift, to Hamiltonian Monte Carlo, and so on, with both theoretical and algorithmic innovations opening new opportunities to practitioners. However, this impressive evolution in capacity is confronted by an even steeper increase in the complexity of the datasets to be addressed. The difficulties of modelling and then handling ever more complex datasets most likely call for a new type of tool for computational inference that dramatically reduces the dimension and size of the raw data while capturing its essential aspects. Approximate models and algorithms may thus be at the core of the next computational revolution.

Proximal Markov chain Monte Carlo algorithms

This paper presents a new Metropolis-adjusted Langevin algorithm (MALA) that uses convex analysis to simulate efficiently from high-dimensional densities that are log-concave, a class of probability distributions that is widely used in modern high-dimensional statistics and data analysis. The method is based on a new first-order approximation for Langevin diffusions that exploits log-concavity to construct Markov chains with favourable convergence properties. This approximation is closely related to Moreau–Yoshida regularisations for convex functions and uses proximity mappings instead of gradient mappings to approximate the continuous-time process. The proposed method complements existing MALA methods in two ways. First, the method is shown to have very robust stability properties and to converge geometrically for many target densities for which other MALA are not geometric, or only if the step size is sufficiently small. Second, the method can be applied to high-dimensional target densities that are not continuously differentiable, a class of distributions that is increasingly used in image processing and machine learning and that is beyond the scope of existing MALA and HMC algorithms. To use this method it is necessary to compute or to approximate efficiently the proximity mappings of the logarithm of the target density. For several popular models, including many Bayesian models used in modern signal and image processing and machine learning, this can be achieved with convex optimisation algorithms and with approximations based on proximal splitting techniques, which can be implemented in parallel. The proposed method is demonstrated on two challenging high-dimensional and non-differentiable models related to image resolution enhancement and low-rank matrix estimation that are not well addressed by existing MCMC methodology.

Segmentation of Skin Lesions in 2-D and 3-D Ultrasound Images Using a Spatially Coherent Generalized Rayleigh Mixture Model

This paper addresses the problem of jointly estimating the statistical distribution and segmenting lesions in multiple-tissue high-frequency skin ultrasound images. The distribution of multiple-tissue images is modeled as a spatially coherent finite mixture of heavy-tailed Rayleigh distributions. Spatial coherence inherent to biological tissues is modeled by enforcing local dependence between the mixture components. An original Bayesian algorithm combined with a Markov chain Monte Carlo method is then proposed to jointly estimate the mixture parameters and a label-vector associating each voxel to a tissue. More precisely, a hybrid Metropolis-within-Gibbs sampler is used to draw samples that are asymptotically distributed according to the posterior distribution of the Bayesian model. The Bayesian estimators of the model parameters are then computed from the generated samples. Simulation results are conducted on synthetic data to illustrate the performance of the proposed estimation strategy. The method is then successfully applied to the segmentation of in vivo skin tumors in high-frequency 2-D and 3-D ultrasound images.

A Survey of Stochastic Simulation and Optimization Methods in Signal Processing

Modern signal processing (SP) methods rely very heavily on probability and statistics to solve challenging SP problems. SP methods are now expected to deal with ever more complex models, requiring ever more sophisticated computational inference techniques. This has driven the development of statistical SP methods based on stochastic simulation and optimization. Stochastic simulation and optimization algorithms are computationally intensive tools for performing statistical inference in models that are analytically intractable and beyond the scope of deterministic inference methods. They have been recently successfully applied to many difficult problems involving complex statistical models and sophisticated (often Bayesian) statistical inference techniques. This survey paper offers an introduction to stochastic simulation and optimization methods in signal and image processing. The paper addresses a variety of high-dimensional Markov chain Monte Carlo (MCMC) methods as well as deterministic surrogate methods, such as variational Bayes, the Bethe approach, belief and expectation propagation and approximate message passing algorithms. It also discusses a range of optimization methods that have been adopted to solve stochastic problems, as well as stochastic methods for deterministic optimization. Subsequently, areas of overlap between simulation and optimization, in particular optimization-within-MCMC and MCMC-driven optimization are discussed.

Maximum marginal likelihood estimation of the granularity coefficient of a Potts-Markov random field within an MCMC algorithm

This paper addresses the problem of estimating the Potts-Markov random field parameter β jointly with the unknown parameters of a Bayesian image segmentation model. We propose a new adaptive Markov chain Monte Carlo (MCMC) algorithm for performing joint maximum marginal likelihood estimation of β and maximum-a-posteriori unsupervised image segmentation. The method is based on a stochastic gradient adaptation technique whose computational complexity is significantly lower than that of the competing MCMC approaches. This adaptation technique can be easily integrated to existing MCMC methods where β was previously assumed to be known. Experimental results on synthetic data and on a real 3D real image show that the proposed method produces segmentation results that are as good as those obtained with state-of-the-art MCMC methods and at much lower computational cost.

Bayesian Nonlinear Hyperspectral Unmixing With Spatial Residual Component Analysis

This paper presents a new Bayesian model and algorithm for nonlinear unmixing of hyperspectral images. The proposed model represents the pixel reflectances as linear combinations of the endmembers, corrupted by nonlinear (with respect to the endmembers) terms and additive Gaussian noise. Prior knowledge about the problem is embedded in a hierarchical model that describes the dependence structure between the model parameters and their constraints. In particular, a gamma Markov random field is used to model the joint distribution of the nonlinear terms, which are expected to exhibit significant spatial correlations. An adaptive Markov chain Monte Carlo algorithm is then proposed to compute the Bayesian estimates of interest and perform Bayesian inference. This algorithm is equipped with a stochastic optimisation adaptation mechanism that automatically adjusts the parameters of the gamma Markov random field by maximum marginal likelihood estimation. Finally, the proposed methodology is demonstrated through a series of experiments with comparisons using synthetic and real data and with competing state-of-the-art approaches.

Collaborative sparse regression using spatially correlated supports - Application to hyperspectral unmixing

This paper presents a new Bayesian collaborative sparse regression method for linear unmixing of hyperspectral images. Our contribution is twofold; first, we propose a new Bayesian model for structured sparse regression in which the supports of the sparse abundance vectors are a priori spatially correlated across pixels (i.e., materials are spatially organized rather than randomly distributed at a pixel level). This prior information is encoded in the model through a truncated multivariate Ising Markov random field, which also takes into consideration the facts that pixels cannot be empty (i.e., there is at least one material present in each pixel), and that different materials may exhibit different degrees of spatial regularity. Second, we propose an advanced Markov chain Monte Carlo algorithm to estimate the posterior probabilities that materials are present or absent in each pixel, and, conditionally to the maximum marginal a posteriori configuration of the support, compute the minimum mean squared error estimates of the abundance vectors. A remarkable property of this algorithm is that it self-adjusts the values of the parameters of the Markov random field, thus relieving practitioners from setting regularization parameters by cross-validation. The performance of the proposed methodology is finally demonstrated through a series of experiments with synthetic and real data and comparisons with other algorithms from the literature.

Maximum-a-posteriori estimation with unknown regularisation parameters

This paper presents two hierarchical Bayesian methods for performing maximum-a-posteriori inference when the value of the regularisation parameter is unknown. The methods are useful for models with homogenous regularisers (i.e., prior sufficient statistics), including all norms, composite norms and compositions of norms with linear operators. A key contribution of this paper is to show that for these models the normalisation factor of the prior has a closed-form analytic expression. This then enables the development of Bayesian inference techniques to either estimate regularisation parameters from the observed data or, alternatively, to remove them from the model by marginalisation followed by inference with the marginalised model. The effectiveness of the proposed methodologies is illustrated on applications to compressive sensing using an l1-wavelet analysis prior, where they outperform a state-of-the-art SURE-based technique, both in terms of estimation accuracy and computing time.

Modeling ultrasound echoes in skin tissues using symmetric α-stable processes

Starting from the widely accepted point-scattering model, this paper establishes, through analytical developments, that ultrasound signals backscattered from skin tissues converge to a complex Levy flight random process with non- Gaussian α-stable statistics. The envelope signal follows a generalized (heavy-tailed) Rayleigh distribution. It is shown that these signal statistics imply that scatterers have heavy-tailed power-law cross sections. This model generalizes the Gaussian framework and provides a formal representation for a new case of non-Gaussian statistics, in which both the number of scatterers and the variance of their cross sections tend to infinity. In addition, analytical expressions are derived to relate the α-stable parameters to scatterer properties. Simulations show that these expressions can be used as rigorous interpretation tools for tissue characterization. Several experimental results supported by excellent goodness-of-fit tests confirm the proposed analytical model. Finally, these fundamental results set the basis for new echography processing methods and quantitative ultrasound characterization tools.