Nikolaos Nasios

Variational learning for Gaussian mixture models

This paper proposes a joint maximum likelihood and Bayesian methodology for estimating Gaussian mixture models. In Bayesian inference, the distributions of parameters are modeled, characterized by hyperparameters. In the case of Gaussian mixtures, the distributions of parameters are considered as Gaussian for the mean, Wishart for the covariance, and Dirichlet for the mixing probability. The learning task consists of estimating the hyperparameters characterizing these distributions. The integration in the parameter space is decoupled using an unsupervised variational methodology entitled variational expectation-maximization (VEM). This paper introduces a hyperparameter initialization procedure for the training algorithm. In the first stage, distributions of parameters resulting from successive runs of the expectation-maximization algorithm are formed. Afterward, maximum-likelihood estimators are applied to find appropriate initial values for the hyperparameters. The proposed initialization provides faster convergence, more accurate hyperparameter estimates, and better generalization for the VEM training algorithm. The proposed methodology is applied in blind signal detection and in color image segmentation

Kernel Bandwidth Estimation for Nonparametric Modeling

Kernel density estimation is a nonparametric procedure for probability density modeling, which has found several applications in various fields. The smoothness and modeling ability of the functional approximation are controlled by the kernel bandwidth. In this paper, we describe a Bayesian estimation method for finding the bandwidth from a given data set. The proposed bandwidth estimation method is applied in three different computational-intelligence methods that rely on kernel density estimation: 1) scale space; 2) mean shift; and 3) quantum clustering. The third method is a novel approach that relies on the principles of quantum mechanics. This method is based on the analogy between data samples and quantum particles and uses the Schrodinger potential as a cost function. The proposed methodology is used for blind-source separation of modulated signals and for terrain segmentation based on topography information.

Kernel-based classification using quantum mechanics

This paper introduces a new nonparametric estimation approach inspired from quantum mechanics. Kernel density estimation associates a function to each data sample. In classical kernel estimation theory the probability density function is calculated by summing up all the kernels. The proposed approach assumes that each data sample is associated with a quantum physics particle that has a radial activation field around it. Schrodinger differential equation is used in quantum mechanics to define locations of particles given their observed energy level. In our approach, we consider the known location of each data sample and we model their corresponding probability density function using the analogy with the quantum potential function. The kernel scale is estimated from distributions of K-nearest neighbours statistics. In order to apply the proposed algorithm to pattern classification we use the local Hessian for detecting the modes in the quantum potential hypersurface. Each mode is assimilated with a nonparametric class which is defined by means of a region growing algorithm. We apply the proposed algorithm on artificial data and for the topography segmentation from radar images of terrain.

Nonparametric clustering using quantum mechanics

This paper introduces a new nonparametric estimation approach that can be used for data that is not necessarily Gaussian distributed. The proposed approach employs the Shrodinger partial differential equation. We assume that each data sample is associated with a quantum physics particle that has a radial field around its value. We consider a statistical estimation approach for finding the size of the influence field around each data sample. By implementing the Shrodinger equation we obtain a potential field that is assimilated with the data density. The regions of minima in the potential are determined by calculating the local Hessian on the potential hypersurface. The quantum clustering approach is applied for blind separation of signals and for segmenting SAR images of terrain based on surface normal orientation.

Blind Source Separation Using Variational Expectation-Maximization Algorithm

In this paper we suggest a new variational Bayesian approach. Variational Expectation-Maximization (VEM) algorithm is proposed in order to estimate a set of hyperparameters modelling distributions of parameters characterizing mixtures of Gaussians. We consider maximum log-likelihood (ML) estimation for the initialization of the hyperparameters. The ML estimation is employed on distributions of parameters obtained from successive runs of the EM algorithm on the same data set. The proposed algorithm is used for unsupervised detection of quadrature amplitude and phase-shift-key modulated signals.

Variational expectation-maximization training for Gaussian networks

This paper introduces variational expectation-maximization (VEM) algorithm for training Gaussian networks. Hyperparameters model distributions of parameters characterizing Gaussian mixture densities. The proposed algorithm employs a hierarchical learning strategy for estimating a set of hyperparameters and the number of Gaussian mixture components. A dual EM algorithm is employed as the initialization stage in the VEM-based learning. In the first stage the EM algorithm is applied on the given data set while the second stage EM is used on distributions of parameters resulted from several runs of the first stage EM. Appropriate maximum log-likelihood estimators are considered for all the parameter distributions involved.

Variational segmentation of color images

A variational Bayesian framework is employed in the paper for image segmentation using color clustering. A Gaussian mixture model is used to represent color distributions. Variational expectation-maximization (VEM) algorithm takes into account the uncertainty in the parameter estimation ensuring a lower bound on the approximation error. In the variational Bayesian approach we integrate over distributions of parameters. The processing task in this case consists of estimating the hyperparameters of these distributions. We propose a maximum log-likelihood initialization approach for the variational expectation-maximization (VEM) algorithm. The proposed algorithm is applied to image segmentation using color clustering when representing the images in the L*u*v color coordinate system.

A variational approach for color image segmentation

In this paper, we use a variational Bayesian framework for color image segmentation. Each image is represented in the Luv color coordinate system before being segmented by the variational algorithm. The model chosen to describe the color images is a Gaussian mixture model. The parameter estimation uses variational learning by taking into account the uncertainty in parameter estimation. In the variational Bayesian approach, we integrate over distributions of parameters. We propose a maximum log-likelihood initialization approach for the variational expectation-maximization (VEM) algorithm and we apply it to color image segmentation. The segmentation task in our approach consists of the estimation of the distribution hyperparameters.

Bayesian Estimation of Kernel Bandwidth for Nonparametric Modelling

Kernel density estimation (KDE) has been used in many computational intelligence and computer vision applications. In this paper we propose a Bayesian estimation method for finding the bandwidth in KDE applications. A Gamma density function is fitted to distributions of variances of K-nearest neighbours data populations while uniform distribution priors are assumed for K. A maximum log-likelihood approach is used to estimate the parameters of the Gamma distribution when fitted to the local data variance. The proposed methodology is applied in three different KDE approaches: kernel sum, mean shift and quantum clustering. The third method relies on the Schrodinger partial differential equation and uses the analogy between the potential function that manifests around particles, as defined in quantum physics, and the probability density function corresponding to data. The proposed algorithm is applied to artificial data and to segment terrain images.

Kernel bandwidth estimation in methods based on probability density function modelling

In kernel density estimation methods, an approximation of the data probability density function is achieved by locating a kernel function at each data location. The smoothness of the functional approximation and the modelling ability are controlled by the kernel bandwidth. In this paper we propose a Bayesian estimation method for finding the kernel bandwidth. The distribution corresponding to the bandwidth is estimated from distributions characterizing the second order statistics estimates calculated from local neighbourhoods. The proposed bandwidth estimation method is applied in three different kernel density estimation based approaches: scale space, mean shift and quantum clustering. The third method is a novel pattern recognition approach using the principles of quantum mechanics.