Sotirios P. Chatzis

A Fuzzy Clustering Approach Toward Hidden Markov Random Field Models for Enhanced Spatially Constrained Image Segmentation

Hidden Markov random field (HMRF) models have been widely used for image segmentation, as they appear naturally in problems where a spatially constrained clustering scheme, taking into account the mutual influences of neighboring sites, is asked for. Fuzzy c-means (FCM) clustering has also been successfully applied in several image segmentation applications. In this paper, we combine the benefits of these two approaches, by proposing a novel treatment of HMRF models, formulated on the basis of a fuzzy clustering principle. We approach the HMRF model treatment problem as an FCM-type clustering problem, effected by introducing the explicit assumptions of the HMRF model into the fuzzy clustering procedure. Our approach utilizes a fuzzy objective function regularized by Kullback--Leibler divergence information, and is facilitated by application of a mean-field-like approximation of the MRF prior. We experimentally demonstrate the superiority of the proposed approach over competing methodologies, considering a series of synthetic and real-world image segmentation applications.

Robust Sequential Data Modeling Using an Outlier Tolerant Hidden Markov Model

Hidden Markov (chain) models using finite Gaussian mixture models as their hidden state distributions have been successfully applied in sequential data modeling and classification applications. Nevertheless, Gaussian mixture models are well known to be highly intolerant to the presence of untypical data within the fitting data sets used for their estimation. Finite Students t-mixture models have recently emerged as a heavier-tailed, robust alternative to Gaussian mixture models, overcoming these hurdles. To exploit these merits of Students t-mixture models in the context of a sequential data modeling setting, we introduce, in this paper, a novel hidden Markov model where the hidden state distributions are considered to be finite mixtures of multivariate Students t-densities. We derive an algorithm for the model parameters estimation under a maximum likelihood framework, assuming full, diagonal, and factor-analyzed covariance matrices. The advantages of the proposed model over conventional approaches are experimentally demonstrated through a series of sequential data modeling applications.

Echo State Gaussian Process

Echo state networks (ESNs) constitute a novel approach to recurrent neural network (RNN) training, with an RNN (the reservoir) being generated randomly, and only a readout being trained using a simple computationally efficient algorithm. ESNs have greatly facilitated the practical application of RNNs, outperforming classical approaches on a number of benchmark tasks. In this paper, we introduce a novel Bayesian approach toward ESNs, the echo state Gaussian process (ESGP). The ESGP combines the merits of ESNs and Gaussian processes to provide a more robust alternative to conventional reservoir computing networks while also offering a measure of confidence on the generated predictions (in the form of a predictive distribution). We exhibit the merits of our approach in a number of applications, considering both benchmark datasets and real-world applications, where we show that our method offers a significant enhancement in the dynamical data modeling capabilities of ESNs. Additionally, we also show that our method is orders of magnitude more computationally efficient compared to existing Gaussian process-based methods for dynamical data modeling, without compromises in the obtained predictive performance.

A fuzzy c-means-type algorithm for clustering of data with mixed numeric and categorical attributes employing a probabilistic dissimilarity functional

Gath-Geva (GG) algorithm is one of the most popular methodologies for fuzzy c-means (FCM)-type clustering of data comprising numeric attributes; it is based on the assumption of data deriving from clusters of Gaussian form, a much more flexible construction compared to the spherical clusters assumption of the original FCM. In this paper, we introduce an extension of the GG algorithm to allow for the effective handling of data with mixed numeric and categorical attributes. Traditionally, fuzzy clustering of such data is conducted by means of the fuzzy k-prototypes algorithm, which merely consists in the execution of the original FCM algorithm using a different dissimilarity functional, suitable for attributes with mixed numeric and categorical attributes. On the contrary, in this work we provide a novel FCM-type algorithm employing a fully probabilistic dissimilarity functional for handling data with mixed-type attributes. Our approach utilizes a fuzzy objective function regularized by Kullback-Leibler (KL) divergence information, and is formulated on the basis of a set of probabilistic assumptions regarding the form of the derived clusters. We evaluate the efficacy of the proposed approach using benchmark data, and we compare it with competing fuzzy and non-fuzzy clustering algorithms.

Robust Visual Behavior Recognition

In this article, we propose a novel framework for robust visual behavior understanding, capable of achieving high recognition rates in demanding real-life environments and in almost real time. Our approach is based on the utilization of holistic visual behavior understanding methods, which perform modeling directly at the pixel level. This way, we eliminate the world representation layer that can be a significant source of errors for the modeling algorithms. Our proposed system is based on the utilization of information from multiple cameras, aiming to alleviate the effects of occlusions and other similar artifacts, which are rather common in real-life installations. To effectively exploit the acquired information for the purpose of real-time activity recognition, appropriate methodologies for modeling of sequential data stemming from multiple sources are examined. Moreover, we explore the efficacy of the additional application of semisupervised learning methodologies, in an effort to reduce the cost of model training in a completely supervised fashion. The performance of the examined approaches is thoroughly evaluated under real-life visual behavior understanding scenarios, and the obtained results are compared and discussed.

The Infinite Hidden Markov Random Field Model

Hidden Markov random field (HMRF) models are widely used for image segmentation, as they appear naturally in problems where a spatially constrained clustering scheme is asked for. A major limitation of HMRF models concerns the automatic selection of the proper number of their states, i.e., the number of region clusters derived by the image segmentation procedure. Existing methods, including likelihood- or entropy-based criteria, and reversible Markov chain Monte Carlo methods, usually tend to yield noisy model size estimates while imposing heavy computational requirements. Recently, Dirichlet process (DP, infinite) mixture models have emerged in the cornerstone of nonparametric Bayesian statistics as promising candidates for clustering applications where the number of clusters is unknown a priori; infinite mixture models based on the original DP or spatially constrained variants of it have been applied in unsupervised image segmentation applications showing promising results. Under this motivation, to resolve the aforementioned issues of HMRF models, in this paper, we introduce a nonparametric Bayesian formulation for the HMRF model, the infinite HMRF model, formulated on the basis of a joint Dirichlet process mixture (DPM) and Markov random field (MRF) construction. We derive an efficient variational Bayesian inference algorithm for the proposed model, and we experimentally demonstrate its advantages over competing methodologies.

Signal Modeling and Classification Using a Robust Latent Space Model Based on

Factor analysis is a statistical covariance modeling technique based on the assumption of normally distributed data. A mixture of factor analyzers can be hence viewed as a special case of Gaussian (normal) mixture models providing a mathematically sound framework for attribute space dimensionality reduction. A significant shortcoming of mixtures of factor analyzers is the vulnerability of normal distributions to outliers. Recently, the replacement of normal distributions with the heavier-tailed Students-t distributions has been proposed as a way to mitigate these shortcomings and the treatment of the resulting model under an expectation-maximization (EM) algorithm framework has been conducted. In this paper, we develop a Bayesian approach to factor analysis modeling based on Students-t distributions. We derive a tractable variational inference algorithm for this model by expressing the Students-t distributed factor analyzers as a marginalization over additional latent variables. Our innovative approach provides an efficient and more robust alternative to EM-based methods, resolving their singularity and overfitting proneness problems, while allowing for the automatic determination of the optimal model size. We demonstrate the superiority of the proposed model over well-known covariance modeling techniques in a wide range of signal processing applications.

t

Factor analysis is a latent subspace model commonly used for local dimensionality reduction tasks. Fuzzy c-means (FCM) type fuzzy clustering approaches are closely related to Gaussian mixture models (GMMs), and expectation-maximization (EM) like algorithms have been employed in fuzzy clustering with regularized objective functions. Students t -mixture models (SMMs) have been proposed recently as an alternative to GMMs, resolving their outlier vulnerability problems. In this paper, we propose a novel FCM-type fuzzy clustering scheme providing two significant benefits when compared with the existing approaches. First, it provides a well-established observation space dimensionality reduction framework for fuzzy clustering algorithms based on factor analysis, allowing concurrent performance of fuzzy clustering and, within each cluster, local dimensionality reduction. Second, it exploits the outlier tolerance advantages of SMMs to provide a novel, soundly founded, nonheuristic, robust fuzzy clustering framework by introducing the effective means to incorporate the explicit assumption about students t -distributed data into the fuzzy clustering procedure. This way, the proposed model yields a significant performance increase for the fuzzy clustering algorithm, as we experimentally demonstrate.

Distributions

In this paper, we propose a robust fuzzy clustering algorithm, based on a fuzzy treatment of finite mixtures of multivariate Students-t distributions, using the fuzzy c-means (FCM) algorithm. As we experimentally demonstrate, the proposed algorithm, by incorporating the assumptions about the probabilistic nature of the clusters being dirived into the fuzzy clustering procedure, allows for the exploitation of the hard tails of the multivariate Students-t distribution, to obtain a robust to outliers fuzzy clustering algorithm, offering increased clustering performance comparing to existing FCM-based algorithms. Our experimental results prove that the proposed fuzzy treatment of finite mixtures of Students-t distributions is more effective comparing to their statistical treatments using EM-type algorithms, while imposing comparable computational loads.

Factor Analysis Latent Subspace Modeling and Robust Fuzzy Clustering Using

Grids and mobile Grids can form the basis and the enabling technology for pervasive and utility computing due to their ability to be open, highly heterogeneous and scalable. However, the process of selecting the appropriate resources and initiating the execution of a job is not enough to provide quality in a dynamic environment such as a mobile Grid, where changes are numerous, highly variable and with unpredictable effects. In this paper we present a scheme for advancing and managing Quality of Service (QoS) attributes contained in Service Level Agreement (SLA) contracts of Grids that follow the Open Grid Services Architecture (OGSA). In order to achieve this, the execution environment of the Grid infrastructure establishes and exploits the synergies between the various modules of the architecture that participate in the management of the execution and the enforcement of the SLA contractual terms. We introduce an Execution Management Service which is in collaboration with both the application services and the network services in order to provide an adjustable quality of the requested services. The components that manage and control the execution in the Grid environment interact with the suit of the SLA-related services exchanging information that is used to provide the quality framework of the execution with respect to the agreed contractual terms. The described scheme has been implemented in the framework of the Akogrimo IST project.