Dong-Yeon Cho

Identification of biochemical networks by S-tree based genetic programming

MOTIVATION Most previous approaches to model biochemical networks have focused either on the characterization of a network structure with a number of components or on the estimation of kinetic parameters of a network with a relatively small number of components. For system-level understanding, however, we should examine both the interactions among the components and the dynamic behaviors of the components. A key obstacle to this simultaneous identification of the structure and parameters is the lack of data compared with the relatively large number of parameters to be estimated. Hence, there are many plausible networks for the given data, but most of them are not likely to exist in the real system. RESULTS We propose a new representation named S-trees for both the structural and dynamical modeling of a biochemical network within a unified scheme. We further present S-tree based genetic programming to identify the structure of a biochemical network and to estimate the corresponding parameter values at the same time. While other evolutionary algorithms require additional techniques for sparse structure identification, our approach can automatically assemble the sparse primitives of a biochemical network in an efficient way. We evaluate our algorithm on the dynamic profiles of an artificial genetic network. In 20 trials for four settings, we obtain the true structure and their relative squared errors are <5% regardless of releasing constraints about structural sparseness. In addition, we confirm that the proposed algorithm is robust within +/-10% noise ratio. Furthermore, the proposed approach ensures a reasonable estimate of a real yeast fermentation pathway. The comparatively less important connections with non-zero parameters can be detected even though their orders are below 10(-2). To demonstrate the usefulness of the proposed algorithm for real experimental biological data, we provide an additional example on the transcriptional network of SOS response to DNA damage in Escherichia coli. We confirm that the proposed algorithm can successfully identify the true structure except only one relation.

Applying Machine Learning Techniques to Analysis of Gene Expression Data: Cancer Diagnosis

Classification of patient samples is a crucial aspect of cancer diagnosis. DNA hybridization arrays simultaneously measure the expression levels of thousands of genes and it has been suggested that gene expression may provide the additional information needed to improve cancer classification and diagnosis. This paper presents methods for analyzing gene expression data to classify cancer types. Machine learning techniques, such as Bayesian networks, neural trees, and radial basis function (RBF) networks, are used for the analysis of the CAMDA Data Set 2. These techniques have their own properties including the ability of finding important genes for cancer classification, revealing relationships among genes, and classifying cancer. This paper reports on comparative evaluation of the experimental results of these methods.

Genetic Programming with Active Data Selection

Genetic programming evolves Lisp-like programs rather than fixed size linear strings. This representational power combined with generality makes genetic programming an interesting tool for automatic programming and machine learning. One weakness is the enormous time required for evolving complex programs. In this paper we present a method for accelerating evolution speed of genetic programming by active selection of fitness cases during the run. In contrast to conventional genetic programming in which all the given training data are used repeatedly, the presented method evolves programs using only a subset of given data chosen incrementally at each generation. This method is applied to the evolution of collective behaviors for multiple robotic agents. Experimental evidence supports that evolving programs on an incrementally selected subset of fitness cases can significantly reduce the fitness evaluation time without sacrificing generalization accuracy of the evolved programs.

Evolutionary Continuous Optimization by Distribution Estimation with Variational Bayesian Independent Component Analyzers Mixture Model

In evolutionary continuous optimization by building and using probabilistic models, the multivariate Gaussian distribution and their variants or extensions such as the mixture of Gaussians have been used popularly. However, this Gaussian assumption is often violated in many real problems. In this paper, we propose a new continuous estimation of distribution algorithms (EDAs) with the variational Bayesian independent component analyzers mixture model (vbICA-MM) for allowing any distribution to be modeled. We examine how this sophisticated density estimation technique has influence on the performance of the optimization by employing the same selection and population alternation schemes used in the previous EDAs. Our experimental results support that the presented EDAs achieve better performance than previous EDAs with ICA and Gaussian mixture- or kernel-based approaches.

Continuous estimation of distribution algorithms with probabilistic principal component analysis

Many evolutionary algorithms have been studied to build and use a probability distribution model of the population for optimization problems. Most of these methods tried to represent explicitly the relationship between variables in the problem with factorization techniques or a graphical model such as Bayesian or Gaussian networks. Thus enormous computational cost is required for constructing those models when the problem size is large. We propose a new estimation of distribution algorithm by using probabilistic principal component analysis (PPCA) which can explain the high order interactions with the latent variables. Since there are no explicit search procedures for the probability density structure, it is possible to rapidly estimate the distribution and readily sample the new individuals from it. Our experimental results support that the presented estimation of distribution algorithms with PPCA can find good solutions more efficiently than other EDAs for the continuous spaces.

System identification using evolutionary Markov chain Monte Carlo

System identification involves determination of the functional structure of a target system that underlies the observed data. In this paper, we present a probabilistic evolutionary method that optimizes system architectures for the identification of unknown target systems. The method is distinguished from existing evolutionary algorithms (EAs) in that the individuals are generated from a probability distribution as in Markov chain Monte Carlo (MCMC). It is also distinguished from conventional MCMC methods in that the search is population-based as in standard evolutionary algorithms. The effectiveness of this hybrid of evolutionary computation and MCMC is tested on a practical problem, i.e., evolving neural net architectures for the identification of nonlinear dynamic systems. Experimental evidence supports that evolutionary MCMC (or eMCMC) exploits the efficiency of simple evolutionary algorithms while maintaining the robustness of MCMC methods and outperforms either approach used alone.

Evolutionary optimization by distribution estimation with mixtures of factor analyzers

Evolutionary optimization algorithms based on the probability models have been studied to capture the relationship between variables in the given problems and finally to find the optimal solutions more efficiently. However, premature convergence to local optima still happens in these algorithms. Many researchers have used the multiple populations to prevent this ill behavior since the key point is to ensure the diversity of the population. In this paper, we propose a new estimation of distribution algorithm by using the mixture of factor analyzers (MFA) which can cluster similar individuals in a group and explain the high order interactions with the latent variables for each group concurrently. We also adopt a stochastic selection method based on the evolutionary Markov chain Monte Carlo (eMCMC). Our experimental results support that the presented estimation of distribution algorithms with MFA and eMCMC-like selection scheme can achieve better performance for continuous optimization problems.

Multi-stage evolutionary algorithms for efficient identification of gene regulatory networks

With the availability of the time series data from the high-throughput technologies, diverse approaches have been proposed to model gene regulatory networks. Compared with others, S-system has the advantage for these tasks in the sense that it can provide both quantitative (structural) and qualitative (dynamical) modeling in one framework. However, it is not easy to identify the structure of the true network since the number of parameters to be estimated is much larger than that of the available data. Moreover, conventional parameter estimation requires the time-consuming numerical integration to reproduce dynamic profiles for the S-system. In this paper, we propose multi-stage evolutionary algorithms to identify gene regulatory networks efficiently. With the symbolic regression by genetic programming (GP), we can evade the numerical integration steps. This is because the estimation of slopes for each time-course data can be obtained from the results of GP. We also develop hybrid evolutionary algorithms and modified fitness evaluation function to identify the structure of gene regulatory networks and to estimate the corresponding parameters at the same time. By applying the proposed method to the identification of an artificial genetic network, we verify its capability of finding the true S-system.

Evolving neural trees for time series prediction using Bayesian evolutionary algorithms

Bayesian evolutionary algorithms (BEAs) are a probabilistic model for evolutionary computation. Instead of simply generating new populations as in conventional evolutionary algorithms, the BEAs attempt to explicitly estimate the posterior distribution of the individuals from their prior probability and likelihood, and then sample offspring from the distribution. We apply the Bayesian evolutionary algorithms to evolving neural trees, i.e. tree-structured neural networks. Explicit formulae for specifying the distributions on the model space are provided in the context of neural trees. The effectiveness and robustness of the method is demonstrated on the time series prediction problem. We also study the effect of the population size and the amount of information exchanged by subtree crossover and subtree mutations. Experimental results show that small-step mutation-oriented variations are most effective when the population size is small, while large-step recombinative variations are more effective for large population sizes.

Evolving complex group behaviors using genetic programming with fitness switching

Genetic programming provides a useful tool for emergent computation and artificial life research. However, conventional genetic programming is not efficient enough to solve realistic multiagent tasks consisting of several emergent behaviors that need to be coordinated in the proper sequence. In this paper, we describe a novel method, called fitness switching, for evolving composite cooperative behaviors in multiple robotic agents using genetic programming. The method maintains a pool of basis fitness functions which are switched from simpler ones to more complex ones. The performance is demonstrated and evaluated in the context of a table transport problem. Experimental results show that the fitness switching method is an effective mechanism for evolving collective behaviors which can not be solved by simple genetic programming.