Zeke S. H. Chan

Short-term ANN load forecasting from limited data using generalization learning strategies

Abstract The emergence of the new competitive electricity market environment has made short-term load forecasting a more complex task, owing to the effect of marketers’ behavior on the load pattern and the reduction of available information due to commercial reasons. In recent years, many ANN-based forecasters are proposed for learning the highly nonlinear load pattern, yet their effectiveness are limited by the reduction of training data, which causes these ANN models to be susceptible to “over-fitting”. “Over-fitting” is a common ANN problem that describes the situation that the model memorizes the training data but fails to generalize well to new data. This paper discusses the problem of “over-fitting” and some common generalization learning techniques in the ANN literature, as well as introducing a new Genetic Algorithm-based regularization method called “GARNET” for short-term load forecasting. As an illustration, four generalization learning techniques, including Early-Stopping, Bayesian Regularization, Adaptive-Regularization and GARNET are applied to train Multi-Layer Perceptrons networks (MLP) for day-ahead load forecasting on limited amount of hourly data from a US utility. Results show that forecasters trained by these four methods consistently produce lower prediction error than those trained by the standard error minimization method.

Fast neural network ensemble learning via negative-correlation data correction

This letter proposes a new negative correlation (NC) learning method that is both easy to implement and has the advantages that: 1) it requires much lesser communication overhead than the standard NC method and 2) it is applicable to ensembles of heterogenous networks.

An efficient greedy K-means algorithm for global gene trajectory clustering

Optimal clustering of co-regulated genes is critical for reliable inference of the underlying biological processes in gene expression analysis, for which the K-means algorithm have been widely employed for its efficiency. However, given that the solution space is large and multimodal, which is typical of gene expression data, K-means is prone to produce inconsistent and sub-optimal cluster solutions that may be unreliable and misleading for biological interpretation. This paper applies a novel global clustering method called the greedy elimination method (GEM) to alleviate these problems. GEM is simple to implement, yet very effective in improving the global optimality of the solutions. Experiments over two sets of gene expression data show that the GEM scores significantly lower clustering errors than the standard K-means and the greedy incremental method.

Gene regulatory network discovery from time-series gene expression data: A computational intelligence approach

The interplay of interactions between DNA, RNA and proteins leads to genetic regulatory networks (GRN) and in turn controls the gene regulation. Directly or indirectly in a cell such molecules either interact in a positive or in repressive manner therefore it is hard to obtain the accurate computational models through which the final state of a cell can be predicted with certain accuracy. This paper describes biological behaviour of actual regulatory systems and we propose a novel method for GRN discovery of a large number of genes from multiple time series gene expression observations over small and irregular time intervals. The method integrates a genetic algorithm (GA) to select a small number of genes and a Kalman filter to derive the GRN of these genes. After GRNs of smaller number of genes are obtained, these GRNs may be integrated in order to create the GRN of a larger group of genes of interest.

Soft computing methods to predict gene regulatory networks: An integrative approach on time-series gene expression data

To unravel the controlling mechanisms of gene regulation, in this paper we present the application of sophisticated soft computing methods applied on an important problem from Bioinformatics-inferring gene regulatory networks (GRN) from time series gene expression microarray data. The main questions addressed in this paper are: (a) what knowledge can be derived from different models? (b) Would an integrated approach be more suitable to reveal about the controls of gene regulation? To reduce the number of genes in addition to apply the appropriate clustering methods, here we have also considered the valuable inputs from the biological experiments. To infer the GRN we have applied: three computational intelligence methods-Least Angle Regression (LARS), Expectation Maximization (EM) with Kalman Filter (KF), and an Evolving Fuzzy Neural Network (EFuNN). The methods are applied on time series microarray data of Schizosaccharomyces pombe yeast cell-cycle genes. Each method reveals some new aspects of the problem and it is agreed that to infer the GRN and to understand the processes behind gene regulation it is more suitable to adopt such integrative approach as ours through which some new knowledge is discovered, such as: using LARS we hypothesize-first, an exoglucanase gene exg1 is now implicated to be tied with MCB cluster regulation and second, a mannosidase with histone linked mannoses. A new quantitative prediction is that the time delay of the interaction between two genes seems to be approximately 30min, or 0.17 cell cycles. Using the method of EM with KF, 25 cell cycle-regulated key genes were successfully clustered into three functionally co-regulated groups. We have also identified two genes namely Cdc22 and Suc22 that indeed interact with each other and are the potential candidates as a control in Ribonucleotide reductase (RNR) activity. Based on the EFuNN results and integrating knowledge from EM-KF method, we hypothesize that interaction between Suc22, Cdc22 and Mrc1 may be mediated by two other genes namely Cds1 and Spd1. The methods discussed and applied here can be used to analyze any kind of short time series of many interacting variables for inferring the regulatory network. Researchers should take such integrative computational intelligence approach seriously to understand the complex phenomenon of gene regulation and thus to simulate the development of the cell.

Bayesian learning of sparse gene regulatory networks

Differential equations (DEs) have been the most widespread formalism for gene regulatory network (GRN) modeling, as they offer natural interpretation of biological processes, easy elucidation of gene relationships, and the capability of using efficient parameter estimation methods. However, an important limitation of DEs is their requirement of O(d(2)) parameters where d is the number of genes modeled, which often causes over-parameterization for large d, leading to the over-fitting of data and dense parameter sets that are hard to interpret. This paper presents the first effort to address the over-parameterization problem by applying the sparse Bayesian learning (SBL) method to sparsify the GRN model of DEs. SBL operates on the parsimony principle, with the objective to reduce the number of effective parameters by driving the redundant parameters to zero. The resulting sparse parameter set offers three important advantages for GRN inference: first, the inferred GRNs are more plausible, since the biological counterparts are known to be sparse; second, gene relationships can be more easily elucidated from sparse sets than from dense sets; and third, the solutions become more optimal and consistent, due to the reduction in the volume of solution space. Experiments are conducted on the yeast Saccharomyces cerevisiae time-series gene expression data, in which known regulatory events related to the cell cycle G1/S phase are reliably reproduced.

Gene trajectory clustering with a hybrid genetic algorithm and expectation maximization method

Clustering time course gene expression data (gene trajectories) is an important step towards solving the complex problem of gene regulatory network (GRN) modeling and discovery as it significantly reduces the dimensionality of the gene space required for analysis. This paper introduces a novel method that hybridizes genetic algorithm (GA) and expectation maximization algorithms (EM) for clustering with the mixtures of multiple linear regression models (MLRs). The proposed method is applied to cluster gene expression time course data into smaller number of classes based on their trajectory similarities. Its performance and application as a generic clustering method to other complex problems are discussed.

Evolutionary Computation For On-Line And Off-Line Parameter Tuning Of Evolving Fuzzy Neural Networksc.

This work applies Evolutionary Computation to achieve completely self-adapting Evolving Fuzzy Neural Networks (EFuNNs) for operating in both incremental (on-line) and batch (off-line) modes. EFuNNs belong to a class of Evolving Connectionist Systems (ECOS), capable of performing clustering-based, on-line, local area learning and rule extraction. Through Evolutionary Computation, its parameters such as learning rates and membership functions are continuously adjusted to reflect the changes in the dynamics of incoming data. The proposed methods are tested on the Mackey–Glass series and the results demonstrate a substantial improvement in EFuNNs performance.

A HYBRID GENETIC ALGORITHM AND EXPECTATION MAXIMIZATION METHOD FOR GLOBAL GENE TRAJECTORY CLUSTERING

Clustering time-course gene expression data (gene trajectories) is an important step towards solving the complex problem of gene regulatory network modeling and discovery as it significantly reduces the dimensionality of the gene space required for analysis. Traditional clustering methods that perform hill-climbing from randomly initialized cluster centers are prone to produce inconsistent and sub-optimal cluster solutions over different runs. This paper introduces a novel method that hybridizes genetic algorithm (GA) and expectation maximization algorithms (EM) for clustering gene trajectories with the mixtures of multiple linear regression models (MLRs), with the objective of improving the global optimality and consistency of the clustering performance. The proposed method is applied to cluster the human fibroblasts and the yeast time-course gene expression data based on their trajectory similarities. It outperforms the standard EM method significantly in terms of both clustering accuracy and consistency. The biological implications of the improved clustering performance are demonstrated.

A Preliminary Study on Negative Correlation Learning via Correlation-Corrected Data (NCCD)

This letter presents a novel cooperative neural network ensemble learning method based on Negative Correlation learning. It enables easy integration of various network models and reduces communication bandwidth significantly for effective parallel speedup. Comparison with the best Negative Correlation learning method reported demonstrates comparable performance at significantly reduced communication overhead.