Athanasios Tsakonas

A comparison of classification accuracy of four genetic programming-evolved intelligent structures

We investigate the effectiveness of GP-generated intelligent structures in classification tasks. Specifically, we present and use four context-free grammars to describe (1) decision trees, (2) fuzzy rule-based systems, (3) feedforward neural networks and (4) fuzzy Petri-nets with genetic programming. We apply cellular encoding in order to express feedforward neural networks and fuzzy Petri-nets with arbitrary size and topology. The models then are examined thoroughly in six well-known real world data sets. Results are presented in detail and the competitive advantages and drawbacks of the selected methodologies are discussed, in respect to the nature of each application domain. Conclusions are drawn on the effectiveness and efficiency of the presented approach.

Bankruptcy prediction with neural logic networks by means of grammar-guided genetic programming

The paper demonstrates the efficient use of hybrid intelligent systems for solving the classification problem of bankruptcy. The aim of the study is to obtain classification schemes able to predict business failure. Previous attempts to form efficient classifiers for the same problem using intelligent or statistical techniques are discussed throughout the paper. The application of neural logic networks by means of genetic programming is proposed. This is an advantageous approach enabling the interpretation of the network structure through set of expert rules, which is a desirable feature for field experts. These evolutionary neural logic networks are consisted of an innovative hybrid intelligent methodology, by which evolutionary programming techniques are used for obtaining the best possible topology of a neural logic network. The genetic programming process is guided using a context-free grammar and indirect encoding of the neural logic networks into the genetic programming individuals. Indicative classification results are presented and discussed in detail in terms of both, classification accuracy and solution interpretability.

Hierarchical classification trees using type-constrained genetic programming

We investigate the capability of the genetic programming approach for producing hierarchical, rule-based, classification trees. These trees can be seen as an extension to the machine learning decision trees concept, where the predicates here can be complex expressions rather than just simple attribute-value comparisons. In order to improve the search ability and to produce meaningful results, type-constraints are applied to the genetic programming procedure, expressed in a BNF grammar. The model is tested in two well-known domains. In the Balance-Scale data, the system achieves in revealing the data creation rule. In the E-Coli Protein Localization Sites data, the system realizes a competitor to the literature classification score, retaining the solution comprehensibility. The training procedure is guided by an adaptive fitness measure. The overall performance of this system denotes its competitiveness to standard computational intelligent procedures.

Local and global optimization for Takagi-Sugeno fuzzy system by memetic genetic programming

This work presents a method to incorporate standard neuro-fuzzy learning for Takagi-Sugeno fuzzy systems that evolve under a grammar driven genetic programming (GP) framework. This is made possible by introducing heteroglossia in the functional GP nodes, enabling them to switch behavior according to the selected learning stage. A context-free grammar supports the expression of arbitrarily sized and composed fuzzy systems and guides the evolution. Recursive least squares and backpropagation gradient descent algorithms are used as local search methods. A second generation memetic approach combines the genetic programming with the local search procedures. Based on our experimental results, a discussion is included regarding the competitiveness of the proposed methodology and its properties. The contributions of the paper are: (i) introduction of an approach which enables the application of local search learning for intelligent systems evolved by genetic programming, (ii) presentation of a model for memetic learning of Takagi-Sugeno fuzzy systems, (iii) experimental results evaluating model variants and comparison with state-of-the-art models in benchmarking and real-world problems, (iv) application of the proposed model in control.

EVOLVING NEURAL-SYMBOLIC SYSTEMS GUIDED BY ADAPTIVE TRAINING SCHEMES: APPLICATIONS IN FINANCE

The article presents a hybrid and adaptive intelligent methodology, based on neural logic networks and grammar-guided genetic programming. The aim of the study is to demonstrate how to generate efficient neural logic networks with the aid of genetic programming methods trained adaptively through an innovative scheme. The proposed adaptive training scheme of the genetic programming mechanism leads to the generation of high-diversity solutions and small-sized individuals. The overall methodology is advantageous due to the adaptive training scheme proposed for offering both accurate and interpretable results in the form of expert rules. Moreover, a sensitivity analysis study is provided within the article, comparing the performance of the proposed evolutionary neural logic networks methodology with well-known competitive inductive machine learning approaches. Two financial domains of application have been selected to demonstrate the capabilities of the proposed methodology: (a) classification of credit applicants for consumer loans of a German bank and (b) the credit-scoring decision-making process in an Australian bank. Results seem encouraging since the proposed methodology outperforms a number of competitive existing statistical and intelligent methodologies, while it also produces handy decision rules, short in length and transparent in meaning and use.

An evolutionary system for neural logic networks using genetic programming and indirect encoding

Abstract Nowadays, intelligent connectionist systems such as artificial neural networks have been proved very powerful in a wide area of applications. Consequently, the ability to interpret their structure was always a desirable feature for experts. In this field, the neural logic networks (NLN) by their definition are able to represent complex human logic and provide knowledge discovery. However, under contemporary methodologies, the training of these networks may often result in non-comprehensible or poorly designed structures. In this work, we propose an evolutionary system that uses current advances in genetic programming that overcome these drawbacks and produces neural logic networks that can be arbitrarily connected and are easily interpretable into expert rules. To accomplish this task, we guide the genetic programming process using a context-free grammar and we encode indirectly the neural logic networks into the genetic programming individuals. We test the proposed system in two problems of medical diagnosis. Our results are examined both in terms of the solution interpretability that can lead in knowledge discovery, and in terms of the achieved accuracy. We draw conclusions about the effectiveness of the system and we propose further research directions.

A fuzzy evolutionary framework for combining ensembles

We propose an evolutionary framework for the production of fuzzy rule bases where each rule executes an ensemble of predictors. The architecture, the rule base and the composition of the ensembles are evolved over time. To achieve this, we employ a context-free grammar within a hybrid genetic programming system using a multi-population model. As base predictors, multilayer perceptron neural networks and support vector machines are available. We apply the system to several function approximation and regression tasks and compare the results with recent research and state-of-the-art models. We conclude that the proposed architecture is competitive and has a number of very desirable features supporting automation of predictive model building and their adaptation over time. Finally, we suggest further potential research directions.

Predicting Defects in Software Using Grammar-Guided Genetic Programming

The knowledge of the software quality can allow an organization to allocate the needed resources for the code maintenance. Maintaining the software is considered as a high cost factor for most organizations. Consequently, there is need to assess software modules in respect of defects that will arise. Addressing the prediction of software defects by means of computational intelligence has only recently become evident. In this paper, we investigate the capability of the genetic programming approach for producing solution composed of decision rules. We applied the model into four software engineering databases of NASA. The overall performance of this system denotes its competitiveness as compared with past methodologies, and is shown capable of producing simple, highly accurate, tangible rules.

From Sensor Readings to Predictions: On the Process of Developing Practical Soft Sensors

Automatic data acquisition systems provide large amounts of streaming data generated by physical sensors. This data forms an input to computational models (soft sensors) routinely used for monitoring and control of industrial processes, traffic patterns, environment and natural hazards, and many more. The majority of these models assume that the data comes in a cleaned and pre-processed form, ready to be fed directly into a predictive model. In practice, to ensure appropriate data quality, most of the modelling efforts concentrate on preparing data from raw sensor readings to be used as model inputs. This study analyzes the process of data preparation for predictive models with streaming sensor data. We present the challenges of data preparation as a four-step process, identify the key challenges in each step, and provide recommendations for handling these issues. The discussion is focused on the approaches that are less commonly used, while, based on our experience, may contribute particularly well to solving practical soft sensor tasks. Our arguments are illustrated with a case study in the chemical production industry.

Towards neural-symbolic integration: the evolutionary neural logic networks

This work presents the application of a new methodology for the production of neural logic networks into two real-world problems from the medical domain. Namely, we apply grammar guided genetic programming using cellular encoding for the representation of neural logic networks into population individuals. The application area is consisted of the diagnosis of diabetes and the diagnosis of the course of hepatitis patients. The system is proved able to generate arbitrarily connected and interpretable evolved solutions leading to potential knowledge extraction.