Cezary Z. Janikow

Fuzzy decision trees: issues and methods

Decision trees are one of the most popular choices for learning and reasoning from feature-based examples. They have undergone a number of alterations to deal with language and measurement uncertainties. We present another modification, aimed at combining symbolic decision trees with approximate reasoning offered by fuzzy representation. The intent is to exploit complementary advantages of both: popularity in applications to learning from examples, high knowledge comprehensibility of decision trees, and the ability to deal with inexact and uncertain information of fuzzy representation. The merger utilizes existing methodologies in both areas to full advantage, but is by no means trivial. In particular, knowledge inferences must be newly defined for the fuzzy tree. We propose a number of alternatives, based on rule-based systems and fuzzy control. We also explore capabilities that the new framework provides. The resulting learning method is most suitable for stationary problems, with both numerical and symbolic features, when the goal is both high knowledge comprehensibility and gradually changing output. We describe the methodology and provide simple illustrations.

A Knowledge-Intensive Genetic Algorithm for Supervised Learning

Supervised learning in attribute-based spaces is one of the most popular machine learning problems studied and, consequently, has attracted considerable attention of the genetic algorithm community. The fullmemory approach developed here uses the same high-level descriptive language that is used in rule-based systems. This allows for an easy utilization of inference rules of the well-known inductive learning methodology, which replace the traditional domain-independent operators and make the search task-specific. Moreover, a closer relationship between the underlying task and the processing mechanisms provides a setting for an application of more powerful task-specific heuristics. Initial results obtained with a prototype implementation for the simplest case of single concepts indicate that genetic algorithms can be effectively used to process high-level concepts and incorporate task-specific knowledge. The method of abstracting the genetic algorithm to the problem level, described here for the supervised inductive learning, can be also extended to other domains and tasks, since it provides a framework for combining recently popular genetic algorithm methods with traditional problem-solving methodologies. Moreover, in this particular case, it provides a very powerful tool enabling study of the widely accepted but not so well understood inductive learning methodology.

A modified genetic algorithm for optimal control problems

Abstract This paper studies the application of a genetic algorithm to discrete-time optimal control problems. Numerical results obtained here are compared with ones yielded by GAMS, a system for construction and solution of large and complex mathematical programming models. While GAMS appears to work well only for linear quadratic optimal control problems or problems with short horizon, the genetic algorithm applies to more general problems equally well.

Genetic algorithms for numerical optimization

Genetic algorithms (GAs) are stochastic adaptive algorithms whose search method is based on simulation of natural genetic inheritance and Darwinian striving for survival. They can be used to find approximate solutions to numerical optimization problems in cases where finding the exact optimum is prohibitively expensive, or where no algorithm is known. However, such applications can encounter problems that sometimes delay, if not prevent, finding the optimal solutions with desired precision. In this paper we describe applications of GAs to numerical optimization, present three novel ways to handle such problems, and give some experimental results.

Genetic algorithms and optimal control problems

The application of the genetic algorithm to discrete-time optimal control problems is studied. The numerical results obtained are compared with a system for construction and solution of large and complex mathematical programming models, GAMS. It is shown that while GAMS appears to work well only for linear-quadratic optimal control problems or problems with a short horizon, the genetic algorithm applies to more general problems and appears to be competitive with search-based methods.<<ETX>>

Exemplar learning in fuzzy decision trees

Decision-tree algorithms provide one of the most popular methodologies for symbolic knowledge acquisition. The resulting knowledge, a symbolic decision tree along with a simple inference mechanism, has been praised for comprehensibility. The most comprehensible decision trees have been designed for perfect symbolic data. Over the years, additional methodologies have been investigated and proposed to deal with continuous or multi-valued data, and with missing or noisy features. Recently, with the growing popularity of fuzzy representation, a few researchers independently have proposed to utilize fuzzy representation in decision trees to deal with similar situations. Fuzzy representation bridges the gap between symbolic and non-symbolic data by linking qualitative linguistic terms with quantitative data. In this paper, we overview our fuzzy decision tree and propose a few new inferences based on exemplar learning.

Fuzzy partitioning with FID3.1

FID3.1 builds fuzzy decision trees, with a range of choices for fuzzy operators and inferences. Various FID algorithms are being widely used for dealing with numeric and/or imprecise data, for fuzzy classification or for generating fuzzy rules. FID 3.0 adds a number of new features, the most important being a fuzzy partitioning mechanism construction of fuzzy sets for continuous variables w/o predefined fuzzy terms. FID3.1 improves the mechanism in a number of ways. The paper describes the partitioning method and presents a few comparative experiments.

An analysis of Gray versus binary encoding in genetic search

This paper employs a Markov model to study the relative performance of binary and Gray coding in genetic algorithms. The results indicate that while there is not much difference between the two for all possible functions, Gray coding does not necessarily improve performance for functions which have fewer local optima in the Gray representation than in binary.

A genetic algorithm method for optimizing fuzzy decision trees

Fuzzy decision trees exploit the popularity of decision tree algorithms for practical knowledge acquisition and the representative power of the fuzzy technology. They are extensions of ID3 trees, with the tree-building routine modified to utilize fuzzy instead of strict domains, and with new inferences combining fuzzy defuzzification with the inductive methodology. As ID3 trees, they require that real-valued and multivalued domains be partitioned prior to tree construction. In this paper, we introduce a methodology aimed at relaxing this requirement. This is done by optimizing the domain partitions. This optimization is based on genetic algorithms, designed to process constraints associated with this task. A simple illustration is also given.

Fuzzy Decision Forest

We investigate the extension of fuzzy decision trees into fuzzy forests. Decision forests attempt to alleviate some problems often associated with decision trees: decision trees are minimalistic in their contained information, and they often degrade in complex domains, when multidimensional relationships are needed, or when there is no preference over similar actions. Moreover, the minimalistic approach often degrades the performance when some necessary features are either missing, noisy, or simply unreliable. These problems have been addressed in the last few years in hybrid systems, in which a number of distinct trees were extracted and used with some voting rules. Fuzzy decision forests follow the same ideas, except that they use more elaborate alternatives specific to local partitioning of the space. Therefore, it is an extension of those hybrid methods. In other words, while in those hybrid systems multiple choices are allowed at the level of the global search space, a fuzzy decision forest allows alternatives at every subspace level. Moreover the choice of available alternatives is data and domain-driven.