Ibrahim F. Imam

Learning decision trees from decision rules: a method and initial results from a comparative study

A standard approach to determining decision trees is to learn them from examples. A disadvantage of this approach is that once a decision tree is learned, it is difficult to modify it to suit different decision making situations. Such problems arise, for example, when an attribute assigned to some node cannot be measured, or there is a significant change in the costs of measuring attributes or in the frequency distribution of events from different decision classes. An attractive approach to resolving this problem is to learn and store knowledge in the form of decision rules, and to generate from them, whenever needed, a decision tree that is most suitable in a given situation. An additional advantage of such an approach is that it facilitates buildingcompact decision trees, which can be much simpler than the logically equivalent conventional decision trees (by compact trees are meant decision trees that may contain branches assigned aset of values, and nodes assignedderived attributes, i.e., attributes that are logical or mathematical functions of the original ones). The paper describes an efficient method, AQDT-1, that takes decision rules generated by an AQ-type learning system (AQ15 or AQ17), and builds from them a decision tree optimizing a given optimality criterion. The method can work in two modes: thestandard mode, which produces conventional decision trees, andcompact mode, which produces compact decision trees. The preliminary experiments with AQDT-1 have shown that the decision trees generated by it from decision rules (conventional and compact) have outperformed those generated from examples by the well-known C4.5 program both in terms of their simplicity and their predictive accuracy.

Learning Problem-Oriented Decision Structures from Decision Rule: The AQDT-2 System

A decision structure is an acyclic graph that specifies an order of tests to be applied to an object (or a situation) to arrive at a decision about that object. and serves as a simple and powerful tool for organizing a decision process. This paper proposes a methodology for learning decision structures that are oriented toward specific decision making situations. The methodology consists of two phases: 1—determining and storing declarative rules describing the decision process, 2—deriving on-line a decision structure from the rules. The first step is performed by an expert or by an AQ-based inductive learning program that learns decision rules from examples of decisions (AQ15 or AQ17). The second step transforms the decision rules to a decision structure that is most suitable for the given decision making situation. The system, AQDT-2, implementing the second step, has been applied to a problem in construction engineering. In the experiments, AQDT-2 outperformed all other programs applied to the same problem in terms of the accuracy and the simplicity of the generated decision structures.

Should Decision Trees be Learned from Examples of from Decision Rules

A standard method for determining decision trees is to learn them from examples. A disadvantage of this approach is that once a decision tree is learned, it is difficult to modify it to suit different decision making situations. An attractive approach that avoids this problem is to learn and store knowledge in a declarative form, e.g., as decision rules, and then, whenever needed, generate from it a decision free that is most suitable in any given situation. This paper describes an efficient method for this purpose, called AQDT-1, which takes decision rules generated by the learning system AQ15 and builds from them a decision tree optimized according to a given quality criterion. The method is able to build conventional decision trees, as well as the so-called “skip noder” trees, in which measuring attributes assigned to some nodes may be avoided. It is shown that “skip-node” trees can be significantly simpler than conventional ones. In the experiments comparing AQDT-1 with C4.5, the former outperformed the latter both in terms of the predictive accuracy as well as the simplicity of the generated decision trees.

On learning decision structures

A decision structure is a simple and powerful tool for organizing a decision process. It differs from a conventional decision tree in that its nodes are assigned tests that can be functions of the attributes, rather than single attributes; the branches stemming from a node can be assigned a subset of attribute values rather than a single attribute value (test outcome); and the leaves can be assigned one or more alternative decisions. We describe a methodology for learning decision structures from declarative knowledge expressed in the form of decision rules. The decision rules are generated by an expert, or by an AQ-type inductive learning program (with or without constructive induction). From a given set of rules, one can generate many different decision structures. The proposed methodology generates the one that is most suitable for the given decision-making situation, according to a multicriterion evaluation function. Experiments with a program implementing the proposed methodology have demonstrated its many useful features.

An Empirical Study on The Incompetence of Attribute Selection Criteria

One of the main tasks in most supervised learning systems is the evaluation of the attributional relevancy in the given databases. Such relevancy is mainly concerned with the relationship between the available attributes and the decision classes. Attributes relevant to the decision classes are used to represent the learned knowledge, while irrelevant attributes are removed or ignored during the learning process. This paper investigates the relationship between attributional relevancy to decision classes and to learning systems. The experimental results from different databases show that some attributes relevant to decision classes may be irrelevant to the learning system. Experiments are performed on eight different databases using the C4.5 system for learning decision trees from examples.

Adaptive Intelligent Agents

Recently, research and development activities in the field of intelligent agents have made considerable impact in the business, industrial and academic communities. Intelligent agents are software systems or machines designed to accomplish tasks on behalf to the ”user.” This is one of the fastest growing areas of research and development within Artificial Intelligence, Information Retrieval and Database Management. One important distinguishing characteristic of such agents is their ability to adapt. Adaptation denotes the ability of agents to evolve: their knowledge of the world, their assigned tasks, their problem solving techniques and mechanisms, and their strategies for communication and negotiation with other agents. Adaptive Intelligent Agents are “systems or machines that utilize inferential or complex computational algorithms to modify or change control parameters, knowledge-bases, problem-solving methodologies, course of actions, or other objects in order to accomplish a set of tasks required by the user” (Imam and Kodratoff, 1997). This special issue of JIIS is dedicated to Adaptive Intelligent Agents. The primary focus is on the adaptive behavior of agents and on methodologies for the construction of adaptive intelligent agents. This issue consists of four papers grouped into two categories: agents for information navigation, and knowledge-level issues for the investigation and development of adaptive intelligent agents. These papers promote an interesting viewpoint of the adaptation process. This view can be characterized as a combination of one or more types of adaptation: 1) Physical adaptation in which agents can transport into another location or environment; 2) Algorithmic/management adaptation in which agents can re-plan, relearn and evolve knowledge, reassign tasks and their priorities, etc.; 3) Content and representation adaptation such as adapting the agent’s knowledge and/or its representation, as well as migrating knowledge from one level to another; and 4) Direct adaptation where a change in the environment cause the agent to adapt.

Learning for Decision Making: the FRD Approach and a Comparative Study

This paper concerns the issue of what is the best form for learning, representing and using knowledge for decision making. The proposed answer is that such knowledge should be learned and represented in a declarative form. When needed for decision making, it should be efficiently transferred to a procedural form that is tailored to the specific decision making situation. Such an approach combines advantages of the declarative representation, which facilitates learning and incremental knowledge modification, and the procedural representation, which facilitates the use of knowledge for decision making. This approach also allows one to determine decision structures that may avoid attributes that unavailable or difficult to measure in any given situation. Experimental investigations of the system, FRD-1, have demonstrated that decision structures obtained via the declarative route often have not only higher predictive accuracy but are also are simpler than those learned directly from facts.