Kenneth A. Kaufman

Mining for Knowledge in Databases: The INLEN Architecture, Initial Implementation and First Results

The architecture of an intelligent multistrategy assistant for knowledge discovery from facts, INLEN, is described and illustrated by an exploratory application. INLEN integrates a database, a knowledge base, and machine learning methods within a uniform user-oriented framework. A variety of machine learning programs are incorporated into the system to serve as high-levelknowledge generation operators (KGOs). These operators can generate diverse kinds of knowledge about the properties and regularities existing in the data. For example, they can hypothesize general rules from facts, optimize the rules according to problem-dependent criteria, determine differences and similarities among groups of facts, propose new variables, create conceptual classifications, determine equations governing numeric variables and the conditions under which the equations apply, deriving statistical properties and using them for qualitative evaluations, etc. The initial implementation of the system, INLEN 1b, is described, and its performance is illustrated by applying it to a database of scientific publications.

The AQ21 Natural Induction Program for Pattern Discovery: Initial Version and its Novel Features

The AQ21 program aims to perform natural induction, a process of generating inductive hypotheses in human-oriented forms that are easy to interpret and understand. This is achieved by employing a highly expressive representation language, attributional calculus, whose statements resemble natural language descriptions. This paper focuses on the pattern discovery mode of AQ21, which produces attributional rules that capture strong regularities in the data, but may not be fully consistent or complete with regard to the training data. AQ21 integrates several novel features, such as optimizing patterns according to multiple criteria, learning attributional rules with exceptions, generating optimized sets of alternative hypotheses, and handling data with unknown, irrelevant and/or non-applicable meta-values

Learning from Inconsistent and Noisy Data: The AQ18 Approach

In concept learning or data mining tasks, the learner is typically faced with a choice of many possible hypotheses characterizing the data. If one can assume that the training data are noise-free, then the generated hypothesis should be complete and consistent with regard to the data. In real-world problems, however, data are often noisy, and an insistence on full completeness and consistency is no longer valid. The problem then is to determine a hypothesis that represents the “best” trade-off between completeness and consistency. This paper presents an approach to this problem in which a learner seeks rules optimizing a description quality criterion that combines completeness and consistency gain, a measure based on consistency that reflects the rule’s benefit. The method has been implemented in the AQ18 learning and data mining system and compared to several other methods. Experiments have indicated the flexibility and power of the proposed method.

Learning patterns in noisy data: the AQ approach

In concept learning and data mining, a typical objective is to determine concept descriptions or patterns that will classify future data points as correctly as possible. If one can assume that the data contain no noise, then it is desirable that descriptions are complete and consistent with regard to all the data, i.e., they characterize all data points in a given class (positive examples) and no data points outside the class (negative examples).

An Optimized Design of Finned-Tube Evaporators Using the Learnable Evolution Model

Optimizing the refrigerant circuitry for a finned-tube evaporator is a daunting task for traditional exhaustive search techniques due to the extremely large number of circuitry possibilities. For this reason, more intelligent search techniques are needed. This paper presents and evaluates a novel optimization system called ISHED1 (intelligent system for heat exchanger design). This system uses a recently developed non-Darwinian evolutionary computation method to seek evaporator circuit designs that maximize the capacity of the evaporator under given technical and environmental constraints. Circuitries were developed for an evaporator with three depth rows of 12 tubes each, based on optimizing the performance with uniform and nonuniform airflow profiles. ISHED1 demonstrated the capability to generate designs with capacity equal or superior to that of best human designs, particularly in cases with non-uniform airflow.

Experimental validations of the learnable evolution model

A recently developed approach to evolutionary computation, called Learnable Evolution Model or LEM, employs machine learning to guide processes of generating new populations. The central new idea of LEM is that it generates new individuals not by mutation and/or recombination, but by processes of hypothesis generation and instantiation. The hypotheses are generated by a machine learning system from examples of high and low performance individuals. When applied to problems of function optimization and parameter estimation for nonlinear filters, LEM significantly outperformed the standard evolutionary computation algorithms used in experiments, sometimes achieving two or more orders of magnitude of evolutionary speed-up (in terms of the number of births). An application of LEM to the problem of optimizing heat exchangers has produced designs equal to or exceeding the best human designs. Further research needs to explore trade-offs and determine best areas for LEM application.

From Data Mining to Knowledge Mining

Support for the Laboratorys research related to the presented results has been provided in part by the National Science Foundation under Grants No. DMI-9496192, IRI-9020266, IIS-9906858 and IIS-0097476; in part by the UMBC/LUCITE #32 grant; in part by the Office of Naval Research under Grant No. N00014-91-J-1351; in part by the Defense Advanced Research Projects Agency under Grant No. N00014-91-J-1854 administered by the Office of Naval Research; and in part by the Defense Advanced Research Projects Agency under Grants No. F49620-92-J-0549 and F49620-95-1-0462 administered by the Air Force Office of Scientific Research.

Building Knowledge Scouts Using KGL Metalanguage

Knowledge scouts are software agents that autonomously synthesize user-oriented knowledge (target knowledge) from information present in local or distributed databases. A knowledge generation metalanguage, KGL, is used to creating scripts defining such knowledge scouts. Knowledge scouts operate in an inductive database, by which we mean a database system in which conventional data and knowledge management operators are integrated with a wide range of data mining and inductive inference operators. Discovered knowledge is represented in two forms: (1) attributional rules, which are rules in attributional calculus—a logic-based language between prepositional and predicate calculus, and (2) association graphs, which graphically and abstractly represent relations expressed by the rules. These graphs can depict multi-argument relationships among different concepts, with a visual indication of the relative strength of each dependency. Presented ideas are illustrated by two simple knowledge scouts, one that seeks relations among lifestyles, environmental conditions, symptoms and diseases in a large medical database, and another that searches for patterns of childrens behavior in the National Youth Survey database. The preliminary results indicate a high potential utility of this methodology for deriving knowledge from databases.

The Development of the Inductive Database System VINLEN: A Review of Current Research

Current research on the VINLEN inductive database system is briefly reviewed and illustrated by selected results. The goal of research on VINLEN is to develop a methodology for deeply integrating a wide range of knowledge generation operators with a relational database and a knowledge base. The current system has already integrated an AQ learning system for generating attributional rules in two modes: theory formation, in which generated rules are consistent and complete with regard to data, and pattern discovery, in which generated rules represent strong patterns, not necessarily consistent or complete. It also has integrated a conceptual clustering module for splitting data into conceptual classes, and providing descriptions of those classes. Preliminary data management and knowledge visualization operators, such as the intelligent target data generator (ITG) and concept association graph display, have also been integrated. To facilitate an easy interaction with the system, a user-oriented visual interface has been implemented. An example of results from applying VINLEN to a medical problem domain is presented to illustrate VINLEN knowledge discovery and representation capabilities.

An Adjustable Description Quality Measure for Pattern Discovery Usingthe AQ Methodology

In concept learning and data mining tasks, the learner is typically faced with a choice of many possible hypotheses or patterns characterizing the input data. If one can assume that training data contain no noise, then the primary conditions a hypothesis must satisfy are consistency and completeness with regard to the data. In real-world applications, however, data are often noisy, and the insistence on the full completeness and consistency of the hypothesis is no longer valid. In such situations, the problem is to determine a hypothesis that represents the best trade-off between completeness and consistency. This paper presents an approach to this problem in which a learner seeks rules optimizing a rule quality criterion that combines the rule coverage (a measure of completeness) and training accuracy (a measure of inconsistency). These factors are combined into a single rule quality measure through a lexicographical evaluation functional (LEF). The method has been implemented in the AQ18 learning system for natural induction and pattern discovery, and compared with several other methods. Experiments have shown that the proposed method can be easily tailored to different problems and can simulate different rule learners by modifying the parameter of the rule quality criterion.