Ryszard S. Michalski

Learning from Observation: Conceptual Clustering

An important form of learning from observation is constructing a classification of given objects or situations. Traditional techniques for this purpose, developed in cluster analysis and numerical taxonomy, are often inadequate because they arrange objects into classes solely on the basis of a numerical measure of object similarity. Such a measure is a function only of compared objects and does not take into consideration any global properties or concepts characterizing object classes. Consequently, the obtained classes may have no simple conceptual description and may be difficult to interpret.

Pattern Recognition as Rule-Guided Inductive Inference

The determination of pattern recognition rules is viewed as a problem of inductive inference, guided by generalization rules, which control the generalization process, and problem knowledge rules, which represent the underlying semantics relevant to the recognition problem under consideration. The paper formulates the theoretical framework and a method for inferring general and optimal (according to certain criteria) descriptions of object classes from examples of classification or partial descriptions. The language for expressing the class descriptions and the guidance rules is an extension of the first-order predicate calculus, called variable-valued logic calculus VL21. VL21 involves typed variables and contains several new operators especially suited for conducting inductive inference, such as selector, internal disjunction, internal conjunction, exception, and generalization. Important aspects of the theory include: 1) a formulation of several kinds of generalization rules; 2) an ability to uniformly and adequately handle descriptors (i.e., variables, functions, and predicates) of different type (nominal, linear, and structured) and of different arity (i.e., different number of arguments); 3) an ability to generate new descriptors, which are derived from the initial descriptors through a rule-based system (i.e., an ability to conduct the so called constructive induction); 4) an ability to use the semantics underlying the problem under consideration. An experimental computer implementation of the method is briefly described and illustrated by an example.

The Logic of Plausible Reasoning: A Core Theory

Abstract The paper presents a core theory of human plausible reasoning based on analysis of peoples answers to everyday questions about the world. The theory consists of three parts: 1. 1. a formal representation of plausible inference patterns; such as deductions, inductions, and analogies, that are frequently employed in answering everyday questions; 2. 2. a set of parameters, such as conditional likelihood, typicality, and similarity, that affect the certainty of peoples answers to such questions; and 3. 3. a system relating the different plausible inference patterns and the different certainty parameters. This is one of the first attempts to construct a formal theory that addresses both the semantic and parametric aspects of the kind of everyday reasoning that pervades. all of human discourse.

Automated Construction of Classifications: Conceptual Clustering Versus Numerical Taxonomy

A method for automated construction of classifications called conceptual clustering is described and compared to methods used in numerical taxonomy. This method arranges objects into classes representing certain descriptive concepts, rather than into classes defined solely by a similarity metric in some a priori defined attribute space. A specific form of the method is conjunctive conceptual clustering, in which descriptive concepts are conjunctive statements involving relations on selected object attributes and optimized according to an assumed global criterion of clustering quality. The method, implemented in program CLUSTER/2, is tested together with 18 numerical taxonomy methods on two exemplary problems: 1) a construction of a classification of popular microcomputers and 2) the reconstruction of a classification of selected plant disease categories. In both experiments, the majority of numerical taxonomy methods (14 out of 18) produced results which were difficult to interpret and seemed to be arbitrary. In contrast to this, the conceptual clustering method produced results that had a simple interpretation and corresponded well to solutions preferred by people.

AN OVERVIEW OF MACHINE LEARNING

Learning is a many-faceted phenomenon. Learning processes include the acquisition of new declarative knowledge, the development of motor and cognitive skills through instruction or practice, the organization of new knowledge into general, effective representations, and the discovery of new facts and theories through observation and experimentation. Since the inception of the computer era, researchers have been striving to implant such capabilities in computers. Solving this problem has been, and remains, a most challenging and fascinating long-range goal in artificial intelligence (AI). The study and computer modeling of learning processes in their multiple manifestations constitutes the subject matter of machine learning.

A COMPARATIVE REVIEW OF SELECTED METHODS FOR LEARNING FROM EXAMPLES

Research in the area of learning structural descriptions from examples is reviewed, giving primary attention to methods of learning characteristic descriptions of single concepts. In particular, we examine methods for finding the maximally-specific conjunctive generalizations (MSC-generalizations) that cover all of the training examples of a given concept. Various important aspects of structural learning in general are examined, and several criteria for evaluating structural learning methods are presented. Briefly, these criteria include (i) adequacy of the representation language, (ii) generalization rules employed, (iii) computational efficiency, and (iv) flexibility and extensibility. Selected learning methods developed by Buchanan, et al., Hayes-Roth, Vere, Winston, and the authors are analyzed according to these criteria. Finally, some goals are suggested for future research.

A theory and methodology of inductive learning

The presented theory views inductive learning as a heuristic search through a space of symbolic descriptions, generated by an application of various inference rules to the initial observational statements. The inference rules include generalization rules, which perform generalizing transformations on descriptions, and conventional truth-preserving deductive rules. The application of the inference rules to descriptions is constrained by problem background knowledge, and guided by criteria evaluating the “quality” of generated inductive assertions.

LEARNABLE EVOLUTION MODEL: Evolutionary Processes Guided by Machine Learning

A new class of evolutionary computation processes is presented, called Learnable Evolution Model or LEM. In contrast to Darwinian-type evolution that relies on mutation, recombination, and selection operators, LEM employs machine learning to generate new populations. Specifically, in Machine Learning mode, a learning system seeks reasons why certain individuals in a population (or a collection of past populations) are superior to others in performing a designated class of tasks. These reasons, expressed as inductive hypotheses, are used to generate new populations. A remarkable property of LEM is that it is capable of quantum leaps (“insight jumps”) of the fitness function, unlike Darwinian-type evolution that typically proceeds through numerous slight improvements. In our early experimental studies, LEM significantly outperformed evolutionary computation methods used in the experiments, sometimes achieving speed-ups of two or more orders of magnitude in terms of the number of evolutionary steps. LEM has a potential for a wide range of applications, in particular, in such domains as complex optimization or search problems, engineering design, drug design, evolvable hardware, software engineering, economics, data mining, and automatic programming.

Inductive Learning of Structural Descriptions: Evaluation Criteria and Comparative Review of Selected Methods

Abstract Some recent work in the area of learning structural descriptions from examples is reviewed in light of the need in many diverse disciplines for programs which can perform conceptual data analysis, i.e., can describe complex data in terms of logical, functional, and causal relationships. Traditional data analysis techniques are not adequate for discovering such relationships. Primary attention is given to methods of learning the simplest form of generalization, namely, the maximally specific conjunctive generalizations (MSC-generalizations) which completely characterize a single set of structural examples. Various important aspects of structural learning in general are examined and criteria for evaluating learning methods are presented. The criteria include the adequacy of the representation language, generalization rules used, computational efficiency, and flexibility and extensibility. Selected learning methods, developed by Buchanan et al. [2–4, 32], Hayes-Roth [8–11], Vere [34–37], Winston [38, 39], and the authors, are analyzed according to these criteria. Finally some goals are suggested for future research.

Knowledge acquisition by encoding expert rules versus computer induction from examples: a case study involving soybean pathology

In view of growing interest in the development of knowledge-based computer consulting systems for various problem domains, the problems of knowledge acquisition have special significance. Current methods of knowledge acquisition rely entirely on the direct representation of knowledge of experts, which usually is a very time and effort consuming task. The paper presents results from an experiment to compare the above method of knowledge acquisition with a method based on inductive learning from examples. The compatison was done in the context of developing rules for soybean disease diagnosis and has demonstrated an advantage of the inductively derived rules in performing a testing task (which involved diagnosing a few hundred cases of soybean diseases).