Attilio Giordana

Search-intensive concept induction

This paper describes REGAL, a distributed genetic algorithm-based system, designed for learning first-order logic concept descriptions from examples. The system is a hybrid of the Pittsburgh and the Michigan approaches, as the population constitutes a redundant set of partial concept descriptions, each evolved separately. In order to increase effectiveness, REGAL is specifically tailored to the concept learning task; hence, REGAL is task-dependent, but, on the other hand, domain-independent. The system proved particularly robust with respect to parameter setting across a variety of different application domains. REGAL is based on a selection operator, called Universal Suffrage operator, provably allowing the population to asymptotically converge, on the average, to an equilibrium state in which several species coexist. The system is presented in both a serial and a parallel version, and a new distributed computational model is proposed and discussed. The system has been tested on a simple artificial domain for the sake of illustration, and on several complex real-world and artificial domains in order to show its power and to analyze its behavior under various conditions. The results obtained so far suggest that genetic search may be a valuable alternative to logic-based approaches to learning concepts, when no (or little) a priori knowledge is available and a very large hypothesis space has to be explored.

Modeling production rules by means of predicate transition networks

Abstract A formal tool, the PREDICATE TRANSITION networks, is proposed for representing rules in production systems. These networks have been introduced by Genrich and Lautenbach as a kind of high-level Petri nets. The peculiarity of such a formal tool lies in its well-formalized semantics, which can be described by means of an algebraic formalism and can be analyzed using the S-invariant method in order to prove properties of the network. A brief review of PREDICATE TRANSITION networks is followed by a description of their use in AI applications. In particular the characteristics which make them suitable for representing knowledge and control strategies in expert systems are analyzed.

A KNOWLEDGE INTENSIVE APPROACH TO CONCEPT INDUCTION

In this paper we present a concept acquisition methodology that uses data (concept examples and counterexamples), domain knowledge and tentative concept descriptions in an integrated way. Domain knowledge can be incomplete and/or incorrect with respect to the given data; moreover, the tentative concept descriptions can be expressed in a form which is not operational. The methodology is aimed at producing discriminant and operational concept descriptions, by integrating inductive and deductive learning. In fact, the domain theory is used in a deductive process, that tries to operationalize the tentative concept descriptions, but the obtained results are tested on the whole learning set rather than on a single example. Moreover, deduction is interleaved with the application of data-driven inductive steps. In this way, a search in a constrained space of possible descriptions can help overcome some limitations of the domain theory (e.g. inconsistency). The method has been tested in the framework of the inductive learning system, “ML-SMART,”, previously developed by the authors, and a simple example is also given.

Automated concept acquisition in noisy environments

A system that performs automated concept acquisition from examples and has been specially designed to work in noisy environments is presented. The learning methodology is aimed at the target problem of finding discriminant descriptions of a given set of concepts and uses both examples and counterexamples. The learned knowledge is expressed in the form of production rules, organized into separate clusters, linked together in a graph structure. Knowledge extraction is guided by a top-down control strategy, through a process of specialization. The system also utilizes a technique of problem reduction to contain the computational complexity. Several criteria are proposed for evaluating the acquired knowledge. The methodology has been tested on a problem in the field of speech recognition and the experimental results obtained are reported and discussed. >

Phase Transitions in Relational Learning

One of the major limitations of relational learning is due to the complexity of verifying hypotheses on examples. In this paper we investigate this task in light of recent published results, which show that many hard problems exhibit a narrow “phase transition” with respect to some order parameter, coupled with a large increase in computational complexity. First we show that matching a class of artificially generated Horn clauses on ground instances presents a typical phase transition in solvability with respect to both the number of literals in the clause and the number of constants occurring in the instance to match. Then, we demonstrate that phase transitions also appear in real-world learning problems, and that learners tend to generate inductive hypotheses lying exactly on the phase transition. On the other hand, an extensive experimenting revealed that not every matching problem inside the phase transition region is intractable. However, unfortunately, identifying those that are feasible cannot be done solely on the basis of the order parameters. To face this problem, we propose a method, based on a Monte Carlo algorithm, to estimate on-line the likelihood that the current matching problem will exceed a given amount of computational resources. The impact of the above findings on relational learning is discussed.

Guiding induction with domain theories

Abstract In this chapter we present a concept-acquisition methodology that uses data (concept examples and counterexamples), domain knowledge, and tentative concept descriptions in an integrated way. Domain knowledge can be incomplete and/or incorrect with respect to the given data; moreover, the tentative concept descriptions can be expressed in a form that is not operational. The methodology is aimed at producing discriminant and operational concept descriptions, by integrating inductive and deductive learning. In fact, the domain theory is used in a deductive process, that tries to operationalize the tentative concept descriptions, but the obtained results are tested on the whole learning set rather than on a single example. Moreover, deduction is interleaved with the application of data-driven inductive steps. In this way, a search in a constrained space of possible descriptions can help overcome some limitations of the domain theory (e.g., inconsistency). The method has been tested in the framework of the inductive learning system “ML-SMART,” previously developed by the authors, and a simple example is also given.

ENIGMA: a system that learns diagnostic knowledge

The results of extensive experimentation aimed at assessing the concrete possibilities of automatically building a diagnostic expert system, to be used in-field in an industrial domain, by means of machine learning techniques, are described. The system, ENIGMA, is an incremental version of the ML-SMART system, which acquires a network of first-order logic rules, starting from a set of classified examples and a domain theory. An application is described that consists of discovering malfunctions in electromechanical apparatus. ENIGMAs efficacy in acquiring sophisticated knowledge and handling complex structured examples is largely due to its underlying database management system, which supports the learning operators, defined at the abstract level, with a set of primitives, taken from the field of deductive databases. An expert system, MEPS, devoted to the same task, has also been manually developed. A number of comparisons along different dimensions of the manual and automatic development process have been possible, allowing some practical indications to be suggested. >

Integrating Multiple Learning Strategies in First Order Logics

This paper describes a representation framework that offers a unifying platform for alternative systems, which learn concepts in First Order Logics. The main aspects of this framework are discussed. First of all, the separation between the hypothesis logical language (a version of the VL21 language) and the representation of data by means of a relational database is motivated. Then, the functional layer between data and hypotheses, which makes the data accessible by the logical level through a set of abstract properties is described. A novelty, in the hypothesis representation language, is the introduction of the construct of internal disjunction; such a construct, first used by the AQ and Induce systems, is here made operational via a set of algorithms, capable to learn it, for both the discrete and the continuous-valued attributes case. These algorithms are embedded in learning systems (SMART+, REGAL, SNAP, WHY, RTL) using different paradigms (symbolic, genetic or connectionist), thus realizing an effective integration among them; in fact, categorical and numerical attributes can be handled in a uniform way. In order to exemplify the effectiveness of the representation framework and of the multistrategy integration, the results obtained by the above systems in some application domains are summarized.

Deduction in top-down inductive learning

Publisher Summary This chapter reviews a flexible strategy for combining analytic and empirical learning to acquire conceptual descriptions in real domains such as fault diagnosis or medical diagnosis, in which imperfect and intractable theories are, in general, available. The learning strategy is seen in the framework of a learning model (presented by the authors in F. Bergadano and A. Giordana, 1988) based on a top-down specialization process, which can now be guided by interleaving deductive and inductive steps. The chapter highlights two main novelties in comparison to earlier ideas. The first novelty consists in the adoption of both logical and dependency relations for describing the domain theories, in which it is possible to distinguish axioms that are sure from axioms which are just possible and other ones which can be just partially specified. This knowledge representation formalism can be effectively exploited to reflect the real domain knowledge possessed by a technician and offers a good background to decide where and how to activate deductive or inductive steps, and where to hypothesize theory incompleteness and inconsistencies. The second novelty consists in the repertory and in the flexibility of the reasoning schemes that can be used in the learning process.

An expert system for mapping acoustic cues into phonetic features

Abstract The paper describes the conception of the sublexical levels of a speech-understanding system as a society of experts. Experts cooperate in extracting and describing acoustic cues, generating and verifying phonetic hypotheses, and accessing a large lexicon. The knowledge of each expert is described by a frame language which allows integration between structural and procedural knowledge. Structural knowledge deals with relations between facts like acoustic-cue descriptions and phonetic-feature hypotheses. Procedural knowledge deals with rules for the use of relations, for the generation of contextual constraints for relation application, and for the extraction of new cues in specified contexts. The main purpose of the research proposed here is that of providing at the same time a model for computer perception and algorithms useful for designing complex systems operating in real time. Some experimental results on the performance of the proposed system are reported.