Keith R. Levi

A goal-driven approach to enterprise component identification and specification

Mapping a business architecture to a component-based software architecture.

Expert systems should be more accurate than human experts: evaluation procedures from human judgement and decision making

Two procedures for the evaluation of the performances of expert systems are illustrated: one procedure evaluates predictive accuracy; the other procedure is complementary in that it uncovers the factors that contribute to predictive accuracy. Using these procedures, it is argued that expert systems should be more accurate than human experts in two senses. One sense is that expert systems must be more accurate to be cost-effective. Previous research is reviewed and original results are presented which show that simple statistical models typically perform better than human experts for the task of combining evidence from a given set of information sources. The results also suggest the second sense in which expert systems should be more accurate than human experts. They reveal that expert systems should share factors that contribute to human accuracy, but not factors that detract from human accuracy. Thus the thesis is that one should both require and expect systems to be more accurate than humans. >

A formal analysis of machine learning systems for knowledge acquisition

Abstract Machine learning techniques can be of great value for automating certain aspects of knowledge acquisition. Given the potential of machine learning for knowledge acquisition, we have begun a systematic investigation of how one might map the functions of knowledge-based systems onto those machine learning systems that provide the required knowledge. The goal of our current research is to provide a general characterization of machine learning systems and their respective application domains.

Learning plans for an intelligent assistant by observing user behavior

A critical requirement of intelligent automated assistants is a representation of actions and goals that is common to both the user and the automated assistant. Updating the intelligent systems knowledge base by observing user behavior is a convenient method for acquiring this common representation. We are developing an explanation based learning system to automate the acquisition of new plans for a large pilot-aiding expert system. We have developed a planning/learning shell that is based on the TWEAK planning system and DeJongs explanation based learning system. We are applying this shell to the pilot-aiding problem in a joint industry/university research effort involving Honeywell, Lockheed, ISX, Search Technology, and the Universities of Illinois and Michigan.

An explanation-based-learning approach to knowledge compilation: a Pilot's Associate application

The use of explanation-based learning as part of a larger knowledge compilation system for automating the development and maintenance of associate knowledge bases is discussed. Specifically, the implemented systems of the Learning Systems for Pilot Aiding (LSPA), which automates portions of the offline process of incorporating new information into the knowledge bases of Pilots Associate, one of the largest and most thoroughly developed associate systems, and then propagates pertinent changes to other Pilots Associate modules, are described. The learning algorithm and some modifications to it are also described. It is shown that the general approach should be relevant and easily generalizable to other intelligent associate systems, such as a submarine commanders associate and a helicopter pilots associate. For substantially different systems, explanation-based learning should be generally valid as a front end for knowledge compilation.<<ETX>>

Identifying knowledge base deficiencies by observing user behavior

We are developing an application of explanation based learning to refine and complete the knowledge base of an expert pilots assistant. A companion paper in this volume reports on the issues specific to planning and temporal (Perschbacher, Levi & Shalin.) In this paper we focus on the role of learning experiences in our project, and how they are used to direct the refinement of the knowledge base of the pilots assistant. The knowledge base of the assistant must share a representation of actions and goals that is common to the user in order to coordinate activity with the user. The first knowledge base refinement problem in this project is to identify deficiencies in the systems knowledge base by observing and explaining unexpected user behavior. The second knowledge base refinement problem is to refine the knowledge base of the underlying EBL system. We present our approach to the first problem, and some comments on the second problem.

The importance of implicit and explicit knowledge in a pilot's associate system

It is pointed out that fielding an operational pilots associate (PA) will require both implicit and explicit representations of knowledge. Speed and memory performance requirements for PA will be aided by the use of implicit representations of knowledge. Acquiring and maintaining the large knowledge bases for PA will, by contrast, be aided by having explicit knowledge representations. Such explicit representations are being investigated in a 10 person-year research project sponsored by the Wright Research and Development Center. A critical contribution of this research has been to develop concepts that make machine learning applicable to real-time control and execution systems such as pilots associate. The authors describe how machine learning techniques can automatically transform explicit representations into the implicit representations required by PA.<<ETX>>

Learning tactical plans for pilot aiding

ABSTRACT We are developing an explanation based learning system to automate the acquisition of new plans for a large pilot-aiding expert system. We have developed a planning/learning shell that is based on DeJongs Explanation Based Learning system (DeJong & Mooney, 1986).