Lee D. Erman

The Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty

The Hearsay-II system, developed during the DARPA-sponsored five-year speech-understanding research program, represents both a specific solution to the speech-understanding problem and a general framework for coordinating independent processes to achieve cooperative problem-solving behavior. As a computational problem, speech understanding reflects a large number of intrinsically interesting issues. Spoken sounds are achieved by a long chain of successive transformations, from intentions, through semantic and syntactic structuring, to the eventually resulting audible acoustic waves. As a consequence, interpreting speech means effectively inverting these transformations to recover the speakers intention from the sound. At each step in the interpretive process, ambiguity and uncertainty arise. The Hearsay-II problem-solving framework reconstructs an intention from hypothetical interpretations formulated at various levels of abstraction. In addition, it allocates limited processing resources first to the most promising incremental actions. The final configuration of the Hearsay-II system comprises problem-solving components to generate and evaluate speech hypotheses, and a focus-of-control mechanism to identify potential actions of greatest value. Many of these specific procedures reveal novel approaches to speech problems. Most important, the system successfully integrates and coordinates all of these independent activities to resolve uncertainty and control combinatorics. Several adaptations of the Hearsay-II framework have already been undertaken in other problem domains, and it is anticipated that this trend will continue; many future systems necessarily will integrate diverse sources of knowledge to solve complex problems cooperatively. Discussed in this paper are the characteristics of the speech problem in particular, the special kinds of problem-solving uncertainty in that domain, the structure of the Hearsay-II system developed to cope with that uncertainty, and the relationship between Hearsay-IIs structure and those of other speech-understanding systems. The paper is intended for the general computer science audience and presupposes no speech or artificial intelligence background.

Organization of the Hearsay II speech understanding system

Hearsay II (HSII) is a system currently under development at Carnegie-Mellon University to study the connected speech understanding problem. It is similar to Hearsay I (HSI) in that it is based on the hypothesize-and-test paradigm, using cooperating independent knowledge sources communicating with each other through a global data structure (blackboard). It differs in the sense that many of the limitations and shortcomings of HSI are resolved in HSII. The main new features of the Hearsay II system structure are: 1) the representation of knowledge as self-activating, asynchronous, parallel processes, 2) the representation of the partial analysis in a generalized three-dimensional network (the dimensions being level of representation (e.g., acoustic, phonetic, phonemic, lexical, syntactic), time, and alternatives) with contextual and structural support connections explicitly specified, 3) a convenient modular structure for incorporating new knowledge into the system at any level, and 4) a system structure suitable for execution on a parallel processing system. The main task domain under study is the retrieval of daily wire-service news stories upon voice request by the user. The main parametric representations used for this study are 1/3-octave filter-bank and linear-predictive coding (LPC)-derived vocal tract parameters [10], [11]. The acoustic segmentation and labeling procedures are parameter-independent [7]. The acoustic, phonetic, and phonological components [23] are feature-based rewriting rules which transform the segmental units into higher level phonetic units. The vocabulary size for the task is approximately 1200 words. This vocabulary information is used to generate word-level hypotheses from phonetic and surface-phonemic levels based on prosodic (stress) information. The syntax for the task permits simple English-like sentences and is used to generate hypotheses based on the probability of occurrence of that grammatical construct [19]. The semantic model is based on the news items of the day, analysis of the conversation, and the presence of certain content words in the partial analysis. This knowledge is to be represented as a production system. The system is expected to be operational on a 16-processor minicomputer system [3] being built at Carnegie-Mellon University. This paper deals primarily with the issues of the system organization of the HSII system.

The Hearsay‐II speech understanding system

The Hearsay‐II System has as its design goal recognition, understanding, and responding to connected speech utterances, particularly in situations where sentences cannot be guaranteed to agree with some predefined, restricted language model, as in the case of the Harpy System. Further, it attempts to view knowledge sources as different and independent which cannot always be integrated into single representation. It is based on the blackboard model [V. R. Lesser, R. D. Fennell, L. D. Erman, and D. R. Reddy, IEEE Trans. Acoust. Speech and Signal Process. ASSP‐23, 11–23 (1975) with knowledge sources as a set of parallel processes which are activated asynchronously depending on data events. The system performs on the Information Retrieval task with accuracy comparable to that of the Harpy system, but runs about 2 to 20 times slower. More complete performance results will be reported. As we get closer to unrestricted vocabularies and nongrammaticality of spoken languages, it will be necessary to have systems w...

ABE: an environment for engineering intelligent systems

The ABE multilevel architecture for developing intelligent systems addresses the key problems of intelligent systems engineering: large-scale applications and the reuse and integration of software components. ABE defines a virtual machine for module-oriented programming and a cooperative operating system that provides access to the capabilities of that virtual machine. On top of the virtual machine, ABE provides a number of system design and development frameworks, which embody such programming metaphors as control flow, blackboards, and dataflow. These frameworks support the construction of capabilities, including knowledge processing tools, which span a range from primitive modules to skeletal systems. Finally, applications can be built on skeletal systems. In addition, ABE supports the importation of existing software, including both conventional and knowledge processing tools. >

The HEARSAY Speech Understanding System

This talk describes the present state of performance of the HEARSAY system. [For more complete descriptions of the system see D. R. Reddy, L. D. Erman, and R. D. Neely, “A Model and a System for Machine Recognition of Speech,” IEEE Trans. Audio Electroacoust. AU‐21, 229–238 (1973) and D. R. Reddy, L. D. Erman, R. D. Fennell, and R. B. Neely, “The HEARSAY Speech Understanding System : An Example of the Recognition Process,” Proc. 3rd Int. Joint Conf. on Artificial Intelligence (Aug. 1973)]. The system uses task and context‐dependent information to help in the recognition of the utterance; this system consists of a set of cooperating parallel processes, each representing a different source of knowledge (e.g., acoustic‐phonetic, syntactic, semantic). The knowledge is used either to predict what may appear in a given context or to verify an hypothesis resulting from a previous prediction. Performance data of the system on several tasks (e.g., medical diagnosis, news retrieval, chess, and programming) will be ...

Noah-A Bottom-Up Word Hypothesizer for Large-Vocabulary Speech Understanding Systems

Current high-accuracy speech understanding systems achieve their performance at the cost of highly constrained grammars over relatively small vocabularies. Less-constrained systems will need to compensate for their loss of top-down constraint by improving bottom-up performance. To do this, they will need to eliminate from consideration at each place in the utterance most words in their vocabularies solely on the basis of acoustic information and expected pronunciations of the words. Towards this goal, we present the design and performance of Noah, a bottom-up word hypothesizer which is capable of handling large vocabularies-more than 10 000 words. Noah takes (machine) segmented and labeled speech as input and produces word hypotheses. The primary concern of this work is the problem of word hypothesizing from large vocabularies. Particular attention has been paid to accuracy, knowledge representation, knowledge acquisition, and flexibility. In this paper we discuss the problem of word hypothesizing, describe how the design of Noah faces these problems, and present the performance of Noah as a function of the vocabulary size.

Frameworks for developing intelligent systems: The ABE systems engineering environment

The development of a system engineering environment called ABE, a Better Environment, as part of the strategic computing program of the US Dept. of Defenses Defense Advanced Research Projects Agency (DARPA) is described. The goal was to create technologies and methodologies for building cooperative, intelligent systems with modular, heterogeneous components. The motivating problems are discussed. The specific needs of intelligent-system developers that the work addressed and the key technical approaches that were adopted are examined. The ABE software system is described. As an example of the use of ABE, PMR, a system for plan monitoring and replanning, is presented. The lessons learned in developing ABE and the issues remaining to be addressed are discussed.<<ETX>>

Overview of Teknowledge's domain-specific software architecture program

As part of the ARPA DSSA program, we are developing a methodology and integrating a suite of supporting tools to help specify, design, validate, package and deploy distributed intelligent control and management (DICAM) applications. Our domain of specialization is vehicle management systems, and our near-term focus is on advanced artillery systems. To attain higher levels of performance and functionality while reducing the time and cost required for development, we are recommending a generic control architecture suitable for use as a single intelligent agent or as multiple cooperating agents. This reference architecture combines a task-oriented domain controller with a meta-controller that schedules activities within the domain controller. The domain controller provides functions for model-based situation assessment and planning, and inter-controller communication. Typically, these functions are performed by components taken from a repository of reusable software. In tasks that are simple, deterministic or time-stressed, the modules may be complied into or replaced by conventional control algorithms. In complex, distributed, cooperative, non-deterministic or unstressed situations, these modules will usually exploit knowledge-based reasoning and deliberative control.To improve the controller development process, we are combining many of the best ideas from software engineering and knowledge engineering in a software environment. This environment includes a blackboard-like development workspace to represent both the software under development and the software development process itself. In this workspace, controllers are realized by mapping requirements into specializations of the reference architecture. The workspace also provides mechanisms for triggering applications of software tools, including knowledge-based software design assistants.We are currently in the third year of a five-year program. In conjunction with our collaborators at ARDEC, we have produced a schema for describing architectures which is being used by ARDECs community of contractors, by an ARPA architecture specification project for the Joint Task Force ATD, and by the Stanford Knowledge Systems Laboratory. We have released the second major version of our development environment, which is being used at ARDEC and in support of this ARPA architecture specification program. This version of the development environment is focused on initial requirements, architecture, and design. It provides both CASE-like editing of architectures and textual browsing/editing of repository descriptions expressed in the schema mentioned above. In the remaining years of the program we will be expanding the suite of tools and improving the methodologies required to build intelligent, distributed, hybrid controllers capable of spanning multiple levels of organization and system hierarchy. This technology holds considerable promise for near-term value, and the associated methodology provides a candidate approach for realizing the goals of mega-programming practice in control software. In assessing this prospect, we discuss some of the remaining shortfalls in both methodology and tools that require additional research and development.

ABE: a cooperative operating system and development environment

Publisher Summary This chapter provides an overview of ABE, a cooperative operating system and development environment. Teknowledge’s ABE™ system addresses the problem of combining conventional computing functions with knowledge processing capabilities. It enables the development of cooperative application systems that can exploit new-generation multiprocessing and distributed hardware. These are called new-generation applications intelligent systems, and ABE embodies excellent methods for engineering these systems. ABE is intended for system architects and application developers. It supports the exploratory and evolutionary development of applications that must integrate both conventional and knowledge processing capabilities. It provides several high-level graphical design and development environments, which we call frameworks. It encourages a high degree of modularity and facilitates radical reorganization of software components and the mapping of those components onto the hardware used to deploy them. ABE, in essence, provides an environment and operating system for intelligent systems. At present, Teknowledge offers the ABE software system as a prototype to a limited number of advanced users. These users typically face the problem of developing applications that must combine several software subsystems into an effective whole. These subsystems may employ conventional or AI capabilities.

Overview of the hearsay speech understanding research

Haarsay is the generic name for much of the speech understanding research in the computer science department at Carnegie-Mellon University (CMU). The major goals of this research include the investigation of computer knowledge-based problem-solving systems and the practical implementation of speech input to computers. An emphasis of this effort is the design of system structures for efficient implementation of such systems.