Samuel Wintermute

Imagery in cognitive architecture: Representation and control at multiple levels of abstraction

In a cognitive architecture, intelligent behavior is contingent upon the use of an appropriate abstract representation of the task. When designing a general-purpose cognitive architecture, two basic challenges related to abstraction arise, which are introduced and examined in this article. The perceptual abstraction problem results from the difficulty of creating a single perception system able to induce appropriate abstract representations in any task the agent might encounter, and the irreducibility problem arises because some tasks are resistant to being abstracted at all. The first contribution of this paper is identifying these problems, and the second contribution is showing a means to address them. This is accomplished through the use of mental imagery. To support imagery, a concrete (highly detailed) representation of the spatial state of the problem is maintained as an intermediate between the external world and an abstract representation. Actions can be simulated (imagined) in terms of this concrete representation, and the agent can derive abstract information by applying perceptual processes to the resulting concrete state. Imagery works to mitigate the perceptual abstraction problem by allowing a given perception system to work in a wider variety of tasks, since perception can be dynamically combined with imagery, and works to mitigate the irreducibility problem by allowing internal simulation of low-level control processes. To demonstrate these benefits, an implementation is described, which is an extension of the Soar architecture. An agent in this architecture that uses reinforcement learning and imagery to play an arcade game and an agent that performs sampling-based motion planning for a car-like vehicle are described, demonstrating the perceptual abstraction and irreducibility problems and the associated use of imagery to mitigate those problems in complex AI tasks. Previous AI systems have incorporated imagery-like processes, however, the functional benefit of imagery in those systems has typically been characterized as the ability to perform more efficient inference through the use of a specialized representation. The use of imagery here shows further benefits related to the perceptual abstraction and irreducibility problems, enriching the broader understanding of the role of imagery in cognitive systems.

Exploring the Functional Advantages of Spatial and Visual Cognition From an Architectural Perspective

We present a general cognitive architecture that tightly integrates symbolic, spatial, and visual representations. A key means to achieving this integration is allowing cognition to move freely between these modes, using mental imagery. The specific components and their integration are motivated by results from psychology, as well as the need for developing a functional and efficient implementation. We discuss functional benefits that result from the combination of multiple content-based representations and the specialized processing units associated with them. Instantiating this theory, we then discuss the architectural components and processes, and illustrate the resulting functional advantages in two spatially and visually rich domains. The theory is then compared to other prominent approaches in the area.