Marc S. Atkin

Neo: learning conceptual knowledge by sensorimotor interaction with an environment

Recent developments in philosophy, linguistics, developmental psychology and arti cial intelligence make it possible to envision a developmental path for an arti cial agent, grounded in activity-based sensorimotor representations. This paper describes how Neo, an arti cial agent, learns concepts by interacting with its simulated environment. Relatively little prior structure is required to learn fairly accurate representations of objects, activities, locations and other aspects of Neos experience. We show how classes (categories) can be abstracted from these representations, and discuss how our representation might be extended to express physical schemas, general, domain-independent activities that could be the building blocks of concept formation.

Hierarchical agent control: a framework for defining agent behavior

The Hierarchical Agent Control Architecture (HAC) is a general toolkit for specifying an agents behavior. HAC supports action abstraction, resource management, sensor integration, and is well suited to controlling large numbers of agents in dynamic environments. It relies on three hierarchies: action, sensor, and context. The action hierarchy controls the agents behavior. It is organized around tasks to be accomplished, not the agents themselves. This facilitates the integration of multi-agent actions and planning into the architecture. The sensor hierarchy provides a principled means for structuring the complexity of reading and transforming sensor information. Each level of the hierarchy integrates the data coming in from the environment into conceptual chunks appropriate for use by actions at this level. Actions and sensors are written using the same formalism. The context hierarchy is a hierarchy of goals. In addition to their primary goals, most actions are operating within a set of implicit assumptions. These assumptions are made explicit through the context hierarchy. We have developed a planner, GRASP, implemented within HAC, which is capable of resolving multiple goals in real time. HAC was intended to have wide applicability. It has been used to control agents in commercial computer games and physical robots. Our primary application domain is a simulator of land-based military engagements called “Capture the Flag.” HACs simulation substrate models physics at an abstract level. HAC supports any domain in which behaviors can be reduced to a small set of primitive effectors such as {\sc move} and {\sc apply-force}. At this time defining agent behavior requires Lisp programming skills; we are moving towards more graphical programming languages.

Monitoring strategies for embedded agents: experiments and analysis

Monitoring is an important activity for any embedded agent. To operate effectively, agents must gather information about their environment. The policy by which they do this is called a monitoring strategy. Our work has focused on classifying different types of monitoring strategies and understanding how strategies depend on features of the task and environment. We have discovered only a few general monitoring strategies, in particular periodic and interval reduction, and speculate that there are no more. The relative advantages and generality of each strategy will be discussed in detail. The wide applicability of interval reduction will be demonstrated both empirically and analytically. We conclude with a number of general laws that state when a strategy is most appropriate.

Using simulation and critical points to define states in continuous search spaces

Many artificial intelligence techniques rely on the notion of a state as an abstraction of the actual state of the world, and an operator as an abstraction of the actions that take you from one state to the next. Much of the art of problem solving depends on choosing the appropriate set of states and operators. However, in realistic, and therefore dynamic and continuous search spaces, finding the right level of abstraction can be difficult. If too many states are chosen, the search space becomes intractable; if too few are chosen, important interactions between operators might be missed, making the search results meaningless. We present the idea of simulating operators using critical points as a way of dynamically defining state boundaries; new states are generated as part of the process of applying operators. Critical point simulation allows the use of standard search and planning techniques in continuous domains, as well as the incorporation of multiple agents, dynamic environments, and non-atomic variable length actions into the search algorithm. We conclude with examples of implemented systems that show how critical points are used in practice.

Learning monitoring strategies: a difficult genetic programming application

Finding optimal or at least good monitoring strategies is an important consideration when designing an agent. We have applied genetic programming to this task, with mixed results. Since the agent control language was kept purposefully general, the set of monitoring strategies constitutes only a small part of the overall space of possible behaviors. Because of this, it was often difficult for the genetic algorithm to evolve them, even though their performance was superior. These results raise questions as to how easy it will be for genetic programming to scale up as the areas it is applied to become more complex.<<ETX>>