Anita Raja

BIG: an agent for resource-bounded information gathering and decision making

Abstract The World Wide Web has become an invaluable information resource but the explosion of available information has made Web search a time consuming and complex process. The large number of information sources and their different levels of accessibility, reliability and associated costs present a complex information gathering control problem. This paper describes the rationale, architecture, and implementation of a next generation information gathering system—a system that integrates several areas of Artificial Intelligence research under a single umbrella. Our solution to the information explosion is an information gathering agent, BIG, that plans to gather information to support a decision process, reasons about the resource trade-offs of different possible gathering approaches, extracts information from both unstructured and structured documents, and uses the extracted information to refine its search and processing activities.

The UMASS intelligent home project

Intelligent environments are an interesting development and research application problem for multi-agent systems. The functional and spatial distribution of tasks naturally lends itself to a multi-agent model and the existence of shared resources creates interactions over which the agents must coordinate. In the UMASS Intelligent Home project we have designed and implemented a set of distributed autonomous home control agents and deployed them in a simulated home environment. Our focus is primarily on resource coordination, though this project has multiple goals and areas of exploration ranging from the intellectual evaluation of the application as a general MAS testbed to the practical evaluation of our agent building and simulation tools.

A framework for meta-level control in multi-agent systems

Sophisticated agents operating in open environments must make decisions that efficiently trade off the use of their limited resources between dynamic deliberative actions and domain actions. This is the meta-level control problem for agents operating in resource-bounded multi-agent environments. Control activities involve decisions on when to invoke and the amount to effort to put into scheduling and coordination of domain activities. The focus of this paper is how to make effective meta-level control decisions. We show that meta-level control with bounded computational overhead allows complex agents to solve problems more efficiently than current approaches in dynamic open multi-agent environments. The meta-level control approach that we present is based on the decision-theoretic use of an abstract representation of the agent state. This abstraction concisely captures critical information necessary for decision making while bounding the cost of meta-level control and is appropriate for use in automatically learning the meta-level control policies.

Critical Infrastructure Integration Modeling and Simulation

The protection of critical infrastructures, such as electrical power grids, has become a primary concern of many nation states in recent years. Critical infrastructures involve multi-dimensional, highly complex collections of technologies, processes, and people, and as such, are vulnerable to potentially catastrophic failures on many levels. Moreover, cross-infrastructure dependencies can give rise to cascading and escalating failures across multiple infrastructures. In order to address the problem of critical infrastructure protection, our research is developing innovative approaches to modeling critical infrastructures, with emphasis on analyzing the ramifications of cross-infrastructure dependencies. This paper presents an initial overview of the research and of the modeling environment under development.

Cognitive Radio Resource Management Using Multi-Agent Systems

This paper investigates cooperative radio resource management for multiple cognitive radio networks in interference environments. The objective of this research is to manage shared radio resources fairly among multiple non- cooperative cognitive radio networks to optimize the overall performance. We emphasize the underlying predictability of network conditions and promote management solutions tailored to different interference environments. A multi-agent-system- based approach is proposed to achieve information sharing and decision distribution among multiple cognitive radio networks in a distributed manner. We address the distributed constraint optimization problem (DCOP) in cognitive radio networks and study the effectiveness of DCOP algorithms to find the optimal radio resource assignment through communications between distributed agents.

Toward robust agent control in open environments

Open environments are characterized by their uncertainty and non-determinism. This poses an inevitable challenge to construction of agents which need to operate in such environments. The agents need to adapt their processing to available resources, deadlines, the goal criteria specified by the clients as well their current problem solving context in order to survive. Our research focuses on constructing a framework for robust agent control, using a soft-real time scheduling approach which satisfices all aspects of problem solving. In this paper, we evaluate the performance of our heuristic-based approach using the performance of the policy generated by an optimal controller as the benchmark.

Modeling Uncertainty and its Implications to Sophisticated Control in Tæms Agents

Open environments are characterized by their uncertainty and non-determinism. Agents need to adapt their task processing to available resources, deadlines, the goal criteria specified by the clients as well their current problem solving context in order to survive in these environments. If there were no resource constraints, then an optimal Markov Decision Process based policy would obviously be the best way for complex problem solving agents to make scheduling decisions. However in many agent systems, these scheduling decisions have to be made on-line or in soft real-time, making the off-line policy computationally infeasible in open environments. The hybrid planner/scheduler used to control Task Analysis, Environment Modeling, and Simulation (TÆMS) agents is the Design-to-Criteria (DTC) agent scheduler. Design-to-Criteria scheduling is the soft real-time process of custom building a plan/schedule to meet an agent’s current objectives which are expressed as dynamic goal criteria (including real-time deadlines), using task models that describe alternate ways to achieve tasks and subtasks. Recent advances in Design-to-Criteria control include the addition of uncertainty to the TÆMS computational task models analyzed by the scheduler and the incorporation of uncertainty in the scheduling process. As we show, the use of uncertainty in TÆMS and Design-to-Criteria enables agents to make better control decisions in uncertain environments. Design-to-Criteria uses a heuristic approach for on-line scheduling of medium granularity tasks.It approximates the analysis used to generate an optimal policy by heuristically reasoning about the implications of uncertainty in task execution. The addition of uncertainty has also spawned a post-scheduling contingency analysis step for situations in which an agent must produce a result by a given deadline (deadline critical situations) and where the added computational cost is worth the expense. We describe the uncertainty representation in TÆMS and how it improves task models and the scheduling process, and provide empirical examples of reasoning about uncertainty in action. We also evaluate the performance of our heuristic-based approach to agent control using the performance of the policy generated by an optimal controller as the benchmark.

Meta-level reasoning in deliberative agents

Deliberative agents operating in open environments must make complex real-time decisions on scheduling and coordination of domain activities. These decisions are made in the context of limited resources and uncertainty about the outcomes of activities. We describe a reinforcement learning based approach for efficient meta-level reasoning. Empirical results showing the effectiveness of meta-level reasoning in a complex domain are provided.

Emergence of Cooperation Using Commitments and Complex Network Dynamics

In this paper, our goal is to achieve the emergence of cooperation in self-interested agent societies operating on highly connected scale-free networks. The novelty of this work is that agents are able to control topological features during the network formation phase. We propose a commitment-based dynamic coalition formation approach that result in a single coalition where agents mutually cooperate. Agents play an iterated Prisoners Dilemma game with their immediate neighbors and offer commitments to their wealthiest neighbors in order to form coalitions. A commitment proposal, that includes a high breaching penalty, incentivizes opponent agents to form coalitions within which they mutually cooperate and thereby increase their payoff. We have analytically determined, and experimentally substantiated, how the value of the penalty should be set with respect to the minimum node degree and the payoff values such that convergence into optimal coalitions is possible. Using a computational model, we determine an appropriate partner selection strategy for the agents that results in a network facilitating the convergence into a single coalition and thereby maximizing average expected payoff.

Multiagent meta-level control for radar coordination

It is crucial for embedded systems to adapt to the dynamics of open environments. This adaptation process becomes especially challenging in the context of multiagent systems. In this paper, we argue that multiagent meta-level control is an effective way to determine when this adaptation process should be done and how much effort should be invested in adaptation as opposed to continuing with the current action plan. We use a reinforcement learning based local optimization algorithm within each agent to learn multiagent meta-level control agent policies in a decentralized fashion. These policies will allow each agent to adapt to changes in environmental conditions while reorganizing the underlying multiagent network when needed. We then augment the agent with a heuristic rule-based algorithm that uses information provided by the reinforcement learning algorithm in order to resolve conflicts among agent policies from a local perspective at both learning and execution stages. We evaluate this mechanism in the context of a multiagent tornado tracking application called NetRads. Empirical results show that adaptive multiagent meta-level control significantly improves the performance of the tornado tracking network for a variety of weather scenarios.