Minjie Zhang

A belief propagation-based method for task allocation in open and dynamic cloud environments

Abstract We propose a decentralized belief propagation-based method, PD-LBP, for multi-agent task allocation in open and dynamic grid and cloud environments where both the sets of agents and tasks constantly change. PD-LBP aims at accelerating the online response to, improving the resilience from the unpredicted changing in the environments, and reducing the message passing for task allocation. To do this, PD-LBP devises two phases, pruning and decomposition. The pruning phase focuses on reducing the search space through pruning the resource providers, and the decomposition addresses decomposing the network into multiple independent parts where belief propagation can be operated in parallel. Comparison between PD-LBP and two other state-of-the-art methods, Loopy Belief Propagation-based method and Reduced Binary Loopy Belief Propagation based method, is performed. The evaluation results demonstrate the desirable efficiency of PD-LBP from both the shorter problem solving time and smaller communication requirement of task allocation in dynamic environments.

Conceptual Design of A Multi-Agent System for Interconnected Power Systems Restoration

Outages and faults in interconnected power systems may cause cascading sequences of events, and catastrophic failures of power systems. How to efficiently manage power systems and restore the systems from faults is a challenging research issue in power engineering. Multi-agent systems are employed to address such a challenge in recent years. A centralized coordination strategy was firstly introduced to manage agents in a power system. Such a strategy usually adopts a single central coordinator to control the whole system for system management, maintenance, and restoration purposes. However, disadvantages such as deficiencies in robustness, openness, and flexibility prevent this strategy from extensive online applications. Consequently, a decentralized coordination strategy was proposed to overcome such limitations. But the decentralized coordination strategy cannot efficiently provide a global solution when serious faults spread out in a power system. In this paper, a conceptual multi-agent system design is introduced to express our proposal in power system modeling. A novel dynamic team forming mechanism is proposed to dynamically manage agents in power system with a flexible coordination structure, so as to balance the effectiveness and efficiency of the introduced multi-agent system. The results from simulations of case studies indicate the performance of the proposed multi-agent model.

A Multi-Agent Solution to Distribution System Management by Considering Distributed Generators

A traditional distribution network carries electricity from a central power resource to consumers, and the power dispatch is controlled centrally. Distributed generators (DGs) emerge as an alternative power resource to distribution networks at a smaller and distributed scale, which will bring benefits such as reduced voltage drop and loss. However, because most of high penetration DGs are not utility owned and characterized by high degree of uncertainty such as solar and wind, the distribution network may perform differently from the conventionally expected behaviors. How to dynamically and efficiently manage the power dispatch in a distribution network to balance the supply and demand by considering the variability of DGs and loads becomes a significant research issue. In this paper, a multi-agent system (MAS) was proposed to solve this problem through introducing five types of autonomous agents, the electricity management mechanisms, the agent communication ontology, and the agent cooperation strategy. The simulation of the MAS by using InterPSS, JADE and JUNE well demonstrates the performance of the system on dynamic supply and demand balance by considering both efficiency and economy.

Adaptive conceding strategies for automated trading agents in dynamic, open markets

One of the crucial issues of automated negotiation in multi-agent systems is how to reach an agreement when a negotiation environment becomes open and dynamic. Even though some strategies have been proposed by researchers, most of them can only work within a static negotiation environment. In this paper, we present a model for designing a strategy for agents that makes adjustable rates of concession by negotiating according to the changes of environments with uncertain and dynamic outside options. This proposal is based on the market-driven agents (MDAs) model, and is guided by four factors in order to determine the degree of concession. These factors are trading opportunity, trading competition, trading time and strategy, and eagerness. The contribution of this paper is extending the MDAs model to an open and dynamic negotiation environment by considering both the current and potential changes of the environment.

KEMNAD: A Knowledge Engineering Methodology For Negotiating Agent Development

Automated negotiation is widely applied in various domains. However, the development of such systems is a complex knowledge and software engineering task. So, a methodology there will be helpful. Unfortunately, none of existing methodologies can offer sufficient, detailed support for such system development. To remove this limitation, this paper develops a new methodology made up of (1) a generic framework (architectural pattern) for the main task, and (2) a library of modular and reusable design pattern (templates) of subtasks. Thus, it is much easier to build a negotiating agent by assembling these standardized components rather than reinventing the wheel each time. Moreover, because these patterns are identified from a wide variety of existing negotiating agents (especially high impact ones), they can also improve the quality of the final systems developed. In addition, our methodology reveals what types of domain knowledge need to be input into the negotiating agents. This in turn provides a basis for developing techniques to acquire the domain knowledge from human users. This is important because negotiation agents act faithfully on the behalf of their human users and thus the relevant domain knowledge must be acquired from the human users. Finally, our methodology is validated with one high impact system.

Self-Adaptation-Based Dynamic Coalition Formation in a Distributed Agent Network: A Mechanism and a Brief Survey

In some real systems, e.g., distributed sensor networks, individual agents often need to form coalitions to accomplish complex tasks. Due to communication and computation constraints, it is infeasible for agents to directly interact with all other agents to form coalitions. Most previous coalition formation studies, however, overlooked this aspect. Those studies did not provide an explicitly modeled agent network or assumed that agents were in a fully connected network, where an agent can directly communicate with all other agents. Thus, to alleviate this problem, it is necessary to provide a neighborhood network structure, within which agents can directly interact only with their neighbors. Toward this end, in this paper, a self-adaptation-based dynamic coalition formation mechanism is proposed. The proposed mechanism operates in a neighborhood agent network. Based on self-adaptation principles, this mechanism enables agents to dynamically adjust their degrees of involvement in multiple coalitions and to join new coalitions at any time. The self-adaptation process, i.e., agents adjusting their degrees of involvement in multiple coalitions, is realized by exploiting a negotiation protocol. The proposed mechanism is evaluated through a comparison with a centralized mechanism (CM) and three other coalition formation mechanisms. Experimental results demonstrate the good performance of the proposed mechanism in terms of the entire network profit and time consumption. Additionally, a brief survey of current coalition formation research is also provided. From this survey, readers can have a general understanding of the focuses and progress of current research. This survey provides a classification of the primary emphasis of each related work in coalition formation, so readers can conveniently find the most related studies.

Multi-Objective Service Composition Using Reinforcement Learning

Web services have the potential to offer the enterprises with the ability to compose internal and external business services in order to accomplish complex processes. Service composition then becomes an increasingly challenging issue when complex and critical applications are built upon services with different QoS criteria. However, most of the existing QoS-aware compositions are simply based on the assumption that multiple criteria, no matter whether these multiple criteria are conflicting or not, can be combined into a single criterion to be optimized, according to some utility functions. In practice, this can be very difficult as utility functions or weights are not well known a priori. In this paper, a novel multi-objective approach is proposed to handle QoS-aware Web service composition with conflicting objectives and various restrictions on quality matrices. The proposed approach uses reinforcement learning to deal with the uncertainty characteristic inherent in open and decentralized environments. Experimental results reveal the ability of the proposed approach to find a set of Pareto optimal solutions, which have the equivalent quality to satisfy multiple QoS-objectives with different user preferences.

A Hybrid Multiagent Framework With Q-Learning for Power Grid Systems Restoration

This paper presents a hybrid multiagent framework with a Q-learning algorithm to support rapid restoration of power grid systems following catastrophic disturbances involving loss of generators. This framework integrates the advantages of both centralized and decentralized architectures to achieve accurate decision making and quick responses when potential cascading failures are detected in power systems. By using this hybrid framework, which does not rely on a centralized controller, the single point of failure in power grid systems can be avoided. Further, the use of the Q-learning algorithm developed in conjunction with the restorative framework can help the agents to make accurate decisions to protect against cascading failures in a timely manner without requiring a global reward signal. Simulation results demonstrate the effectiveness of the proposed approach in comparison with the typical centralized and decentralized approaches based on several evaluation attributes.

New Trends in Agent-Based Complex Automated Negotiations

Complex Automated Negotiations represent an important, emerging area in the field of Autonomous Agents and Multi-Agent Systems. Automated negotiations can be complex, since there are a lot of factors that characterize such negotiations. These factors include the number of issues, dependencies between these issues, representation of utilities, the negotiation protocol, the number of parties in the negotiation (bilateral or multi-party), time constraints, etc. Software agents can support automation or simulation of such complex negotiations on the behalf of their owners, and can provide them with efficient bargaining strategies. To realize such a complex automated negotiation, we have to incorporate advanced Artificial Intelligence technologies includes search, CSP, graphical utility models, Bayes nets, auctions, utility graphs, predicting and learning methods. Applications could include e-commerce tools, decision-making support tools, negotiation support tools, collaboration tools, etc. This book aims to provide a description of the new trends in Agent-based, Complex Automated Negotiation, based on the papers from leading researchers. Moreover, it gives an overview of the latest scientific efforts in this field, such as the platform and strategies of automated negotiating techniques.

Collective learning for the emergence of social norms in networked multiagent systems

Social norms such as social rules and conventions play a pivotal role in sustaining system order by regulating and controlling individual behaviors toward a global consensus in large-scale distributed systems. Systematic studies of efficient mechanisms that can facilitate the emergence of social norms enable us to build and design robust distributed systems, such as electronic institutions and norm-governed sensor networks. This paper studies the emergence of social norms via learning from repeated local interactions in networked multiagent systems. A collective learning framework, which imitates the opinion aggregation process in human decision making, is proposed to study the impact of agent local collective behaviors on the emergence of social norms in a number of different situations. In the framework, each agent interacts repeatedly with all of its neighbors. At each step, an agent first takes a best-response action toward each of its neighbors and then combines all of these actions into a final action using ensemble learning methods. Extensive experiments are carried out to evaluate the framework with respect to different network topologies, learning strategies, numbers of actions, influences of nonlearning agents, and so on. Experimental results reveal some significant insights into the manipulation and control of norm emergence in networked multiagent systems achieved through local collective behaviors.