Fenghui Ren

Conceptual Design of A Multi-Agent System for Interconnected Power Systems Restoration

Outages and faults in interconnected power systems may cause cascading sequences of events, and catastrophic failures of power systems. How to efficiently manage power systems and restore the systems from faults is a challenging research issue in power engineering. Multi-agent systems are employed to address such a challenge in recent years. A centralized coordination strategy was firstly introduced to manage agents in a power system. Such a strategy usually adopts a single central coordinator to control the whole system for system management, maintenance, and restoration purposes. However, disadvantages such as deficiencies in robustness, openness, and flexibility prevent this strategy from extensive online applications. Consequently, a decentralized coordination strategy was proposed to overcome such limitations. But the decentralized coordination strategy cannot efficiently provide a global solution when serious faults spread out in a power system. In this paper, a conceptual multi-agent system design is introduced to express our proposal in power system modeling. A novel dynamic team forming mechanism is proposed to dynamically manage agents in power system with a flexible coordination structure, so as to balance the effectiveness and efficiency of the introduced multi-agent system. The results from simulations of case studies indicate the performance of the proposed multi-agent model.

A Multi-Agent Solution to Distribution System Management by Considering Distributed Generators

A traditional distribution network carries electricity from a central power resource to consumers, and the power dispatch is controlled centrally. Distributed generators (DGs) emerge as an alternative power resource to distribution networks at a smaller and distributed scale, which will bring benefits such as reduced voltage drop and loss. However, because most of high penetration DGs are not utility owned and characterized by high degree of uncertainty such as solar and wind, the distribution network may perform differently from the conventionally expected behaviors. How to dynamically and efficiently manage the power dispatch in a distribution network to balance the supply and demand by considering the variability of DGs and loads becomes a significant research issue. In this paper, a multi-agent system (MAS) was proposed to solve this problem through introducing five types of autonomous agents, the electricity management mechanisms, the agent communication ontology, and the agent cooperation strategy. The simulation of the MAS by using InterPSS, JADE and JUNE well demonstrates the performance of the system on dynamic supply and demand balance by considering both efficiency and economy.

Adaptive conceding strategies for automated trading agents in dynamic, open markets

One of the crucial issues of automated negotiation in multi-agent systems is how to reach an agreement when a negotiation environment becomes open and dynamic. Even though some strategies have been proposed by researchers, most of them can only work within a static negotiation environment. In this paper, we present a model for designing a strategy for agents that makes adjustable rates of concession by negotiating according to the changes of environments with uncertain and dynamic outside options. This proposal is based on the market-driven agents (MDAs) model, and is guided by four factors in order to determine the degree of concession. These factors are trading opportunity, trading competition, trading time and strategy, and eagerness. The contribution of this paper is extending the MDAs model to an open and dynamic negotiation environment by considering both the current and potential changes of the environment.

Collective learning for the emergence of social norms in networked multiagent systems

Social norms such as social rules and conventions play a pivotal role in sustaining system order by regulating and controlling individual behaviors toward a global consensus in large-scale distributed systems. Systematic studies of efficient mechanisms that can facilitate the emergence of social norms enable us to build and design robust distributed systems, such as electronic institutions and norm-governed sensor networks. This paper studies the emergence of social norms via learning from repeated local interactions in networked multiagent systems. A collective learning framework, which imitates the opinion aggregation process in human decision making, is proposed to study the impact of agent local collective behaviors on the emergence of social norms in a number of different situations. In the framework, each agent interacts repeatedly with all of its neighbors. At each step, an agent first takes a best-response action toward each of its neighbors and then combines all of these actions into a final action using ensemble learning methods. Extensive experiments are carried out to evaluate the framework with respect to different network topologies, learning strategies, numbers of actions, influences of nonlearning agents, and so on. Experimental results reveal some significant insights into the manipulation and control of norm emergence in networked multiagent systems achieved through local collective behaviors.

Emotional Multiagent Reinforcement Learning in Spatial Social Dilemmas

Social dilemmas have attracted extensive interest in the research of multiagent systems in order to study the emergence of cooperative behaviors among selfish agents. Understanding how agents can achieve cooperation in social dilemmas through learning from local experience is a critical problem that has motivated researchers for decades. This paper investigates the possibility of exploiting emotions in agent learning in order to facilitate the emergence of cooperation in social dilemmas. In particular, the spatial version of social dilemmas is considered to study the impact of local interactions on the emergence of cooperation in the whole system. A double-layered emotional multiagent reinforcement learning framework is proposed to endow agents with internal cognitive and emotional capabilities that can drive these agents to learn cooperative behaviors. Experimental results reveal that various network topologies and agent heterogeneities have significant impacts on agent learning behaviors in the proposed framework, and under certain circumstances, high levels of cooperation can be achieved among the agents.

Bilateral single-issue negotiation model considering nonlinear utility and time constraint

Bilateral agent negotiation is considered as a fundamental research issue in autonomous agent negotiation, and was studied well by researchers. Generally, a predefined negotiation decision function and utility function are used to generate an offer in each negotiation round according to a negotiators negotiation strategy, preference, and restrictions. However, such a negotiation procedure may not work well when the negotiators utility function is nonlinear, and the unique offer is difficult to be generated. That is because if the negotiators utility function is non-monotonic, the negotiator may find several offers that come with the same utility at the same time; and if the negotiators utility function is discrete, the negotiator may not find an offer to satisfy its expected utility exactly. In order to solve such a problem, we propose a novel negotiation model in this paper. Firstly, a 3D model is introduced to illustrate the relationships between an agents utility function, negotiation decision function and offer generation function. Then two negotiation mechanisms are proposed to handle two types of nonlinear utility functions respectively, i.e. a multiple offer mechanism is introduced to handle non-monotonic utility functions, and an approximating offer mechanism is introduced to handle discrete utility functions. Lastly, a combined negotiation mechanism is proposed to handle nonlinear utility functions in general situations by considering both the non-monotonic and discrete. The experimental results demonstrate the effectiveness and efficiency of the proposed negotiation model.

An Adaptive Bilateral Negotiation Model Based on Bayesian Learning

Endowing the negotiation agent with a learning ability such that a more beneficial agreement might be obtained is increasingly gaining attention in agent negotiation research community. In this paper, we propose a novel bilateral negotiation model based on Bayesian learning to enable self-interested agents to adapt negotiation strategies dynamically during the negotiation process. Specifically, we assume that two agents negotiate over a single issue based on time-dependent tactic. The learning agent has a belief about the probability distribution of its opponent’s negotiation parameters (i.e., the deadline and reservation offer). By observing opponent’s historical offers and comparing them with the fitted offers derived from a regression analysis, the agent can revise its belief using the Bayesian updating rule and can correspondingly adapt its concession strategy to benefit itself. By being evaluated empirically, this model shows its effectiveness for the agent to learn the possible range of its opponent’s private information and alter its concession strategy adaptively, as a result a better negotiation outcome can be achieved.

Hierarchical Learning for Emergence of Social Norms in Networked Multiagent Systems

In this paper, a hierarchical learning framework is proposed for emergence of social norms in networked multiagent systems. This framework features a bottom level of agents and several levels of supervisors. Agents in the bottom level interact with each other using reinforcement learning methods, and report their information to their supervisors after each interaction. Supervisors then aggregate the reported information and produce guide policies by exchanging information with other supervisors. The guide policies are then passed down to the subordinate agents in order to adjust their learning behaviors heuristically. Experiments are carried out to explore the efficiency of norm emergence under the proposed framework, and results verify that learning from local interactions integrating hierarchical supervision can be an effective mechanism for emergence of social norms.

Modelling Adaptive Learning Behaviours for Consensus Formation in Human Societies.

Learning is an important capability of humans and plays a vital role in human society for forming beliefs and opinions. In this paper, we investigate how learning affects the dynamics of opinion formation in social networks. A novel learning model is proposed, in which agents can dynamically adapt their learning behaviours in order to facilitate the formation of consensus among them, and thus establish a consistent social norm in the whole population more efficiently. In the model, agents adapt their opinions through trail-and-error interactions with others. By exploiting historical interaction experience, a guiding opinion, which is considered to be the most successful opinion in the neighbourhood, can be generated based on the principle of evolutionary game theory. Then, depending on the consistency between its own opinion and the guiding opinion, a focal agent can realize whether its opinion complies with the social norm (i.e., the majority opinion that has been adopted) in the population, and adapt its behaviours accordingly. The highlight of the model lies in that it captures the essential features of people’s adaptive learning behaviours during the evolution and formation of opinions. Experimental results show that the proposed model can facilitate the formation of consensus among agents, and some critical factors such as size of opinion space and network topology can have significant influences on opinion dynamics.

Market-driven agents with uncertain and dynamic outside options

One of the most crucial criterion in automated negotiation is how to reach a consensus agreement for all negotiators under any negotiation environment. Currently, most negotiation strategies can work under the static environment only. This paper presents a model for designing negotiation agents that makes adjustable rates of concession by reacting to changing market situations with uncertain and dynamic outside options. This work is based on the model of market-driven agents (MDAs). To determine the amount of the concession for each trading cycle, these market-driven agents are guided by four mathematical functions of trading opportunity, trading competition, trading time and strategy and trading eagerness. The contribution of this paper is designing and developing an extended MDA model with the flexibility to respond to uncertain and dynamic outside options, so as to increase problem solving ability for agent negotiation in broad application domains.