Stéphane Airiau

Towards a pareto-optimal solution in general-sum games

Multiagent learning literature has investigated iterated two-player games to develop mechanisms that allow agents to learn to converge on Nash Equilibrium strategy profiles. Such equilibrium configuration implies that there is no motivation for one player to change its strategy if the other does not. Often, in general sum games, a higher payoff can be obtained by both players if one chooses not to respond optimally to the other player. By developing mutual trust, agents can avoid iterated best responses that will lead to a lesser payoff Nash Equilibrium. In this paper we work with agents who select actions based on expected utility calculations that incorporates the observed frequencies of the actions of the opponent(s). We augment this stochastically-greedy agents with an interesting action revelation strategy that involves strategic revealing of ones action to avoid worst-case, pessimistic moves. We argue that in certain situations, such apparently risky revealing can indeed produce better payoff than a non-revealing approach. In particular, it is possible to obtain Pareto-optimal solutions that dominate Nash Equilibrium. We present results over a large number of randomly generated payoff matrices of varying sizes and compare the payoffs of strategically revealing learners to payoffs at Nash equilibrium.

Emergence of conventions through social learning

Societal norms or conventions help identify one of many appropriate behaviors during an interaction between agents. The offline study of norms is an active research area where one can reason about normative systems and include research on designing and enforcing appropriate norms at specification time. In our work, we consider the problem of the emergence of conventions in a society through distributed adaptation by agents from their online experiences at run time. The agents are connected to each other within a fixed network topology and interact over time only with their neighbours in the network. Agents recognize a social situation involving two agents that must choose one available action from multiple ones. No default behavior is specified. We study the emergence of system-wide conventions via the process of social learning where an agent learns to choose one of several available behaviors by interacting repeatedly with randomly chosen neighbors without considering the identity of the interacting agent in any particular interaction. While multiagent learning literature has primarily focused on developing learning mechanisms that produce desired behavior when two agents repeatedly interact with each other, relatively little work exists in understanding and characterizing the dynamics and emergence of conventions through social learning. We experimentally show that social learning always produces conventions for random, fully connected and ring networks and study the effect of population size, number of behavior options, different learning algorithms for behavior adoption, and influence of fixed agents on the speed of convention emergence. We also observe and explain the formation of stable, distinct subconventions and hence the lack of emergence of a global convention when agents are connected in a scale-free network.

Emergence of Norms with Biased Interactions in Heterogeneous Agent Societies

Effective norms, emerging from sustained individual interactions over time, can complement societal rules and significantly enhance performance of individual agents and agent societies. We have recently used a model that supports the emergence of social norms via learning from interaction experiences (Sen and Airiau, 2007). Each interaction is framed as a stage game. An agent learns a policy to play the game from repeated interactions with multiple agents. We are particularly interested in finding out if the entire population learns to converge to a consistent norm when multiple action combinations yield the same optimal payoff. In this extension, we explore the effects of heterogeneous populations where different agents may be using different learning algorithms. We also investigate norm emergence when an agent is more likely to interact with other agents near by it.

Effect of joining decisions on peer clusters

Super-peer networks have been proposed to address the issue of search latency and scalability in traditional peer-to-peer (P2P) networks. In a super-peer network, instead of having a fully distributed systems of peer nodes with similar or comparable capabilities, some nodes that possess considerable computing power and resources are designated as super-peers. Each super-peer acts as a server for multiple client peers under it. This hierarchical structure of a super-peer network improves the performance of a super-peer network over traditional P2P networks by handling most search queries between the few super-peer nodes, thereby reducing overall network traffic and improving search latency. In this paper, we address the problem of mutual selection by super-peers and client peers. In particular, we evaluate alternative decision functions used by super-peers to allow new client peers to join the cluster of clients under it. We experiment with peers with known resources and demands. By formally representing and reasoning with capability and query distributions, we develop peer-selection functions that either promote concentration or diversification of capabilities within a cluster. We evaluate the effectiveness of these different selection functions for different environments where peer capabilities are aligned or are independent of their queries. We offer insight and analysis on the effects on inter and intra-peer bandwidth consumption which will allow a super-peer to adopt appropriate peer-selection functions given their expectations about the environment.

Iterated Majority Voting

We study a model in which a group of agents make a sequence of collective decisions on whether to remain in the current state of the system or switch to an alternative state, as proposed by one of them. Examples for instantiations of this model include the step-wise refinement of a bill of law by means of amendments to be voted on, as well as resource allocation problems, where agents successively alter the current allocation by means of a sequence of deals. We specifically focus on cases where the majority rule is used to make each of the collective decisions, as well as variations of the majority rule where different quotas need to be met to get a proposal accepted. In addition, we allow for cases in which the same proposal may be made more than once. As this can lead to infinite sequences, we investigate the effects of introducing a deadline bounding the number of proposals that can be made. We use both analytical and experimental means to characterise situations in which we can expect to see a convergence effect, in the sense that the expected payoff of each agent will become independent from the initial state of the system, as long as the deadline is chosen large enough.

Aggregating dependency graphs into voting agendas in multi-issue elections

Many collective decision making problems have a combinatorial structure: the agents involved must decide on multiple issues and their preferences over one issue may depend on the choices adopted for some of the others. Voting is an attractive method for making collective decisions, but conducting a multi-issue election is challenging. On the one hand, requiring agents to vote by expressing their preferences over all combinations of issues is computationally infeasible; on the other, decomposing the problem into several elections on smaller sets of issues can lead to paradoxical outcomes. Any pragmatic method for running a multi-issue election will have to balance these two concerns. We identify and analyse the problem of generating an agenda for a given election, specifying which issues to vote on together in local elections and in which order to schedule those local elections.

Enhancing the Adaptation of BDI Agents Using Learning Techniques

Belief, Desire, and Intentions (BDI) agents are well suited for complex applications with (soft) real-time reasoning and control requirements. BDI agents are adaptive in the sense that they can quickly reason and react to asynchronous events and act accordingly. However, BDI agents lack learning capabilities to modify their behavior when failures occur frequently. We discuss the use of past experience to improve the agents behavior. More precisely, we use past experience to improve the context conditions of the plans contained in the plan library, initially set by a BDI programmer. First, we consider a deterministic and fully observable environment and we discuss how to modify the BDI agent to prevent re-occurrence of failures, which is not a trivial task. Then, we discuss how we can use decision trees to improve the agents behavior in a non-deterministic environment.

On the stability of an Optimal Coalition Structure

The two main questions in coalition games are 1) what coalitions should form and 2) how to distribute the value of each coalition between its members. When a game is not superadditive, other coalition structures (CSs) may be more attractive than the grand coalition. For example, if the agents care about the total payoff generated by the entire society, CSs that maximize utilitarian social welfare are of interest. The search for such optimal CSs has been a very active area of research. Stability concepts have been defined for games with coalition structure, under the assumption that the agents agree first on a CS, and then the members of each coalition decide on how to share the value of their coalition. An agent can refer to the values of coalitions with agents outside of its current coalition to argue for a larger share of the coalition payoff. To use this approach, one can find the CS s* with optimal value and use one of these stability concepts for the game with s*. However, it may not be fair for some agents to form s*, e.g., for those that form a singleton coalition and cannot benefit from collaboration with other agents. We explore the possibility of allowing side-payments across coalitions to improve the stability of an optimal CS. We adapt existing stability concepts and prove that some of them are non-empty under our proposed scheme.

Norm Emergence with Biased Agents

Effective norms can significantly enhance performance of individual agents and agent societies. We consider individual agents that repeatedly interact over instances of a given scenario. Each interaction is framed as a stage game where multiple action combinations yield the same optimal payoff. An agent learns to play the game over repeated interactions with multiple, unknown, agents. The key research question is to find out whether a consistent norm emerges when all agents are learning at the same time. In real-life, agents may have pre-formed biases or preferences which may hinder or even preclude norm emergence. We study the success and speed of norm emergence when different subsets of the population have different initial biases. In particular we characterize the relative speed of norm emergence under varying biases and the success of majority/minority groups in enforcing their biases on the rest of the population given different bias strengths.

Strategic bidding for multiple units in simultaneous and sequential auctions

A buyer may be interested in buying a bundle of items, where any one item in the bundle may not be of particular interest. The emergence of online auctions allows such users to obtain bundles by bidding on different simultaneous or sequentially run auctions. Because the number of auctions and the number of combinations to form the bundles may be large, the bundle bidding problem becomes intractable and the user is likely to make suboptimal decision given time constraints and information overload. We believe that an automated agent that can take user preferences and budgetary constraints and can strategically bid on behalf of a user can significantly enhance user profit and satisfaction. Our first step to build such an agent is to consider bundles containing many units of a single a item and auctions that sell only multiple units of one item type. We assume that users obtain goods over several days. Expectations of auctions and their outcome in the future allow the agent to bid strategically on currently open auctions. The agent decides how many items to bid for in the current auctions, and the maximum price to bid for each item. We evaluate our proposed strategy in different configurations: number of items sold, number of auctions opened, expected closing prices, etc. The agent produces greater returns in situations where future auctions can provide better profit, and where not too many agents use our proposed strategy.