A Norm Emergence Framework for Normative MAS -- Position Paper
AA Norm Emergence Framework for NormativeMAS – Position Paper
Andreasa Morris-Martin , Marina De Vos , andJulian Padget − − − University of Bath, Bath, UK { a.l.morris.martin,m.d.vos,j.a.padget } @bath.ac.uk Abstract.
Norm emergence is typically studied in the context of multia-gent systems (MAS) where norms are implicit, and participating agentsuse simplistic decision-making mechanisms. These implicit norms areusually unconsciously shared and adopted through agent interaction. Anorm is deemed to have emerged when a threshold or predeterminedpercentage of agents follow the “norm”. Conversely, in normative MAS,norms are typically explicit and agents deliberately share norms throughcommunication or are informed about norms by an authority, followingwhich an agent decides whether to adopt the norm or not. The decisionto adopt a norm by the agent can happen immediately after recogni-tion or when an applicable situation arises. In this paper, we make thecase that, similarly, a norm has emerged in a normative MAS when apercentage of agents adopt the norm. Furthermore, we posit that agentsthemselves can and should be involved in norm synthesis, and henceinfluence the norms governing the MAS, in line with Ostrom’s eightprinciples. Consequently, we put forward a framework for the emergenceof norms within a normative MAS, that allows participating agents topropose/request changes to the normative system, while special-purposesynthesizer agents formulate new norms or revisions in response to theserequests. Synthesizers must collectively agree that the new norm or normrevision should proceed, and then finally be approved by an “Oracle”.The normative system is then modified to incorporate the norm.
Keywords:
Norm synthesis · Synthesiser agents · Normative MAS · Normative System.
Multiagent systems (MAS) enable participating agents to interact with eachother and their environment to accomplish individual goals and collective goals.MAS utilise norms to encourage coordination and cooperation, and to avoid theoccurrence of undesirable states. Hollander and Wu [10] define a normative MASas a system which combines concepts of norms with explicit representations ofnormative information in order to provide a solution to problems relating toopenness in MAS. In contrast, in some MAS norms may not be considered at a r X i v : . [ c s . M A ] A p r A. Morris-Martin et al. all or the concept of norms is present but only as implicit representations ofnormative information.The normative system, the set of norms in the MAS, is normally created asa result of the process of norm synthesis. Norm synthesis is the creation andupdating of norms to avoid conflict situations – unwanted states in the MAS.Norm synthesis can be offline, which occurs mostly during design time [12] or asa separate process outside the system governed by the norms [19], where normsare determined by the designer or other stakeholders. A normative system re-sulting from offline synthesis is typically fixed for the lifetime of the system. But,over time, with changing environments, the norms can become partly or whollyirrelevant. Consequently, it becomes necessary to update the normative systemleading to the introduction of online norm synthesis. This occurs while the systemis live, and (new/revised) norms are typically determined by a centralised mech-anism with global knowledge and without any input from the participants [14,1].Conversely, the introduction of norms into a MAS, without an explicit norma-tive system, is orchestrated by participating agents in simulation models of normemergence [31,21,29], where the norms are defined as the preferred action froma set of actions, and the norm is usually learnt through agent interactions.In this paper, we introduce a framework that allows participating agents toaffect the online norm synthesis process. During runtime, participating agentscan identify the norms or situations that require normative regulation, as longas there are the necessary affordances [22] for participating agents to propose orrequest changes to the normative system. We propose synthesizer agents which,upon request from participating agents, synthesise norms for the MAS to meetthe identified need. We believe that after we encode the synthesised norm intothe normative system, and it is adopted by a sufficient number of agents, wemay ultimately observe the emergence of synthesised norm(s).The rest of the paper is structured as follows. Section 2 which situates theproposed framework by discussing the gap identified in the literature. Section 3gives a high level description of the framework, after which we discuss the stagesof the framework in more detail: Section 4 highlights the main contributionwhich defines the norm creation stage, Section 5 discusses norm propagation,then norm adoption in Section 6, finishing in Section 7, with norm emergence.The paper concludes in Section 8, where we briefly discuss how the frameworkcan facilitate the emergence of norms in open normative MAS and allow forthe normative component to be changed in response to issues affecting agentsparticipating in the MAS.
Norms in the (MAS) literature are looked at from two distinct perspectives:norms as deontic concepts and norms as a preference behaviour. Each perspec-tive implements the life cycle of norms differently, but can broadly be seen tofollow similar stages. The norm life cycle defines the stages a norm goes throughin its lifetime. Hollander and Wu [10] define a norm life cycle consisting of:
Norm Emergence Framework for Normative MAS – Position Paper 3 creation, transmission, recognition, enforcement, acceptance, modification, in-ternalisation, emergence, forgetting and evolution. These processes are encap-sulated into three super processes: enforcement, internalisation and emergencewith evolution defined as an end-to-end processes. The widely accepted norm lifecycle [28], based on simulation studies has a three stage process: norm forma-tion/creation, norm propagation and norm emergence. A closer look at the normlife cycle enables us to better understand the different perspectives on norms.
The first perspective, norms as deontic concepts, views norms as permissionsor prohibitions and obligations, or commitments and is referred to as norms asprescriptions [7], or the prescriptive approach [27]. Here we refer to it as the prescriptive perspective . Its literature uses an explicit representation of norms,typically internal as agent beliefs, but can also be external and referenceable.The norm life cycle in a society with explicit norm representation exhibitsthe following stages: (i) creation: a norm is introduced into the society, sourceoften unknown, (ii) identification: agents become aware of the norm and updatetheir beliefs, (iii) adoption: agents reason about adopting the norm, (iv) prop-agation: agents deliberately, or possibly unintentionally, inform others of thenorm, (v) emergence: not usually considered, but inferrable when a predeter-mined threshold of agents have adopted the norm.The prescriptive perspective branch of the literature presents the norm lifecycle with a focus on norm adoption. It also considers the synthesis of normsas an alternative to the norm creation stage, as it defines the norms that willregulate the behaviour of the agents within the MAS. Norm synthesis is gearedtowards creating norms to avoid conflicts and can occur both online and offline.Recently in normative MAS, online norm synthesis is becoming a popular topic,as it allows for the introduction of norms into the MAS at runtime to meetchanging circumstances. Another benefit of online norm synthesis is that it cancater for observed states, or agent capabilities that had not been conceived atthe time the MAS was designed [9]. The online norm synthesis mechanisms cited[15,16,17,1,14,6] here operate by monitoring the state of the environment andproposing norms to resolve conflicts, that can or have occurred, to prevent theiroccurrence in the future. For example, [15,16,17] do so by prohibiting the actionof an agent in the timestep before the conflict. The majority of online methodscited utilise a centralised mechanism with global knowledge, and to the best ofour knowledge only AOCMAS [6] employs a decentralised mechanism.AOCMAS is a two-level distributed MAS architecture that equips an organ-isation with adaptive capabilities. At one level there are domain agents who areconcerned with their individual goals, and above them are assistant agents whoare concerned with the organisation’s goals and help to facilitate the adaption.Assistants are responsible for or oversee a cluster of domain agents. Assistantagents partially observe the state of the organisation at runtime and proposeregulations, rules or legal norms, for problems identified. Before proposing new
A. Morris-Martin et al. solutions, assistants check if an existing stored solution is applicable using case-based reasoning (CBR). If none is found, they propose a set of regulations, andeach regulation in the set is voted on by all assistants. The regulations with themajority vote become the new set of regulations, that over time are evaluatedfor effectiveness and may be updated or removed.We note that the literature on norm synthesis, which is applicable to thenorm creation stage of the norm life cycle, fails to investigate the impact ofthe remainder of the norm life cycle. Instead the focus is on determining anappropriate set of norms, after which it is assumed that either these norms willbe adopted by agents or that the norms will be regimented. It is surprisingthat a disconnect exists within a single perspective of the norm literature, butresearch in different sections of the literature investigate various processes of thenorm life cycle independently, with little attempt to combine or sequence theactivities together. This gap, we believe, can readily be filled, by using normsynthesis in lieu of norm creation, while the other stages continue as normal inthe prescriptive approach.
The second perspective of norms is as a preference behaviour. This views normsas a predetermined or computed preference behaviour to execute from among aset of behaviours in a given situation. The preference behaviour perspective isreferred to as norms as conventions [7] or the emergence approach [27], here werefer to it as the emergence perspective . Its literature typically uses an implicitnorm representation.The norm life cycle in a society with implicit norms usually has these stages:(i) creation: the initial or predetermined strategy for an agent from a set of avail-able actions, (ii) propagation: the sharing of an agent’s strategy either uninten-tionally or deliberately, (iii) emergence: a percentage of agents follow this strat-egy for every instance of the triggering situation. The norm life cycle with implicitnorms is usually seen in simulation models of norm emergence. Savarimuthu andCranefield [28] examine simulation models of norms in MAS and present an ex-panded five stage norm life cycle: creation, identification, spreading, enforcementand emergence. They also categorise various norm-based simulation works intoeach stage of the life cycle based on the mechanisms employed in each.The emergence perspective literature is focused primarily on the study ofnorm emergence. Norm emergence is normally defined as the point in a MASwhen a threshold or predetermined percentage of agents adopt a norm. Thereis generally no discussion of the activities that precede or follow this point inthe state of the MAS. In [20] we present the notion that a refinement of normemergence is needed, as it is difficult to fully understand the emergence of anorm without taking into account the preceding activities of norm creation andpropagation. Therefore norm emergence is henceforward what is normally thenorm life cycle, but including norm creation and propagation.Utilising the interactions of agents at runtime as a basis for the creation ofnorms, as in norm emergence, is another potential solution for the source of
Norm Emergence Framework for Normative MAS – Position Paper 5 norms in the prescriptive perspective. This is not an automatic fix however, asthere are disadvantages to using norm emergence alone. Morales et al. [19] sug-gest that norm emergence is inappropriate to synthesise norms for MAS wherethere are numerous inter-dependent conflict situations. Additionally, norm emer-gence does not allow for the explicit representations of norms. This can be prob-lematic, because when norms are implicit, there can be confusion among theagents about the prevailing norm, because different subsets of agents may adoptdifferent norms. Additionally agents can have different interpretations about theprevailing norm(s) based on their beliefs [10].
As we have alluded to earlier, norm emergence in its traditional sense is com-parable to online norm synthesis, since both involve the creation of norms atruntime. In norm synthesis the creation of the norms is motivated by a singleexternal entity: external in the sense that it does not involve participating agents,while in norm emergence it is achieved internally by the participating agents’behaviour. The creation of norms in norm emergence is naturally distributedand online, while norm synthesis is usually centralised and often times offline.We submit that it can be beneficial to use participating agents within theMAS in the norm synthesis process, thereby taking the benefits of the distributedand online approach of norm emergence and adopting it into the norm synthe-sis process, to produce explicitly represented norms. The final product is anapproach that facilitates the emergence of norms in a normative MAS system.Building on the concept of distributed agents having the ability to create norms,we put forward a framework that allows participating agents to request the inclu-sion/modification of norms in the normative system, based on their experienceswithin the MAS.We situate this notion of participating agents contributing to the norms ontwo of Ostrom’s [23] eight principles for governing the commons. Foremost, theneed to allow participatory decision making, especially from those who are likelyto be affected by the rules/decisions, and secondly the need to ensure the rulesin place meet the needs of the local context, that is the participant’s needs.We believe these two principles, which are intended to aid proper governance ofthe commons, are applicable to defining rules for normative MAS with differentcontexts.Haynes et al. [9] examine how emergent behaviours in a system can be ben-eficial to the system and should be encouraged and spread, while non-beneficialemergent behaviour should be discouraged. We suggest that beneficial emergentbehaviour can further be encoded explicitly as legal norms within the normativeMAS. Therefore beneficial emergent behaviours can give rise to the emergenceof obligation norms, permission norms and prohibition norms for alternativebehaviours. Likewise, non-beneficial emergent behaviours could give rise to theemergence of prohibition norms, obligation norms to avoid certain states andrevocation of permission norms.
A. Morris-Martin et al.
Text
Ideation
Norm Creation
Decision Norm SynthesisProposalSynthesisDiscussion/DeliberationAgreement Norm Adoption NormEmergence NormPropagation
Fig. 1.
A conceptualisation of norm emergence for normative MAS
Note that we do not expect participating agents to be able to properly syn-thesise norms for the MAS, because they would need access to domain knowledgeand history of the MAS, which a participating agent would not normally have.Additionally, there is a need for higher minimum requirements on the cognitiveabilities of participating agents that may make the MAS inaccessible to someagent types. Therefore, we propose agents communicate requests to synthesizeragents, who are responsible for a subset of agents within the MAS. Each syn-thesizer agent is capable of synthesising norms, based on the request from theparticipating agents, and their (partial) knowledge of the domain and the en-vironment. We contend that the request for norm change from a participatingagent aligns with triggering the initiation of norm synthesis from the bottom up.
Agents operating in a MAS, like in human societies, could potentially determinethe norms that would better regulate the society. In the emergence perspectiveliterature, it is accepted that agents learn their behaviour from interacting withother agents. This means that the strategy of one agent or a set of agents canbecome the norm in the society. The usefulness of employing the concept of theagents determining the norm, as a potential answer to the source of the norm inthe prescriptive perspective, is worth investigating, thereby allowing the normsof a system to be determined by the agents that participate in the system.The goal of this approach is to explore techniques for developing self-governingsystems, in which agents participate in the revision of the norms that affect them.This would preclude the need for direct human intervention to synthesise normsfor changing environments, or at the least, require minimal human intervention.The removal of human involvement though useful in this context unearths risksthat must be considered in developing MAS. We provide a brief explanation inSection 4.3 Decision Stage.We conceptualise a model that pulls together three research strands, namely:(i) the life cycle of norms in the prescriptive perspective (ii) the life cycle of
Norm Emergence Framework for Normative MAS – Position Paper 7 norms in the emergence perspective, and (iii) the norm synthesis process , inorder to construct a framework that facilitates the emergence of norms in anormative MAS. Thereby, we aim to identify the complementary activities –norm creation in emergence and norm identification and adoption in normativeMAS and explicitly represented norms from norm synthesis, remove duplicateand unnecessary activities, and sequence them in a way that a new model isrevealed. The resulting model enables us to define a norm emergence frameworkfor normative MAS which we sketch in Figure 2.The model details the process of norms being synthesised, which begins withthe initial recognition of the need for a new norm in the MAS by a participatingagent. We conceive of two types of agents operating within the MAS: participantagents and synthesizer agents. The topology of the agents within the MAS willnot be considered as we do not consider interactions within the MAS to beinfluenced by social distance or connections between participating agents. Wewill however aim for there to be a uniform distribution of agents to synthesizersbased on the number of synthesizer agents in use.
Participating Agents
We put forward some assumptions about the agents thatmay participate in such a MAS, specifically: (i) the agent has an explicit internalrepresentation of norms and some non-trivial cognitive or reasoning abilities,(ii) the agent is capable of perceiving a need for a norm change, either as a newor revised norm, and (iii) the agent considers norms in their action-selection andplanning processes.The agent’s perception of the need for a norm change is predicated on agentsconsidering norms when acting, and being able to observe the effect of theiractions on the environment via action feedback. The participating agents in ourmodel are inherently normative agents . Synthesizer Agents
The framework calls for a distinguished set of agents withpartial perception of the MAS, that we refer to as synthesizers. Participatingagents are assigned a synthesizer agent upon entry to the MAS. Synthesizeragents are agents designed with knowledge of the domain context: goals, actions,conflicting states, norms. They are capable of perceiving all the actions of theagents for which they are responsible and the environment state at any giventime. Synthesizer agents in this model are inspired by the assistant agents in [6],which compute regulations utilising a partial perspective of the MAS, but doso after observing a problem, whereas synthesizer agents here await requestsfrom participating agents. Additionally, assistants [6] vote on the regulationsdetermining which shall be included, and similarly synthesizer agents must voteto decide whether or not to include all the norms proposed after discussions.We note that synthesizer agents in our model can potentially be proactiveas well. They can be proactive by examining traces and identifying when con-flicting states occur, then proposing norms to avoid them in the future, withoutwaiting on instruction from the agent to do so. Such an approach echoes ele-ments of [15,17,18,19]. Synthesizer agents could also have other functions withina MAS, such as being responsible for the enforcement of sanctions if violationoccurs. We have however decided to limit the functions of the synthesizers in this
A. Morris-Martin et al. framework to participating in the process of synthesising norms for the norma-tive system only upon receipt of requests from agents. This we believe allows usto demonstrate norm synthesis initiated from the bottom-up which is differentfrom existing research on norm synthesis.
Components of the framework
The stages of the conceptual model of ourframework are: (i) norm creation, (ii) norm propagation, (iii) norm adoption,and (iv) norm emergence as shown in Figure 1 and Figure 2. We discuss each ofthese in the remaining sections of the paper.
The creation stage comprises three sub-processes (i) ideation (ii) norm synthesis,and (iii) decision, depicted in Figure 2, component 1. In the ideal scenario, at theend of creation, there is a norm that must be incorporated into the normativesystem. If not, the request would not have been approved, and thereafter thesynthesizers and the initial requesting agent must be informed.
The ideation process models the norm creation stage of agents in the emergenceperspective, Figure 2, component 1(a), by enabling the agent to request changesbased on feedback from its interaction with the environment and other agents,similar to how an agent would learn a norm in the emergence perspective. Thedifference here is that the agent itself is not capable of making changes to thenormative system, but can communicate with a synthesizer agent to initiate thatchange on their behalf.A participating agent in a society may determine there is a need for a normchange: a new norm or the revision of an existing norm governing the MAS. Normchange requests are typically predicated on the following circumstances: (i) con-flict situation/state arising from compliance with prevailing norms, (ii) reasoningthat repeatedly determines that violation of current norms is a rational choice,(iii) a new norm or norm revision that can potentially bring about a better out-come for the agent or a better state in the MAS; we refer to this as an innovationnorm, (iv) dissatisfaction with the current norms, e.g. prohibition of actions thatcan help agents achieve goals more efficiently.An agent operating within the society may likely recognise one of the abovesituations developing long before an external observer can do so. The agentthen informs their assigned synthesizer agent of their perceived need for a normchange. Each request must specify the context of the request, the reason for therequest and the actual norm proposed, if the agent is capable of synthesising thenorm. It may be useful to provide a template to the agents which specifies whatneeds to be included in the request. Individual agents may or may not be able todetermine what the new norm can be, since they have limited knowledge of thesystem. We could put more responsibility on the participating agents to be ableto perform norm synthesis themselves, but this would require us to set minimum
Norm Emergence Framework for Normative MAS – Position Paper 9 A gen t r ea s on s abou t t he ne w no r m R eque s t f o r ne w no r m o r no r m c hange S end s ano t i fic a t i on f o r ad i sc u ss i on t oo t he r s i m il a r agen t t y pe s N e w no r m p r opo s a l o r no r m r e v i s i on s en t t o b l a ck bo x N e w no r m p r opo s a l o r no r m r e v i s i on no t a cc ep t ed , agen t i s i n f o r m ed O R A C L E D e c i s i on S ys t e m N e w N o r m o r no r m r e v i s i on t o be i m p l e m en t ed A gen t de c i de s w he t he r t oadop t o r i gno r e no r m A gen t i n t e r na li s e s t he ne w no r m A gen t r e c ogn i s e s t ha t a ne w no r m o r no r m r e v i s i on i s needed A gen t s end s r eque s t f o r ne w no r m o r no r m r e v i s i on o f c u rr en t no r m a t i v e sys t e m I n f o r m s o t he r S y n t he s i s e r s abou t ne w r eque s t R e c e i v e r eque s tf o r ne w no r m o r no r m r e v i s i on o f c u rr en t no r m a t i v e sys t e m s S o li c i t D i sc u ss i on A g r ee m en t S y n t he s i s e ne w no r m S y n t he s i s e r ss end s ne w no r m o r no r m r e v i s i on t o O r a c l e f o r app r o v a l S y n t he s i s e r s end s r e j e c t i ono f ne w o rr e v i s i on no r m p r opo s a l t o A gen t N e w N o r m o r no r m r e v i s i on no t app r o v ed A gen t i s i n f o r m edo f p r opo s a l r e j e c t i on I d ea t i on S t a g e D ec i s i on S t a g e N o r m E m e r g e n ce S t a g e II ( a ) - I d ea t i on ( b ) - N o r m S y n t h es i s ( c ) - D ec i s i on - N o r m A dop t i on I d ea t i on S t a g e - N o r m P r op a g a t i on S t a g e - N o r m C r ea t i on S t a g e A gen t s a r e i n f o r m ed o f c hange s t o t heno r m a t i v e sys t e m S y n t he s i s e r s i n f o r m ed o f de c i s i on t o i m p l e m en t no r m S y n t he s i s e r s b r oad c a s t ne w no r m t o agen t s i t s upe r v i s e s S y n t he s i s e r s i n f o r m ed o f de c i s i on t o N O T i m p l e m en t no r m S y n t he s i s e r s end s r e j e c t i on o f ne w o rr e v i s i on no r m p r opo s a l t o A gen t w ho m ade i n i t i a l p r opo s a l A p r ede t e r m i nedpe r c en t age o f agen t s ob s e r v ed adop t i ng t he ne w no r m - N o r m E m e r g e n ce N e w N o r m o r no r m r e v i s i on A pp r o v ed / N o t A pp r o v ed F i g . . C o n c e p t u a l M o d e l f o r n o r m e m e r g e n c e i nn o r m a t i v e M A S capabilities, such as their having high cognitive ability and providing them accessto knowledge of activities within the MAS. The framework is intended to beaccessible for most normative agents so we prefer an approach which does notimpose significant requirements. The norm synthesis process, as depicted in Figure 2, component 1(b), enablesthe encoding of the agents’ request, which would have been influenced by theirbehaviour, into an explicit legal norm that can address the issue within therequest. Synthesizer agents, upon receipt of a proposal, begin the process of syn-thesising a new norm to address the proposal, but additionally, the synthesizeragents engage in discussion and the establishment of a consensus about the re-quest. The sub-processes of the norm synthesis stage are: proposal, synthesis,discussion/deliberation, and agreement, each of which we now discuss in moredetail.
Proposal
After receiving a proposal, synthesiser agents must parse the requestin preparation for synthesising one or more norms. Agents utilise a providedtemplate to make proposals and the synthesizer will be equipped to interpret it.Synthesizers will normally address requests chronologically and will only handleone request at a time; if multiple requests occur over a short space of time, theyare queued. It is at this point that synthesisers have to decide which requestswarrant action, by applying a filtering mechanism. Synthesizers could employan automated rejection process, which returns a message to participating agentswhen their norm requests have been deemed to not warrant action. We notehowever that a filtering service could potentially be made available to partic-ipating agents, whereby an intermediate mechanism processes the complaint,provides feedback and then the agent decides whether to make a request of thesynthesizer agent.
Synthesis
The act of synthesising or encoding the norm occurs here. This pro-cess utilises the originating agents request, information about the environmentthat it can perceive and the domain context. Alternatively, if the participatingagent is able to synthesise a norm that will meet the identified need and includeit in the request, then this process entails the synthesizer agent determining thevalidity of the norm.During synthesis, the synthesizer agent must reason whether the synthesisednorm(s) be capable of addressing the need identified by the agent. For example,for a conflict situation, the norm should ensure that once followed, that conflictsituation is no longer observed, or for an innovation norm, the norm once adoptedensures that the agents are more efficient in accomplishing their goals, and thoseof the MAS, or the observation of more acceptable states in the MAS.Existing literature employs different mechanisms for synthesising norms.SENSE [19] considers the context of the interacting agents in the time-step beforethe conflict situation and the actions taken by the agents in that time-step. Itthen synthesises a norm by prohibiting the action of any one of the participating
Norm Emergence Framework for Normative MAS – Position Paper 11 agents. This is similar to IRON [15,17]. Another technique, utilised in [13,2], isbuilt on inductive logic programming and uses the following inputs: a normativesystem (which could be an empty set), the observation traces and the normativeconditions that must hold in the final states. It utilises the proceeding to revisethe normative system producing norms (rules) that are compatible with thesupplied trace and the condition. Both [13,2] and [19] are offline mechanisms,but can potentially be replicated for online use. Finally, the proposed new normis put forward to the other synthesizer agents for discussion and agreement.
Discussion/Deliberation
The synthesising agent informs other synthesizeragents about the new norm and solicits a discussion via a discussion mechanism,which we will call a discussion board, that is visible and accessible only to syn-thesizer agents. There is potential to have the agents in the system be able toview this discussion board. We considered the possibility of providing read-only access to participating agents, but this would require them to have the abilityto parse and understand the discussions on the board, erecting further technicalbarriers to access.Synthesizers are alerted when there is a new activity on the discussion boardand they can engage in the discussion when ready. The presenting synthesizeragent may be required to defend their proposed norm. An argumentation ornegotiation scheme may be appropriate here and can be developed based onexisting argumentation frameworks such as [25,5,4,3]. Synthesiser agents will berequired to defend the validity of the proposed norm under scrutiny to othersynthesizers, not unlike the agents in [3], who have to defend their preferredaction or abort.The resolution of internal norm conflicts and modifications, based on newperspectives highlighted by the other synthesizers, can result in the revision ofthe proposed norm. It is the responsibility of the synthesizer that proposes thenorm to ensure that after any changes made during this process, the modifiednorm can still meet the needs of the initial request. Therefore, it is possiblethat the initial proposed norm may be revised, and the actual outcome is aconsequence of modifications made as part of this stage.
Agreement
To proceed to the next stage, there needs to be agreement, where,say, a predetermined percentage of synthesisers must agree that the new normshould be introduced into the normative system. Synthesizers can come to agree-ment using several methods: (i) a clear direction forward can be establishedbased on the outcome(s) of the previous discussion phase, (ii) synthesizers canattempt to reach a consensus on the norm proposal for or against it proceeding,or (iii) synthesizers can utilise a voting mechanism (agreed in advance).If a decision to allow the norm proposal to proceed cannot be reached, thepresenting synthesizer must inform the proposing agent that the norm changewas rejected. The synthesizer can provide reasons for the rejection so that theproposing agent can possibly refine and submit a new request/proposal. Once adecision is reached, the agreed norm change is passed on to the decision stagefor a final decision on its inclusion in the normative system of the MAS.
The decision about whether a norm change will be made is done by a decisionmaking mechanism, which we will refer to as the “Oracle” as depicted in Figure 2,component 1(c). The Oracle is assumed to have access to the entire state ofthe MAS. The context and domain of the MAS may require that this decisionincorporates human input [33,24]. Human input might be necessary to precludethe risk of destabilisation of the MAS due to repeated norm synthesis or thesynthesis of norms that are potentially detrimental to the purpose of the MAS.The domain of the MAS being modelled will determine whether the Oracle’sfinal decision to modify the normative system requires human input, or if anautomated decision only can be allowed. For example, in a MAS interactingdirectly with humans or whose impact can be catastrophic, it may be preferredif a human(s) authorises the changes to the normative system.A possible solution would be for the mechanism to make a recommendationthat needs to be accepted by a human oversight process before implementation.We note that though including a human at this stage can provide human ac-countability, it raises the challenge of ensuring that the human and the systemhave the same understanding of the goals of the system. Otherwise we risk asituation where both human and system may need to prove which party is cor-rect. This could necessitate further arbitration mechanisms to avoid stagnationwithin the MAS.Though norm conflicts would have been considered in discussions with othersynthesizers, there may still be norm conflicts arising from external interaction.This can occur when the MAS may be itself governed by a higher level MAS [11]and the norms of this MAS must not conflict with the norms of the governingMAS. These conflicts, if they exist must be resolved, and this can be achievedutilising techniques similar to [11]. The stability of the normative system withinthe MAS must also be considered and should be incorporated into the finaldecision on the modification of the normative system. At the end of this process,the decision to include or reject the norm proposed is communicated to thesynthesizer agents.
At the end of the norm creation stage, a norm has either been accepted for inclu-sion into the normative system or been rejected. The preceding decision must becommunicated to the synthesizer agents by the Oracle. The norm propagationactivities are depicted in Figure 2, component 2. In the case of a modification tothe normative system, synthesiser agents are tasked with spreading this informa-tion to the agents they are responsible for as it is necessary that all participatingagents in the MAS be aware of any changes to the normative system. Synthe-sizers can choose an appropriate norm propagation mechanism to communicatewith their assigned participating agents. Broadcasting is one solution, a commonknowledge source, as used in [30] is another, which advises agents when new in-formation is available, or synthesizers can use a distributed sharing mechanism,
Norm Emergence Framework for Normative MAS – Position Paper 13 where they inform some agents, who then inform other agents they are connectedto. This latter approach could possibly affect whether all agents become awareof the modifications to the normative system in a timely manner. Alternatively,if the norm change proposal was rejected, the synthesizer needs to communicatethis to the agent that made the initial proposal.
The adoption of the norm requires the agent to internalise and reason aboutadopting the norm. The norm adoption activities are depicted in Figure 2, com-ponent 3. Internalising the norm means making an internal representation ofthe norm, while the adoption of the norm is the decision whether to adopt thenorm after reasoning about it. The internalisation of the norm is initiated whenagents are made aware of modifications to the normative system. As agents areassumed to have an explicit internal representation of norms, see Section 3, thisinternalisation of the norm means agents will have to incorporate the norm aspart of their beliefs. Once internalised, agents can reason about the validity andapplicability of the norm and decide whether it is useful to adopt it. Some agentswill do this once and adopt the norm in every situation that it is applicable goingforward, as with the normative agents in [8]. Other agents will reason about itevery time a decision to act needs to be made, similar to agents in [26,4] andgraded normative agents in [8].
The norm emergence activities are depicted in Figure 2, component 4. Once anagent has adopted the norm and complies with it, the percentage of agents doingso may reach the threshold for emergence. At this point it can be said that thenorm has emerged within the MAS. If the percentage of agents adopting thenorm never reaches the threshold for emergence, it may be useful to see if theopposite prevails. That is, where a large percentage of agents violate the norm.If this is the case then it may be necessary to reevaluate the norm’s place withinthe normative system, which could ultimately lead to its removal.
In this paper we present a framework for norm emergence in normative MASpremised on utilising the experiences of participating agents to trigger changesin the normative system. We describe a distinguished population of synthesizeragents, who accept requests from participating agents and synthesise norms inresponse to these requests. We believe that this is a useful approach for openMAS as there is no need to impose any requirements on the participating agentsbut can instead provide a set of synthesizer agents that have the requisite capa-bilities to perform the role. The introduction of synthesisers, that can propose changes to the normative system, provides participating agents with recoursewhen they are not satisfied with their experience in the MAS. Instead of leavingthe MAS, they can potentially initiate changes within the MAS that will notonly improve the experience for themselves, but perhaps for other agents. Par-ticipating agents will only need to know who their synthesizer agent is and whatinformation should be provided.We posit that the use of special-purpose synthesizer agents that address theneeds of participating agents, equips the MAS to modify its normative system atruntime, thereby facilitating decentralised runtime norm synthesis in the MAS.Furthermore we posit that the inclusion of discussion and agreement phases forsynthesizer agents to agree on the inclusion of norms is an important addition tothe framework. A synthesizer will synthesise norms based on a partial context,and as a result it could synthesise norms that, though capable of resolving theproposed issue, are not useful for the collective MAS.The remaining synthesiser agents will be able to assess the proposed normand determine how it will affect their own view of the MAS. The intention isthat only norms that are beneficial to a majority or all agents of the MAS shallachieve consensus to proceed to the final stage of verification in the MAS: the decision stage . The goal is that this additional layer will aid the MAS by helpingto maintain the stability of the normative system and preventing agents fromintroducing norms that are detrimental to it. It is also the stage where norms canbe checked for compliance with governing external MASs, if they exist and/ornon-negotiable MAS norms/rules. This stage can also allow for human inputdepending on the domain of the MAS.Synthesizers then discuss and decide if the proposed norm should be intro-duced into the normative system. Finally an Oracle mechanism must approvethe inclusion of the norm into the MAS. The normative system is modified if thenorm is approved, and agents within the MAS need to be informed of changesto the normative MAS. Participating agents will then reason about adoptingthe norm and over time a threshold of agents may choose to adopt the norm.Ultimately we may observe the emergence of the norm in the MAS that wassynthesised based on a request by a participating agent.
References
1. Alechina, N., Bulling, N., Dastani, M., Logan, B.: Practical run-time norm enforce-ment with bounded lookahead. In: Weiss et al. [32], pp. 443–4512. Athakravi, D., Corapi, D., Russo, A., Vos, M.D., Padget, J.A., Satoh, K.: Han-dling change in normative specifications. In: Baldoni, M., Dennis, L.A., Mascardi,V., Vasconcelos, W.W. (eds.) Declarative Agent Languages and Technologies X -10th International Workshop, DALT 2012, Valencia, Spain, June 4, 2012, RevisedSelected Papers. Lecture Notes in Computer Science, vol. 7784, pp. 1–19. Springer(2012). DOI: 10.1007/978-3-642-37890-4 13. Atkinson, K., Bench-Capon, T.: Practical reasoning as presumptive argumentationusing action based alternating transition systems. Artificial Intelligence (10-15), 855–874 (Jul 2007). DOI: 10.1016/j.artint.2007.04.009 Norm Emergence Framework for Normative MAS – Position Paper 154. Bench-Capon, T., Modgil, S.: Norms and value based reasoning: justifying compli-ance and violation. Artificial Intelligence and Law (1), 29–64 (Mar 2017). DOI:10.1007/s10506-017-9194-95. Boella, G., van der Torre, L.: BDI and BOID argumentation. In: Procs. of IJCAIWorkshop on Computational Models of Natural Argument. vol. 3 (2003)6. Campos, J., Lopez-Sanchez, M., Salam, M., Avila, P., Rodrguez-Aguilar, J.A.:Robust Regulation Adaptation in Multi-Agent Systems. ACM Transactions onAutonomous and Adaptive Systems (3), 1–27 (Sep 2013). DOI: 10.1145/25173287. Conte, R., Castelfranchi, C.: From conventions to prescriptions. Towards an inte-grated view of norms. Artificial intelligence and Law (4), 323–340 (1999)8. Criado, N., Argente, E., Botti, V.J.: Normative Deliberation in Graded BDIAgents. In: Mates. vol. 6251, pp. 52–63. Springer (2010)9. Haynes, C., Luck, M., McBurney, P., Mahmoud, S., V´ı tek, T., Miles, S.: Engineer-ing the emergence of norms: a review. The Knowledge Engineering Review , e18(2017). DOI: 10.1017/S026988891700016910. Hollander, C.D., Wu, A.S.: The current state of normative agent-based systems. J.Artificial Societies and Social Simulation (2) (2011). DOI: 10.18564/jasss.175011. King, T.C., De Vos, M., Dignum, V., Jonker, C.M., Li, T., Padget, J., vanRiemsdijk, M.B.: Automated multi-level governance compliance checking. Au-tonomous Agents and Multi-Agent Systems (6), 1283–1343 (Nov 2017). DOI:10.1007/s10458-017-9363-y12. Lee, J., Padget, J., Logan, B., Dybalova, D., Alechina, N.: N-Jason: Run-time normcompliance in AgentSpeak(L). In: Dalpiaz, F., Dix, J., van Riemsdijk, M.B. (eds.)Engineering Multi-Agent Systems - Second International Workshop, EMAS 2014,Paris, France, May 5-6, 2014, Revised Selected Papers. Lecture Notes in ComputerScience, vol. 8758, pp. 367–387. Springer (2014). DOI: 10.1007/978-3-319-14484-9 1913. Li, T.: Normative Conflict Detection and Resolution in Cooperating Institutions.Ph.D. thesis, University of Bath (July 2014), https://researchportal.bath.ac.uk/files/123040954/Tingting thesis final.pdf14. Mashayekhi, M., Du, H., List, G.F., Singh, M.P.: Silk: A simulation study of reg-ulating open normative multiagent systems. In: Proceedings of the Twenty-FifthInternational Joint Conference on Artificial Intelligence. pp. 373–379. IJCAI’16,AAAI Press (2016)15. Morales, J., Lopez-Sanchez, M., Rodriguez-Aguilar, J.A., Wooldridge, M., Vascon-celos, W.: Automated synthesis of normative systems. In: Proceedings of the 2013international conference on Autonomous agents and multi-agent systems. pp. 483–490. International Foundation for Autonomous Agents and Multiagent Systems(2013)16. Morales, J., L´opez-S´anchez, M., Rodr´ıguez-Aguilar, J.A., Wooldridge, M.J., Vas-concelos, W.W.: Synthesising liberal normative systems. In: Weiss et al. [32], pp.433–44117. Morales, J., Lpez-snchez, M., Rodriguez-Aguilar, J.A., Vasconcelos, W.,Wooldridge, M.: Online Automated Synthesis of Compact Normative Systems.ACM Transactions on Autonomous and Adaptive Systems (1), 1–33 (Mar 2015).DOI: 10.1145/272002418. Morales, J., Wooldridge, M., Rodr´ıguez-Aguilar, J.A., L´opez-S´anchez, M.: Evolu-tionary synthesis of stable normative systems. In: Proceedings of the 16th Con-ference on Autonomous Agents and MultiAgent Systems. pp. 1646–1648. AAMAS’17, International Foundation for Autonomous Agents and Multiagent Systems,Richland, SC (2017)6 A. Morris-Martin et al.19. Morales, J., Wooldridge, M., Rodrguez-Aguilar, J.A., Lpez-Snchez, M.: Off-linesynthesis of evolutionarily stable normative systems. Autonomous Agents andMulti-Agent Systems (Jun 2018). DOI: 10.1007/s10458-018-9390-320. Morris-Martin, A., De Vos, M., Padget, J.: Norm emergence in multiagent systems:a viewpoint paper. Autonomous Agents and Multi-Agent Systems (6), 706–749(Nov 2019). DOI: 10.1007/s10458-019-09422-021. Mukherjee, P., Sen, S., Airiau, S.: Emergence of norms with biased interactions inheterogeneous agent societies. In: Web Intelligence and Intelligent Agent Technol-ogy Workshops, 2007 IEEE/WIC/ACM International Conferences on. pp. 512–515.IEEE (2007). DOI: 10.1109/WI-IATW.2007.11522. Noriega, P., Sabater-Mir, J., Verhagen, H., Padget, J., d’Inverno, M.: Identify-ing affordances for modelling second-order emergent phenomena with the WIT framework. In: Sukthankar, G., Rodriguez-Aguilar, J.A. (eds.) Autonomous Agentsand Multiagent Systems - AAMAS 2017 Workshops, Visionary Papers, S˜ao Paulo,Brazil, May 8-12, 2017, Revised Selected Papers. Lecture Notes in Computer Sci-ence, vol. 10643, pp. 208–227. Springer (2017). DOI: 10.1007/978-3-319-71679-4 1423. Ostrom, E.: Governing the Commons. The Evolution of Institutions for CollectiveAction. CUP (1990)24. Rahwan, I.: Society-in-the-loop: programming the algorithmic social contract.Ethics and Information Technology (1), 5–14 (Mar 2018). DOI: 10.1007/s10676-017-9430-825. Riveret, R., Rotolo, A., Sartor, G.: Probabilistic rule-based argumentation fornorm-governed learning agents. Artificial Intelligence and Law (4), 383–420 (Nov2012). DOI: 10.1007/s10506-012-9134-726. dos Santos Neto, B.F., da Silva, V.T., de Lucena, C.J.: Developing Goal-OrientedNormative Agents: The NBDI Architecture. In: International Conference on Agentsand Artificial Intelligence. pp. 176–191. Springer (2011). DOI: 10.1007/978-3-642-29966-7 1227. Savarimuthu, B.T.R., Arulanandam, R., Savarimuthu, S.: Emergence of a shar-ing norm in a simulated hunter-gatherer society. In: Proceedings of the 2011IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT2011, Campus Scientifique de la Doua, Lyon, France, August 22-27, 2011. pp. 34–37(2011). DOI: 10.1109/WI-IAT.2011.14128. Savarimuthu, B.T.R., Cranefield, S.: Norm creation, spreading and emergence: Asurvey of simulation models of norms in multi-agent systems. Multiagent and GridSystems (1), 21–54 (May 2011). DOI: 10.3233/MGS-2011-016729. Savarimuthu, B.T.R., Cranefield, S., Purvis, M.K., Purvis, M.A.: Norm emergencein agent societies formed by dynamically changing networks. Web Intelligence andAgent Systems: An International Journal (3), 223–232 (2009). DOI: 10.3233/WIA-2009-016430. Savarimuthu, B.T.R., Purvis, M., Purvis, M., Cranefield, S.: Social norm emer-gence in virtual agent societies. In: International Workshop on Declarative AgentLanguages and Technologies. pp. 18–28. Springer (2008). DOI: 10.1007/978-3-540-93920-7 231. Villatoro, D., Sen, S., Sabater-Mir, J.: Topology and Memory Effect on ConventionEmergence. pp. 233–240. IEEE (2009). DOI: 10.1109/WI-IAT.2009.15532. Weiss, G., Yolum, P., Bordini, R.H., Elkind, E. (eds.): Proceedings of the 2015 In-ternational Conference on Autonomous Agents and Multiagent Systems, AAMAS2015, Istanbul, Turkey, May 4-8, 2015. ACM (2015)33. Zanzotto, F.M.: Viewpoint: Human-in-the-Loop Artificial Intelligence. J. Artif. Int.Res.64