Computer Science

In this paper, a framework for monitoring human physiological response during Human-Robot Collaborative (HRC) task is presented. The framework highlights the importance of generation of event markers related to both human and robot, and also synchronization of data collected. This framework enables continuous data collection during an HRC task when changing robot movements as a form of stimuli to invoke a human physiological response. It also presents two case studies based on this framework and a data visualization tool for representation and easy analysis of the collected data during an HRC experiment.

Providing resources to different users or applications is fundamental to cloud computing. This is a challenging problem as a cloud service provider may have insufficient resources to satisfy all user requests. Furthermore, allocating available resources optimally to different applications is also challenging. Resource sharing among different cloud service providers can improve resource availability and resource utilization as certain cloud service providers may have free resources available that can be ``rented'' by other service providers. However, different cloud service providers can have different objectives or \emph{utilities}. Therefore, there is a need for a framework that can share and allocate resources in an efficient and effective way, while taking into account the objectives of various service providers that results in a \emph{multi-objective optimization} problem. In this paper, we present a \emph{Cooperative Game Theory} (CGT) based framework for resource sharing and allocation among different service providers with varying objectives that form a coalition. We show that the resource sharing problem can be modeled as an N− player \emph{canonical} cooperative game with \emph{non-transferable utility} (NTU) and prove that the game is convex for monotonic non-decreasing utilities. We propose an O(N) algorithm that provides an allocation from the \emph{core}, hence guaranteeing \emph{Pareto optimality}. We evaluate the performance of our proposed resource sharing framework in a number of simulation settings and show that our proposed framework improves user satisfaction and utility of service providers.

Many underlying relationships among multi-agent systems (MAS) in various scenarios, especially agents working on dangerous, hazardous, and risky situations, can be represented in terms of game theory. In adversarial environments, the adversaries can be intentional or unintentional based on their needs and motivations. Agents will adopt suitable decision-making strategies to maximize their current needs and minimize their expected costs. In this paper, we propose a new network model called Game-Theoretic Utility Tree (GUT) to achieve cooperative decision-making for MAS in adversarial environments combining the core principles of game theory, utility theory, and probabilistic graphical models. Through calculating multi-level Game-Theoretic computation units, GUT can decompose high-level strategies into executable lower levels. Then, we design Explorers and Monsters Game to validate our model against a cooperative decision-making algorithm based on the state-of-the-art QMIX approach. Also, we implement different predictive models for MAS working with incomplete information to estimate adversaries' state. Our experimental results demonstrate that the GUT significantly enhances cooperation among MAS to successfully complete the assigned tasks with lower costs and higher winning probabilities against adversaries.

Belief propagation approaches, such as Max-Sum and its variants, are a kind of important methods to solve large-scale Distributed Constraint Optimization Problems (DCOPs). However, for problems with n-ary constraints, these algorithms face a huge challenge since their computational complexity scales exponentially with the number of variables a function holds. In this paper, we present a generic and easy-to-use method based on a branch-and-bound technique to solve the issue, called Function Decomposing and State Pruning (FDSP). We theoretically prove that FDSP can provide monotonically non-increasing upper bounds and speed up belief propagation based DCOP algorithms without an effect on solution quality. Also, our empirically evaluation indicates that FDSP can reduce 97\% of the search space at least and effectively accelerate Max-Sum, compared with the state-of-the-art.

This paper considers the persistent coverage of a 2-D manifold that has been embedded in 3-D space. The manifold is subject to continual impact by intruders which travel at constant velocities along arbitrarily oriented straight-line trajectories. The trajectories of intruders are estimated online with an extended Kalman filter and their predicted impact points contribute normally distributed decay terms to the coverage level. A formal hybrid control strategy is presented that allows for power-constrained 3-D free-flyer agents to persistently monitor the domain, track and intercept intruders, and periodically deploy from and return to a single charging station on the manifold. Guarantees on intruder interception with respect to agent power lifespans are formally proven. The efficacy of the algorithm is demonstrated through simulation.

In this paper, we propose a maximum mutual information (MMI) framework for multi-agent reinforcement learning (MARL) to enable multiple agents to learn coordinated behaviors by regularizing the accumulated return with the mutual information between actions. By introducing a latent variable to induce nonzero mutual information between actions and applying a variational bound, we derive a tractable lower bound on the considered MMI-regularized objective function. Applying policy iteration to maximize the derived lower bound, we propose a practical algorithm named variational maximum mutual information multi-agent actor-critic (VM3-AC), which follows centralized learning with decentralized execution (CTDE). We evaluated VM3-AC for several games requiring coordination, and numerical results show that VM3-AC outperforms MADDPG and other MARL algorithms in multi-agent tasks requiring coordination.

This paper introduces a microscopic approach to model epidemics, which can explicitly consider the consequences of individual's decisions on the spread of the disease. We first formulate a microscopic multi-agent epidemic model where every agent can choose its activity level that affects the spread of the disease. Then by minimizing agents' cost functions, we solve for the optimal decisions for individual agents in the framework of game theory and multi-agent reinforcement learning. Given the optimal decisions of all agents, we can make predictions about the spread of the disease. We show that there are negative externalities in the sense that infected agents do not have enough incentives to protect others, which then necessitates external interventions to regulate agents' behaviors. In the discussion section, future directions are pointed out to make the model more realistic.

This paper presents a Multi-Agent System (MAS) approach for designing an air pollution simulator. The aim is to simulate the concentration of air pollutants emitted from sources (e.g. factories) and to investigate the emergence of cooperation between the emission source managers and the impact this has on air quality. The emission sources are controlled by agents. The agents try to achieve their goals (i.e. increase production, which has the side effect of raising air pollution) and also cooperate with others agents by altering their emission rate according to the air quality. The agents play an adapted version of the evolutionary N-Person Prisoners' Dilemma game in a non-deterministic environment; they have two decisions: decrease or increase the emission. The rewards/penalties are influenced by the pollutant concentration which is, in turn, determined using climatic parameters. In order to give predictions about the Plume Dispersion) model and an ANN (Artificial Neural Network) prediction model. The prediction is calculated using the dispersal information and real data about climatic parameters (wind speed, humidity, temperature and rainfall). Every agent cooperates with its neighbours that emit the same pollutant, and it learns how to adapt its strategy to gain more reward. When the pollution level exceeds the maximum allowed level, agents are penalised according to their participation. The system has been tested using real data from the region of Annaba (North-East Algeria). It helped to investigate how the regulations enhance the cooperation and may help controlling the air quality. The designed system helps the environmental agencies to assess their air pollution controlling policies.

Auctions are protocols to allocate goods to buyers who have preferences over them, and collect payments in return. Economists have invested significant effort in designing auction rules that result in allocations of the goods that are desirable for the group as a whole. However, for settings where participants' valuations of the items on sale are their private information, the rules of the auction must deter buyers from misreporting their preferences, so as to maximize their own utility, since misreported preferences hinder the ability for the auctioneer to allocate goods to those who want them most. Manual auction design has yielded excellent mechanisms for specific settings, but requires significant effort when tackling new domains. We propose a deep learning based approach to automatically design auctions in a wide variety of domains, shifting the design work from human to machine. We assume that participants' valuations for the items for sale are independently sampled from an unknown but fixed distribution. Our system receives a data-set consisting of such valuation samples, and outputs an auction rule encoding the desired incentive structure. We focus on producing truthful and efficient auctions that minimize the economic burden on participants. We evaluate the auctions designed by our framework on well-studied domains, such as multi-unit and combinatorial auctions, showing that they outperform known auction designs in terms of the economic burden placed on participants.

Norm emergence is typically studied in the context of multiagent systems (MAS) where norms are implicit, and participating agents use simplistic decision-making mechanisms. These implicit norms are usually unconsciously shared and adopted through agent interaction. A norm is deemed to have emerged when a threshold or predetermined percentage of agents follow the "norm". Conversely, in normative MAS, norms are typically explicit and agents deliberately share norms through communication or are informed about norms by an authority, following which an agent decides whether to adopt the norm or not. The decision to adopt a norm by the agent can happen immediately after recognition or when an applicable situation arises. In this paper, we make the case that, similarly, a norm has emerged in a normative MAS when a percentage of agents adopt the norm. Furthermore, we posit that agents themselves can and should be involved in norm synthesis, and hence influence the norms governing the MAS, in line with Ostrom's eight principles. Consequently, we put forward a framework for the emergence of norms within a normative MAS, that allows participating agents to propose/request changes to the normative system, while special-purpose synthesizer agents formulate new norms or revisions in response to these requests. Synthesizers must collectively agree that the new norm or norm revision should proceed, and then finally be approved by an "Oracle". The normative system is then modified to incorporate the norm.