Janyl Jumadinova

A multi-agent system with reinforcement learning agents for biomedical text mining

Due to the expanding growth of information in the biomedical literature and biomedical databases, researchers and practitioners in the biomedical field require efficient methods of handling and extracting useful information. We present a novel framework for biomedical text mining based on a learning multi-agent system. Our distributed system comprises of several software agents, where each agent uses a reinforcement learning method to update the sentiment of a relevant text from a particular set of research articles related to specific keywords. Our system was tested on the biomedical research articles from PubMed, where the goal of each agent is to accrue utility by correctly determining the relevant information that is communicated with other agents. Our results tested on the abstracts collected from PubMed related to muscular atrophy, Alzheimers disease, and diabetes show that our system is able to appropriately learn the sentiment score related to specific keywords by parallel and distributed analysis of the documents by multiple software agents.

Prediction Market-Based Information Aggregation for Multi-sensor Information Processing

Prediction markets have been shown to be a useful tool in forecasting the outcome of future events by aggregating public opinion about the event’s outcome. We consider an analogous problem of information fusion from multiple sensors of different types with the objective of improving the confidence of inference tasks, such as object classification. We develop a multi-agent prediction market-based technique to solve this information fusion problem. To monitor the improvement in the confidence of the object classification as well as to dis-incentivize agents from misreporting information, we have introduced a market maker that rewards the agents based on the quality of the submitted reports. We have implemented the market maker’s reward calculation in the form of a scoring rule and have shown analytically that it incentivizes truthful revelation by each agent. We have experimentally verified our technique for multi-sensor information fusion for an automated landmine detection scenario. Our experimental results show that, for identical data distributions and settings, using our information aggregation technique increases the accuracy of object classification favorably as compared to two other commonly used techniques for information fusion for landmine detection.

The COMRADE System for Multirobot Autonomous Landmine Detection in Postconflict Regions

We consider the problem of autonomous landmine detection using a team of mobile robots. Previous research on robotic landmine detection mostly employs a single robot equipped with a landmine detection sensor to detect landmines. We envisage that the quality of landmine detection can be significantly improved if multiple robots are coordinated to detect landmines in a cooperative manner by incrementally fusing the landmine-related sensor information they collect and then use that information to visit locations of potential landmines. Towards this objective, we describe a multirobot system called COMRADES to address different aspects of the autonomous landmine detection problem including distributed area coverage to detect and locate landmines, information aggregation to fuse the sensor information obtained by different robots, and multirobot task allocation (MRTA) to enable different robots to determine a suitable sequence to visit locations of potential landmines while reducing the time required and battery expended. We have used commercially available all-terrain robots called Coroware Explorer that are customized with a metal detector to detect metallic objects including landmines, as well as indoor Corobot robots, both in simulation and in physical experiments, to test the different techniques in COMRADES.

A multi-agent prediction market based on partially observable stochastic game

We present a novel, game theoretic representation called POSGI (partially observable stochastic game with information) for distributed information aggregation using a multi-agent based prediction market model. We then describe a correlated equilibrium (CE)-based solution strategy for this game which enables each agent to dynamically calculate the prices at which it should trade a security in the prediction market. We have extended our results to risk averse traders and shown that a Pareto optimal correlated equilibrium strategy can be used to incentively truthful revelations from risk averse agents. Simulation results comparing our CE strategy with five other strategies commonly used in similar markets, with both risk neutral and risk averse agents, show that the CE strategy improves price predictions and provides higher utilities to the agents as compared to other existing strategies.

Strategic Capability-Learning for Improved Multi-agent Collaboration in Ad-hoc Environments

We consider the problem of distributed collaboration among multiple agents in an ad hoc setting. We have analyzed this problem within a multiagent task execution scenario, in which every task requires collaboration among multiple agents to get completed. Tasks are also ad hoc in the sense that they appear dynamically and require different sets of expertise or capabilities from agents for completion. We model collaboration within this framework as a decision-making problem in which agents have to determine what capabilities to learn and from which agents to learn them so that they can form teams that have the capabilities required to perform the current tasks satisfactorily. Our proposed technique refers to principles from human learning theory to enable an agent to strategically select appropriate capabilities to learn from other agents. We also use two openness parameters to model the dynamic nature of tasks and agents in the environment. Experimental results within the Repast agent simulator show that by using the appropriate learning strategy, the overall utility of the agents improves considerably. The performance of the agents and their utilities are also dependent on the repetitiveness of tasks and reencounter with agents within the environment. Our results also show that the agents that are able to learn more capabilities from another expert agent outperform the agents who learn only one capability at a time from many agents, and agents who use an intelligent utility maximizing strategy to choose which capabilities to learn outperform the agents who randomly make the learning decision.

Automated Pricing in a Multiagent Prediction Market Using a Partially Observable Stochastic Game

Prediction markets offer an efficient market-based mechanism to aggregate large amounts of dispersed or distributed information from different people to predict the possible outcome of future events. Recently, automated prediction markets where software trading agents perform market operations such as trading and updating beliefs on behalf of humans have been proposed. A challenging aspect in automated prediction markets is to develop suitable techniques that can be used by automated trading agents to update the price at which they should trade securities related to an event so that they can increase their profit. This problem is nontrivial, as the decision to trade and the price at which trading should occur depends on several dynamic factors, such as incoming information related to the event for which the security is being traded, the belief-update mechanism and risk attitude of the trading agent, and the trading decision and trading prices of other agents. To address this problem, we have proposed a new behavior model for trading agents based on a game-theoretic framework called partially observable stochastic game with information (POSGI). We propose a correlated equilibrium (CE)-based solution strategy for this game that allows each agent to dynamically choose an action (to buy or sell or hold) in the prediction market. We have also performed extensive simulation experiments using the data obtained from the Intrade prediction market for four different prediction markets. Our results show that our POSGI model and CE strategy produces prices that are strongly correlated with the prices of the real prediction markets. Results comparing our CE strategy with five other strategies commonly used in similar market show that our CE strategy improves price predictions and provides higher utilities to the agents compared to other existing strategies.

Distributed prediction markets modeled by weighted Bayesian graphical games

We consider a novel, yet practical setting of prediction markets called distributed prediction markets, where the aggregated price of a security of an event in one prediction market is affected dynamically by the prices of securities of similar events in other, simultaneously running prediction markets. We focus on the problem of decision making facing a market maker to determine the price of a security within such a setting. We propose a formal framework based on graphical games called a weighted Bayesian graphical game (WBGG) to model the distributed prediction market setting and to capture the local interactions between multiple market makers. We then describe a distributed message passing algorithm based on NashProp algorithm to calculate the Bayes-Nash equilibrium in a WBGG. We provide analytical results including convergence and incentivizing truthful revelation among market makers. Our experimental results show that market makers that consider the influence of other market makers in a distributed prediction market setting while using our proposed WBGG-based algorithm obtain higher utilities and set prices more accurately in comparison to market makers using a greedy strategy to set prices or those that do not consider the influence of other market makers. We also observe that extreme sizes of the neighborhood of a market maker have an adverse impact on its utilities.