Meir Kalech

Practical voting rules with partial information

Voting is an essential mechanism that allows multiple agents to reach a joint decision. The joint decision, representing a function over the preferences of all agents, is the winner among all possible (candidate) decisions. To compute the winning candidate, previous work has typically assumed that voters send their complete set of preferences for computation, and in fact this has been shown to be required in the worst case. However, in practice, it may be infeasible for all agents to send a complete set of preferences due to communication limitations and willingness to keep as much information private as possible. The goal of this paper is to empirically evaluate algorithms to reduce communication on various sets of experiments. Accordingly, we propose an iterative algorithm that allows the agents to send only part of their preferences, incrementally. Experiments with simulated and real-world data show that this algorithm results in an average of 35% savings in communications, while guaranteeing that the actual winning candidate is revealed. A second algorithm applies a greedy heuristic to save up to 90% of communications. While this heuristic algorithm cannot guarantee that a true winning candidate is found, we show that in practice, close approximations are obtained.

On the design of coordination diagnosis algorithms for teams of situated agents

Teamwork demands agreement among team-members in order to collaborate and coordinate effectively. When a disagreement between teammates occurs (due to failures), team-members should ideally diagnose its causes, to resolve the disagreement. Such diagnosis of social failures can be expensive in communication and computation, challenges which previous work has not addressed. We present a novel design space of diagnosis algorithms, distinguishing several phases in the diagnosis process, and providing alternative algorithms for each phase. We then combine these algorithms in different ways to empirically explore specific design choices in a complex domain, on thousands of failure cases. The results show that different phases of diagnosis affect communication and computation overhead. In particular, centralizing the diagnosis disambiguation process is a key factor in reducing communications, while runtime is affected mainly by the amount of reasoning about other agents. These results contrast with previous work in disagreement detection (without diagnosis), in which distributed algorithms reduce communications.

Diagnosing a Team of Agents: Scaling-Up

Agents in a team must be in agreement. Unfortunately, they may come to disagree due to sensing uncertainty, communication failures, etc. Once a disagreement occurs we should detect the disagreement and diagnose it. Unfortunately, current diagnosis techniques do not scale well with the number of agents, as they have high communication and computation complexity. We suggest three techniques to reduce this complexity: (i) reducing the amount of diagnostic reasoning by sending targeted queries; (ii) using light-weight behavior recognition to recognize which beliefs of the agents might be in conflict; and (iii) grouping the agents according to their role and behavior and then diagnosing the groups based on representative agents. We examine these techniques in large-scale teams, in two domains, and show that combining the techniques produces a diagnosis process which is highly scalable in both communication and computation.

Online data-driven anomaly detection in autonomous robots

The use of autonomous robots is appealing for tasks, which are dangerous to humans. Autonomous robots might fail to perform their tasks since they are susceptible to varied sorts of faults such as point and contextual faults. Not all faults can be known in advance, and hence, anomaly detection is required. In this paper, we present an online data-driven anomaly detection approach (ODDAD) for autonomous robots. ODDAD is suitable for the dynamic nature of autonomous robots since it declares a fault based only on data collected online. In addition, it is unsupervised, model free and domain independent. ODDAD proceeds in three steps: data filtering, attributes grouping based on dependency between attributes and outliers detection for each group. Above a calculated threshold, an anomaly is declared. We empirically evaluate ODDAD in different domains: commercial unmanned aerial vehicles (UAVs), a vacuum-cleaning robot, a high-fidelity flight simulator and an electrical power system of a spacecraft. We show the significance and impact of each component of ODDAD . By comparing ODDAD to other state-of-the-art competing anomaly detection algorithms, we show its advantages.

A novel SAT-based approach to model based diagnosis

This paper introduces a novel encoding of Model Based Diagnosis (MBD) to Boolean Satisfaction (SAT) focusing on minimal cardinality diagnosis. The encoding is based on a combination of sophisticated MBD preprocessing algorithms and the application of a SAT compiler which optimizes the encoding to provide more succinct CNF representations than obtained with previous works. Experimental evidence indicates that our approach is superior to all published algorithms for minimal cardinality MBD. In particular, we can determine, for the first time, minimal cardinality diagnoses for the entire standard ISCAS-85 and 74XXX benchmarks. Our results open the way to improve the state-of-the-art on a range of similar MBD problems.

Reaching a joint decision with minimal elicitation of voter preferences

Abstract Sometimes voters are required to reach a joint decision and find an item that best suits the group’s preferences. Voters may wish to state preferences only when necessary, particularly in cases where there are many available options, therefore it is unpractical to assume that all voter preferences are known at all times. In order to elicit voter preferences at a minimal cost, a preference elicitation process is required. We introduce a general approach for reaching a joint decision with minimal elicitation of voter preferences. The approach is probabilistic and uses voting rules to find a necessary winning item which is presented to the group as their best option. We propose computing a voter-item probability distribution and developing methods based on this distribution that can then determine which voter-item pair to query. Computing the optimal minimal set of voter-item queries is computationally intractable; therefore we propose novel heuristic algorithms, named DIG and ES, which proceed iteratively until the identification of a winning item. The probabilistic voting distribution is updated as more information is revealed. Experiments on simulated data examine the benefits of each of the algorithms under different settings. Experiments with the real-world Netflix data show that the proposed algorithms reduce the required number of ratings for identifying the winning item by more than 50%.

Who is going to win the next Association for the Advancement of Artificial Intelligence Fellowship Award? Evaluating researchers by mining bibliographic data

Accurately evaluating a researcher and the quality of his or her work is an important task when decision makers have to decide on such matters as promotions and awards. Publications and citations play a key role in this task, and many previous studies have proposed using measurements based on them for evaluating researchers. Machine learning techniques as a way of enhancing the evaluating process have been relatively unexplored. We propose using a machine learning approach for evaluating researchers. In particular, the proposed method combines the outputs of three learning techniques (logistics regression, decision trees, and artificial neural networks) to obtain a unified prediction with improved accuracy. We conducted several experiments to evaluate the models ability to: (a) classify researchers in the field of artificial intelligence as Association for the Advancement of Artificial Intelligence (AAAI) fellows and (b) predict the next AAAI fellowship winners. We show that both our classification and prediction methods are more accurate than are previous measurement methods, and reach a precision rate of 96% and a recall of 92%.

Diagnosis of coordination failures: a matrix-based approach

One of the key requirements in many multi-agent teams is that agents coordinate specific aspects of their joint task. Unfortunately, this coordination may fail due to intermittent faults in sensor readings, communication faults, etc. A key challenge in the model-based diagnosis (MBD) of coordination failures is to represent a model of the coordination among the agents in a way that allows efficient detection and diagnosis, based on observation of the agents involved. Previously developed mechanisms are useful only for small groups of agents, since they represent the coordination with binary constraints. This paper presents a MBD approach to coordination failures in which non-binary constraints are allowed. This model has two inherent advantages: (1) the model enables to address real problems, (2) the model enables to address large groups by gathering multiple coordinations in one constraint. To solve the diagnosis problem, we propose a matrix-based approach to represent the basic building blocks of the MBD formalization. Theoretical and empirical evaluations show that this representation is efficient for large-scale teams.

COORDINATION DIAGNOSTIC ALGORITHMS FOR TEAMS OF SITUATED AGENTS: SCALING UP

Agents in a team should be in agreement. Unfortunately, they may come to disagree due to sensor uncertainty, intermittent communication failures, etc. Once a disagreement occurs, the agents should detect and diagnose the disagreement. Current diagnostic techniques do not scale well with the number of agents, as they have high communication and computation complexity. We present novel techniques that enable scalability in three ways. First, we use communications early in the diagnostic process to stave off unneeded reasoning, which ultimately leads to unneeded communications. Second, we use light‐weight (and inaccurate) behavior recognition to focus the diagnostic reasoning on beliefs of agents that might be in conflict. Finally, we propose diagnosing only to a limited number of representative agents (instead of all the agents). We examine these techniques in large‐scale teams of situated agents in two domains and show that combining the techniques produces a diagnostic process that is highly scalable in both communication and computation.

A hybrid approach for improving unsupervised fault detection for robotic systems

From unsupervised to supervised learning a fault detection model (for robots).Insights to why and when it becomes more accurate.Theoretical analysis and a prediction tool.Empirical results on 3 real-world domains that back these insights. The use of robots in our daily lives is increasing. As we rely more on robots, thus it becomes more important for us that the robots will continue on with their mission successfully. Unfortunately, these sophisticated, and sometimes very expensive, machines are susceptible to different kinds of faults. It becomes important to apply a Fault Detection (FD) mechanism which is suitable for the domain of robots. Two important requirements of such a mechanism are: high accuracy and low computational-load during operation (online). Supervised learning can potentially produce very accurate FD models, and if the learning takes place offline then the online computational-load can be reduced. Yet, the domain of robots is characterized with the absence of labeled data (e.g., faulty, normal) required by supervised approaches, and consequently, unsupervised approaches are being used. In this paper we propose a hybrid approach - an unsupervised approach can label a data set, with a low degree of inaccuracy, and then the labeled data set is used offline by a supervised approach to produce an online FD model. Now, we are faced with a choice should we use the unsupervised or the hybrid fault detector? Seemingly, there is no way to validate the choice due to the absence of (a priori) labeled data. In this paper we give an insight to why, and a tool to predict when, the hybrid approach is more accurate. In particular, the main impacts of our work are (1) we theoretically analyze the conditions under which the hybrid approach is expected to be more accurate. (2) Our theoretical findings are backed with empirical analysis. We use data sets of three different robotic domains: a high fidelity flight simulator, a laboratory robot, and a commercial Unmanned Arial Vehicle (UAV). (3) We analyze how different unsupervised FD approaches are improved by the hybrid technique and (4) how well this improvement fits our prediction tool. The significance of the hybrid approach and the prediction tool is the potential benefit to expert and intelligent systems in which labeled data is absent or expensive to create.