Solomon Eyal Shimony

Finding MAPs for belief networks is NP-hard

Given a probabilistic world model, an important problem is to find the maximum a-posteriori probability (MAP) instantiation of all the random variables given the evidence. Numerous researchers using such models employ some graph representation for the distributions, such as a Bayesian belief network. This representation simplifies the complexity of specifying the distributions from exponential in n, the number of variables in the model, to linear in n, in many interesting cases. We show, however, that finding the MAP is NP-hard in the general case when these representations are used, even if the size of the representation happens to be linear in n. Furthermore, minor modifications to the proof show that the problem remains NP-hard for various restrictions of the topology of the graphs. The same technique can be applied to the results of a related paper (by Cooper), to further restrict belief network topology in the proof that probabilistic inference is NP-hard.

On graphical modeling of preference and importance

In recent years, CP-nets have emerged as a useful tool for supporting preference elicitation, reasoning, and representation. CP-nets capture and support reasoning with qualitative conditional preference statements, statements that are relatively natural for users to express. In this paper, we extend the CP-nets formalism to handle another class of very natural qualitative statements one often uses in expressing preferences in daily life - statements of relative importance of attributes. The resulting formalism, TCP-nets, maintains the spirit of CP-nets, in that it remains focused on using only simple and natural preference statements, uses the ceteris paribus semantics, and utilizes a graphical representation of this information to reason about its consistency and to perform, possibly constrained, optimization using it. The extra expressiveness it provides allows us to better model tradeoffs users would like to make, more faithfully representing their preferences.

Computing frequent graph patterns from semistructured data

Whereas data mining in structured data focuses on frequent data values, in semistructured and graph data the emphasis is on frequent labels and common topologies. Here, the structure of the data is just as important as its content. We study the problem of discovering typical patterns of graph data. The discovered patterns can be useful for many applications, including: compact representation of source information and a road-map for browsing and querying information sources. Difficulties arise in the discovery task from the complexity of some of the required sub-tasks, such as sub-graph isomorphism. This paper proposes a new algorithm for mining graph data, based on a novel definition of support. Empirical evidence shows practical, as well as theoretical, advantages of our approach.

Cost-based abduction and MAP explanation

Abstract Cost-based abduction attempts to find the best explanation for a set of facts by finding a minimal cost proof for the facts. The costs are computed by summing the costs of the assumptions necessary for the proof plus the cost of the rules. We examine existing methods for constructing explanations (proofs), as a minimization problem on a DAG (directed acyclic graph). We then define a probabilistic semantics for the costs, and prove the equivalence of the cost minimization problem to the Bayesian network MAP (maximum a posteriori probability) solution of the system. A simple best-first algorithm for finding least-cost proofs is presented, and possible improvements are suggested. The semantics of cost-based abduction for complete models are then generalized to handle negation. This, in turn, allows us to apply the best-first search algorithm as a novel way of computing MAP assignments to belief networks that can enumerate assignments in order of decreasing probability. An important point is that improvement results for the best-first search algorithm carry over to the computation of MAPs.

Discovering Frequent Graph Patterns Using Disjoint Paths

Whereas data mining in structured data focuses on frequent data values, in semistructured and graph data mining, the issue is frequent labels and common specific topologies. The structure of the data is just as important as its content. We study the problem of discovering typical patterns of graph data, a task made difficult because of the complexity of required subtasks, especially subgraph isomorphism. In this paper, we propose a new apriori-based algorithm for mining graph data, where the basic building blocks are relatively large, disjoint paths. The algorithm is proven to be sound and complete. Empirical evidence shows practical advantages of our approach for certain categories of graphs

The role of relevance in explanation I: Irrelevance as statistical independence

Abstract We evaluate current explanation schemes. These are either insufficiently general, or suffer from other serious drawbacks. A domain-independent explanation theory, based on ignoring irrelevant variables in a probabilistic setting, is proposed. Independence-based maximum aposteriori probability (IB-MAP) explanations, an instance of irrelevance-based explanation, has several interesting properties, which provide for simple algorithms for computing such explanations. A best-first algorithm that generates IB-MAP explanations is presented, and evaluated empirically. The algorithm shows reasonable performance for up to medium-size problems on a set of randomly generated belief networks. An alternate algorithm, based on linear systems of inequalities, is discussed.

Support measures for graph data

The concept of support is central to data mining. While the definition of support in transaction databases is intuitive and simple, that is not the case in graph datasets and databases. Most mining algorithms require the support of a pattern to be no greater than that of its subpatterns, a property called anti-monotonicity, or admissibility. This paper examines the requirements for admissibility of a support measure. Support measures for mining graphs are usually based on the notion of an instance graph---a graph representing all the instances of the pattern in a database and their intersection properties. Necessary and sufficient conditions for support measure admissibility, based on operations on instance graphs, are developed and proved. The sufficient conditions are used to prove admissibility of one support measure—the size of the independent set in the instance graph. Conversely, the necessary conditions are used to quickly show that some other support measures, such as weighted count of instances, are not admissible.

Introducing Preferences over NFPs into Service Selection in SOA

When implementing a business or software activity in SOA, a match is sought between the required functionality and that provided by a web service. In selecting services to perform a certain business functionality, often only hard constraints are considered. However, client requirements over QoS or other NFP types are often soft and allow tradeoffs. We use a graphical language for specifying hard constraints, preferences and tradeoffs over NFPs as well as service level objectives (SLO). In particular, we use the TCP and UCP network formalisms to allow for a simple yet very flexible specification of hard constraints, preferences, and tradeoffs over these properties. Algorithms for selecting web services according to the hard constraints, as well as for optimizing the selected web service configuration, according to the specification, were developed.

Implicitly preserving semantics during incremental knowledge base acquisition under uncertainty

Abstract New knowledge is incrementally introduced to an existing knowledge base in a typical knowledge-engineering cycle. Unfortunately, at most given stages, the knowledge base is incomplete but must still satisfy sufficient consistency conditions in order to provide sound semantics. Maintaining semantics for uncertainty is of primary concern. We examine Bayesian knowledge bases (BKBs), which are a generalization of Bayesian networks. BKBs provide a highly flexible and intuitive representation following a basic “if-then” structure in conjunction with probability theory. We present new theoretical and algorithmic results concerning BKBs and how they can naturally and implicitly preserve semantics as new knowledge is added. In particular, equivalence of rule weights and conditional probabilities is achieved through stability of inferencing in BKBs. Furthermore, efficient algorithms are developed to guarantee stability of BKBs during construction. Finally, we examine and prove formal conditions that hold during the incremental construction of BKBs.

Belief updating by enumerating high-probabilityindependence-based assignments

Independence-based (IB) assignments to Bayesian belief networks were originally proposed as abductive explanations. IB assignments assign fewer variables in abductive explanations than do schemes assigning values to all evidentially supported variables. We use IB assignments to approximate marginal probabilities in Bayesian belief networks. Recent work in belief updating for Bayes networks attempts to approximate posterior probabilities by finding a small number of the highest probability complete (or perhaps evidentially supported) assignments. Under certain assumptions, the probability mass in the union of these assignments is sufficient to obtain a good approximation. Such methods are especially useful for highly-connected networks, where the maximum clique size or the cutset size make the standard algorithms intractable. Since IB assignments contain fewer assigned variables, the probability mass in each assignment is greater than in the respective complete assignment. Thus, fewer IB assignments are sufficient, and a good approximation can be obtained more efficiently. IB assignments can be used for efficiently approximating posterior node probabilities even in cases which do not obey the rather strict skewness assumptions used in previous research. Two algorithms for finding the high probability IB assignments are suggested: one by doing a best-first heuristic search, and another by special-purpose integer linear programming. Experimental results show that this approach is feasible for highly connected belief networks.