Fahiem Bacchus

LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE

A new approach for learning Bayesian belief networks from raw data is presented. The approach is based on Rissanens minimal description length (MDL) principle, which is particularly well suited for this task. Our approach does not require any prior assumptions about the distribution being learned. In particular, our method can learn unrestricted multiply‐connected belief networks. Furthermore, unlike other approaches our method allows us to trade off accuracy and complexity in the learned model. This is important since if the learned model is very complex (highly connected) it can be conceptually and computationally intractable. In such a case it would be preferable to use a simpler model even if it is less accurate. The MDL principle offers a reasoned method for making this trade‐off. We also show that our method generalizes previous approaches based on Kullback cross‐entropy. Experiments have been conducted to demonstrate the feasibility of the approach.

Using temporal logics to express search control knowledge for planning

Over the years increasingly sophisticated planning algorithms have been developed. These have made for more efficient planners, but unfortunately these planners still suffer from combinatorial complexity even in simple domains. Theoretical results demonstrate that planning is in the worst case intractable. Nevertheless, planning in particular domains can often be made tractable by utilizing additional domain structure. In fact, it has long been acknowledged that domain independent planners need domain dependent information to help them plan effectively. In this work we present an approach for representing and utilizing domain specific control knowledge. In particular, we show how domain dependent search control knowledge can be represented in a temporal logic, and then utilized to effectively control a forward-chaining planner. There are a number of advantages to our approach, including a declarative semantics for the search control knowledge; a high degree of modularity (new search control knowledge can be added without affecting previous control knowledge); and an independence of this knowledge from the details of the planning algorithm. We have implemented our ideas in the TLPLAN system, and have been able to demonstrate its remarkable effectiveness in a wide range of planning domains.

Planning for temporally extended goals

In planning, goals have traditionally been viewed as specifying a set of desirable final states. Any plan that transforms the current state to one of these desirable states is viewed to be correct. Goals of this form are limited in what they can specify, and they also do not allow us to constrain the manner in which the plan achieves its objectives. We propose viewing goals as specifying desirable sequences of states, and a plan to be correct if its execution yields one of these desirable sequences. We present a logical language, a temporal logic, for specifying goals with this semantics. Our language is rich and allows the representation of a range of temporally extended goals, including classical goals, goals with temporal deadlines, quantified goals (with both universal and existential quantification), safety goals, and maintenance goals. Our formalism is simple and yet extends previous approaches in this area. We also present a planning algorithm that can generate correct plans for these goals. This algorithm has been implemented, and we provide some examples of the formalism at work. The end result is a planning system which can generate plans that satisfy a novel and useful set of conditions.

Representing and reasoning with probabilistic knowledge: a logical approach to probabilities

Probabilistic information has many uses in an intelligent system. This book explores logical formalisms for representing and reasoning with probabilistic information that will be of particular value to researchers in nonmonotonic reasoning, applications of probabilities, and knowledge representation. It demonstrates that probabilities are not limited to particular applications, like expert systems; they have an important role to play in the formal design and specification of intelligent systems in general.Fahiem Bacchus focuses on two distinct notions of probabilities: one propositional, involving degrees of belief, the other proportional, involving statistics. He constructs distinct logics with different semantics for each type of probability that are a significant advance in the formal tools available for representing and reasoning with probabilities. These logics can represent an extensive variety of qualitative assertions, eliminating requirements for exact point-valued probabilities, and they can represent first-order logical information. The logics also have proof theories which give a formal specification for a class of reasoning that subsumes and integrates most of the probabilistic reasoning schemes so far developed in AI.Using the new logical tools to connect statistical with propositional probability, Bacchus also proposes a system of direct inference in which degrees of belief can be inferred from statistical knowledge and demonstrates how this mechanism can be applied to yield a powerful and intuitively satisfying system of defeasible or default reasoning.Contents: Introduction. Propositional Probabilities. Statistical Probabilities. Combining Statistical and Propositional Probabilities Default Inferences from Statistical Knowledge.

From statistical knowledge bases to degrees of belief

An intelligent agent will often be uncertain about various properties of its environment, and when acting in that environment it will frequently need to quantify its uncertainty. For example, if the agent wishes to employ the expected-utility paradigm of decision theory to guide its actions, it will need to assign degrees of belief (subjective probabilities) to various assertions. Of course, these degrees of belief should not be arbitrary, but rather should be based on the information available to the agent. This paper describes one approach for inducing degrees of belief from very rich knowledge bases, that can include information about particular individuals, statistical correlations, physical laws, and default rules. We call our approach the random-worlds method. The method is based on the principle of indifference: it treats all of the worlds the agent considers possible as being equally likely. It is able to integrate qualitative default reasoning with quantitative probabilistic reasoning by providing a language in which both types of information can be easily expressed. Our results show that a number of desiderata that arise in direct inference (reasoning from statistical information to conclusions about individuals) and default reasoning follow directly from the semantics of random worlds. For example, random worlds captures important patterns of reasoning such as specificity, inheritance, indifference to irrelevant information, and default assumptions of independence. Furthermore, the expressive power of the language used and the intuitive semantics of random worlds allow the method to deal with problems that are beyond the scope of many other nondeductive reasoning systems.

Effective preprocessing with hyper-resolution and equality reduction

HypBinRes, a particular form of hyper-resolution, was first employed in the SAT solver 2cls+eq. In 2cls+eq, HypBinRes and equality reduction are used at every node of a DPLL search tree, pruning much of the search tree. This allowed 2cls+eq to display the best all-around performance in the 2002 SAT solver competition. In particular, it was the only solver to qualify for the second round of the competition in all three benchmark categories. In this paper we investigate the use of HypBinRes and equality reduction in a preprocessor that can be used to simplify a CNF formula prior to SAT solving. We present empirical evidence demonstrating that such a preprocessor can be extremely effective on large structured problems, making some previously unsolvable problems solvable. The preprocessor is also able to solve a number of non-trivial instances by itself. Since the preprocessor does not have to worry about undoing changes on backtrack, nor about keeping track of reasons for intelligent backtracking, we are able to develop a new algorithm for applying HypBinRes that can be orders of magnitude more efficient than the algorithm employed inside of 2cls+eq. The net result is a technique that improves our ability to solve hard problems SAT problems.

Algorithms and complexity results for #SAT and Bayesian inference

Bayesian inference is an important problem with numerous applications in probabilistic reasoning. Counting satisfying assignments is a closely related problem of fundamental theoretical importance. In this paper, we show that plain old DPLL equipped with memorization (an algorithm we call #DPLLCache) can solve both of these problems with time complexity that is at least as good as state-of-the-art exact algorithms, and that it can also achieve the best known time-space tradeoff. We then proceed to show that there are instances where #DPLLCache can achieve an exponential speedup over existing algorithms.

Dynamic Variable Ordering in CSPs

We investigate the dynamic variable ordering (DVO) technique commonly used in conjunction with tree-search algorithms for solving constraint satisfaction problems. We first provide an implementation methodology for adding DVO to an arbitrary tree-search algorithm. Our methodology is applicable to a wide range of algorithms including those that maintain complicated information about the search history, like backmarking. We then investigate the popular reordering heuristic of next instantiating the variable with the minimum remaining values (MRV). We prove some interesting theorems about the MRV heuristic which demonstrate that if one wants to use the MRV heuristic one should use it with forward checking. Finally, we investigate the empirical performance of 12 different algorithms with and without DVO. Our experiments and theoretical results demonstrate that forward checking equipped with dynamic variable ordering is a very good algorithm for solving CSPs.

Downward refinement and the efficiency of hierarchical problem solving

Abstract Analysis and experiments have shown that hierarchical problem solving is most effective when the hierarchy satisfies the downward refinement property (DRP), whereby every abstract solution can be refined to a concrete-level solution without backtracking across abstraction levels. However, the DRP is a strong requirement that is not often met in practice. In this paper we examine the case when the DRP fails, and provide an analytical model of search complexity parameterized by the probability of an abstract solution being refinable. Our model provides a more accurate picture of the effectiveness of hierarchical problem solving. We then formalize the DRP in Abstrips -style hierarchies, providing a syntactic test that can be applied to determine if a hierarchy satisfies the DRP. Finally, we describe an algorithm called Highpoint that we have developed. This algorithm builds on the Alpine algorithm of Knoblock in that it automatically generates abstraction hierarchies. However, it uses the theoretical tools we have developed to generate hierarchies superior to those generated by Alpine . This superiority is demonstrated empirically. 2

A heuristic search approach to planning with temporally extended preferences

In this paper we propose a suite of techniques for planning with temporally extended preferences (TEPs). To this end, we propose a method for compiling TEP planning problems into simpler domains containing only final-state (simple) preferences and metric functions. With this simplified problem in hand, we propose a variety of heuristic functions for planning with final-state preferences, together with an incremental best-first planning algorithm. A key feature of the planning algorithm is its ability to prune the search space. We identify conditions under which our planning algorithm will generate optimal plans. We implemented our algorithm as an extension to the TLPLAN planning system and report on extensive testing performed to evaluate the effectiveness of our heuristics and algorithm. Our planner, HPLAN-P, competed in the 5th International Planning Competition, achieving distinguished performance in the qualitative preferences track.