Adele E. Howe

Experiences with selecting search engines using metasearch

Search engines are among the most useful and high-profile resources on the Internet. The problem of finding information on the Internet has been replaced with the problem of knowing where search engines are, what they are designed to retrieve, and how to use them. This article describes and evaluates SavvySearch, a metasearch engine designed to intelligently select and interface with multiple remote search engines. The primary metasearch issue examined is the importance of carefully selecting and ranking remote search engines for user queries. We studied the efficacy of SavvySearchs incrementally acquired metaindex approach to selecting search engines by analyzing the effect of time and experience on performance. We also compared the metaindex approach to the simpler categorical approach and showed how much experience is required to surpass the simple scheme.

Trial by fire: understanding the design requirements for agents in complex environments

Phoenix is a real-time, adaptive planner that manages forest fires in a simulated environment. Alternatively, Phoenix is a search for functional relationships between the designs of agents, their behaviors, and the environments in which they work. In fact, both characterizations are appropriate and together exemplify a research methodology that emphasizes complex, dynamic environments and complete, autonomous agents. Within the Phoenix system, we empirically explore the constraints the environment places on the design of intelligent agents. This article describes the underlying methodology and illustrates the architecture and behavior of Phoenix agents.

SAVVYSEARCH: A Metasearch Engine That Learns Which Search Engines to Query

Search engines are among the most successful applications on the web today. So many search engines have been created that it is difficult for users to know where they are, how to use them, and what topics they best address. Metasearch engines reduce the user burden by dispatching queries to multiple search engines in parallel. The SAVVYSEARCH metasearch engine is designed to efficiently query other search engines by carefully selecting those search engines likely to return useful results and responding to fluctuating load demands on the web. SAVVYSEARCH learns to identify which search engines are most appropriate for particular queries, reasons about resource demands, and represents an iterative parallel search strategy as a simple plan.

Scheduling Space–Ground Communications for the Air Force Satellite Control Network

We present the first coupled formal and empirical analysis of the Satellite Range Scheduling application. We structure our study as a progression; we start by studying a simplified version of the problem in which only one resource is present. We show that the simplified version of the problem is equivalent to a well-known machine scheduling problem and use this result to prove that Satellite Range Scheduling is NP-complete. We also show that for the one-resource version of the problem, algorithms from the machine scheduling domain outperform a genetic algorithm previously identified as one of the best algorithms for Satellite Range Scheduling. Next, we investigate if these performance results generalize for the problem with multiple resources. We exploit two sources of data: actual request data from the U.S. Air Force Satellite Control Network (AFSCN) circa 1992 and data created by our problem generator, which is designed to produce problems similar to the ones currently solved by AFSCN. Three main results emerge from our empirical study of algorithm performance for multiple-resource problems. First, the performance results obtained for the single-resource version of the problem do not generalize: the algorithms from the machine scheduling domain perform poorly for the multiple-resource problems. Second, a simple heuristic is shown to perform well on the old problems from 1992; however it fails to scale to larger, more complex generated problems. Finally, a genetic algorithm is found to yield the best overall performance on the larger, more difficult problems produced by our generator.

Contrasting Structured and Random Permutation Flow-Shop Scheduling Problems: Search-Space Topology and Algorithm Performance

The use of random test problems to evaluate algorithm performance raises an important, and generally unanswered, question: Are the results generalizable to more realistic problems? Researchers generally assume that algorithms with superior performance on difficult, random test problems will also perform well on more realistic, structured problems. Our research explores this assumption for the permutation flow-shop scheduling problem. We introduce a method for generating structured flow-shop problems, which are modeled after features found in some real-world manufacturing environments. We perform experiments that indicate significant differences exist between the search-space topologies of random and structured flow-shop problems, and demonstrate that these differencescan affect the performance of certain algorithms. Yet despite these differences, and in contrast to difficult random problems, the majority of structured flow-shop problems were easily solved to optimality by most algorithms. For the problems not optimally solved, differences in performance were minor. We conclude that more realistic, structured permutation flow-shop problems are actually relatively easy to solve. Our results also raise doubts as to whether superior performance on difficult random scheduling problems translates into superior performance on more realistic kinds of scheduling problems.

Problem difficulty for tabu search in job-shop scheduling

Tabu search algorithms are among the most effective approaches for solving the job-shop scheduling problem (JSP). Yet, we have little understanding of why these algorithms work so well, and under what conditions. We develop a model of problem difficulty for tabu search in the JSP, borrowing from similar models developed for SAT and other NP-complete problems. We show that the mean distance between random local optima and the nearest optimal solution is highly correlated with the cost of locating optimal solutions to typical, random JSPs. Additionally, this model accounts for the cost of locating sub-optimal solutions, and provides an explanation for differences in the relative difficulty of square versus rectangular JSPs. We also identify two important limitations of our model. First, model accuracy is inversely correlated with problem difficulty, and is exceptionally poor for rare, very high-cost problem instances. Second, the model is significantly less accurate for structured, non-random JSPs. Our results are also likely to be useful in future research on difficulty models of local search in SAT, as local search cost in both SAT and the JSP is largely dictated by the same search space features. Similarly, our research represents the first attempt to quantitatively model the cost of tabu search for any NP-complete problem, and may possibly be leveraged in an effort to understand tabu search in problems other than job-shop scheduling.

Test Case Generation as an AI Planning Problem

While Artificial Intelligence techniques have been applied to a variety of software engineering applications, the area of automated software testing remains largely unexplored. Yet, test cases for certain types of systems (e.g., those with command language interfaces and transaction based systems) are similar to plans. We have exploited this similarity by constructing an automated test case generator with an AI planning system at its core. We compared the functionality and output of two systems, one based on Software Engineering techniques and the other on planning, for a real application: the StorageTek robot tape library command language. From this, we showed that AI planning is a viable technique for test case generation and that the two approaches are complementary in their capabilities.

A critical assessment of benchmark comparison in planning

Recent trends in planning research have led to empirical comparison becoming commonplace. The field has started to settle into a methodology for such comparisons, which for obvious practical reasons requires running a subset of planners on a subset of problems. In this paper, we characterize the methodology and examine eight implicit assumptions about the problems, planners and metrics used in many of these comparisons. The problem assumptions are: PR1) the performance of a general purpose planner should not be penalized/biased if executed on a sampling of problems and domains, PR2) minor syntactic differences in representation do not affect performance, and PR3) problems should be solvable by STRIPS capable planners unless they require ADL. The planner assumptions are: PL1) the latest version of a planner is the best one to use, PL2) default parameter settings approximate good performance, and PL3) time cut-offs do not unduly bias outcome. The metrics assumptions are: M1) performance degrades similarly for each planner when run on degraded runtime environments (e.g., machine platform) and M2) the number of plan steps distinguishes performance. We find that most of these assumptions are not supported empirically; in particular, that planners are affected differently by these assumptions. We conclude with a call to the community to devote research resources to improving the state of the practice and especially to enhancing the available benchmark problems.

The Traveling Salesrep Problem, Edge Assembly Crossover, and 2-opt

Optimal results for the Traveling Salesrep Problem have been reported on problems with up to 3038 cities using a GA with Edge Assembly Crossover (EAX). This paper first attempts to independently replicate these results on Padbergs 532 city problem. We then evaluate the performance contribution of the various algorithm components. The incorporation of 2-opt into the EAX GA is also explored. Finally, comparative results are presented for a population-based form of 2-opt that uses partial restarts.

DOMINIC: A DOMAIN-INDEPENDENT PROGRAM FOR MECHANICAL ENGINEERING DESIGN.

Abstract We describe an Artificial Intelligence (AI) program for mechanical engineering design. The program, called Dominic, characterizes design as best-first search through a space of possible designs. Dominic is a general architecture for a class of mechanical engineering design problems; within its redesign framework, in which a design is iteratively modified and improved, one can design a variety of mechanical devices. Dominics performance on two design problems is evaluated, and a battery of experiments with Dominic is discussed.