Leonard D. Shapiro

Implementation techniques for main memory database systems

With the availability of very large, relatively inexpensive main memories, it is becoming possible keep large databases resident in main memory In this paper we consider the changes necessary to permit a relational database system to take advantage of large amounts of main memory We evaluate AVL vs B+-tree access methods for main memory databases, hash-based query processing strategies vs sort-merge, and study recovery issues when most or all of the database fits in main memory As expected, B+-trees are the preferred storage mechanism unless more than 80--90% of the database fits in main memory A somewhat surprising result is that hash based query processing strategies are advantageous for large memory situations

Join processing in database systems with large main memories

We study algorithms for computing the equijoin of two relations in a system with a standard architecture hut with large amounts of main memory. Our algorithms are especially efficient when the main memory available is a significant fraction of the size of one of the relations to he joined; but they can be applied whenever there is memory equal to approximately the square root of the size of one relation. We present a new algorithm which is a hybrid of two hash-based algorithms and which dominates the other algorithms we present, including sort-merge. Even in a virtual memory environment, the hybrid algorithm dominates all the others we study. Finally, we describe how three popular tools to increase the efficiency of joins, namely filters, Babb arrays, and semijoins, can he grafted onto any of our algorithms.

Data compression and database performance

Data compression is widely used in data management to save storage space and network bandwidth. The authors outline the performance improvements that can be achieved by exploiting data compression in query processing. The novel idea is to leave data in compressed state as long as possible, and to only uncompress data when absolutely necessary. They show that many query processing algorithms can manipulate compressed data just as well as decompressed data, and that processing compressed data can speed query processing by a factor much larger than the compression factor.<<ETX>>

Sort vs. hash revisited

Efficient algorithms for processing large volumes of data are very important both for relational and new object-oriented database systems. Many query-processing operations can be implemented using sort- or hash-based algorithms, e.g. intersections, joins, and duplicate elimination. In the early relational database systems, only sort-based algorithms were employed. In the last decade, hash-based algorithms have gained acceptance and popularity, and are often considered generally superior to sort-based algorithms such as merge-join. In this article, we compare the concepts behind sort- and hash-based query-processing algorithms and conclude that (1) many dualities exist between the two types of algorithms, (2) their costs differ mostly by percentages rather than by factors, (3) several special cases exist that favor one or the other choice, and (4) there is a strong reason why both hash- and sort-based algorithms should be available in a query-processing system. Our conclusions are supported by experiments performed using the Volcano query execution engine. >

Exploiting upper and lower bounds in top-down query optimization

System Rs bottom-up query optimizer architecture forms the basis of most current commercial database managers. The paper compares the performance of top-down and bottom-up optimizers, using the measure of the number of plans generated during optimization. Top down optimizers are superior according to this measure because they can use upper and lower bounds to avoid generating groups of plans. Early during the optimization of a query, a top-down optimizer can derive upper bounds for the costs of the plans it generates. These bounds are not available to typical bottom-up optimizers since such optimizers generate and cost all subplans before considering larger containing plans. These upper bounds can be combined with lower bounds, based solely on logical properties of groups of logically equivalent subqueries, to eliminate entire groups of plans from consideration. We have implemented such a search strategy, in a top-down optimizer called Columbia. Our performance results show that the use of these bounds is quite effective, while preserving the optimality of the resulting plans. In many circumstances this new search strategy is even more effective than heuristics such as considering only left deep plans.

Optimization of extended database query languages

In this paper we examme the problem of query optlrmzatlon for extended data mampulation languages We propose a set of tactics that can be used m optlmzmg sequences of data base operations and descnbe the corresponding transformation procedures These transformations result in new equivalent sequences w&h better space and time performance The proposed techniques are especially useful in artificial mtelhgence and engmeermg apphcatlons where sequences of commands are executed over high volume databases 7 Department of Electrical Engmeermg and Computer Science, Umverslty of Cahforma, Berkeley, CA 94720

Algorithms and applications for universal quantification in relational databases

Department of Computer Science, North Dakota State Umverslty, Fargo, ND 58105 This research was sponsored by the US Ax Force O&e of Scientific Research Grant 830254 PermIssIon to copy wlthout fee all or part of this material 1s granted provided that the copies are not made or dlstrlbuted for dmxt commercial advantage, the ACM copyrlght notice and the title of the pubhcatlon and US date appear, and notice IS given that copymg IS by permlsslon of the Association for Computmg Machmery To copy otherwlse. or to repubhsh, requires a fee and/or specific permrsslon @ 1985 ACM 0-89791-160-l/85/005/0424

Decentralized Dynamics in Duopoly with Pareto Optimal Outcomes

00 75

Universal Quantification in Relational Databases: A Classification of Data and Algorithms

Queries containing universal quantification are used in many applications, including business intelligence applications and in particular data mining. We present a comprehensive survey of the structure and performance of algorithms for universal quantification. We introduce a framework that results in a complete classification of input data for universal quantification. Then we go on to identify the most efficient algorithm for each such class. One of the input data classes has not been covered so far. For this class, we propose several new algorithms. Thus, for the first time, we are able to identify the optimal algorithm to use for any given input dataset.These two classifications of optimal algorithms and input data are important for query optimization. They allow a query optimizer to make the best selection when optimizing at intermediate steps for the quantification problem.In addition to the classification, we show the relationship between relational division and the set containment join and we illustrate the usefulness of employing universal quantifications by presenting a novel approach for frequent itemset discovery.

A Digital GeoLibrary: Integrating Keywords And Place Names

This paper describes a particular pair of strategies to be followed by duopolists. At time zero duopolists are producing at a Cournot equilibrium. By following these strategies, the duopolists converge to a Pareto optimum with respect to their profits. The structure of these dynamic strategies is based on managerial considerations. The strategies are self-reinforcing in that profits increase for both producers at all times. They are centralized in that neither duopolist knows directly about the others cost function or profits or production level.