John T. Robinson

Data cache management using frequency-based replacement

We propose a new frequency-based replacement algorithm for managing caches used for disk blocks by a file system, database management system, or disk control unit, which we refer to here as data caches. Previously, LRU replacement has usually been used for such caches. We describe a replacement algorithm based on the concept of maintaining reference counts in which locality has been “factored out”. In this algorithm replacement choices are made using a combination of reference frequency and block age. Simulation results based on traces of file system and I/O activity from actual systems show that this algorithm can offer up to 34% performance improvement over LRU replacement, where the improvement is expressed as the fraction of the performance gain achieved between LRU replacement and the theoretically optimal policy in which the reference string must be known in advance. Furthermore, the implementation complexity and efficiency of this algorithm is comparable to one using LRU replacement.

Limitations of concurrency in transaction processing

Given the pairwise probability of conflict p among transactions in a transaction processing system, together with the total number of concurrent transactions n, the effective level of concurrency E(n,p) is defined as the expected number of the n transactions that can run concurrently and actually do useful work. Using a random graph model of concurrency, we show for three general classes of concurrency control methods, examples of which are (1) standard locking, (2) strict priority scheduling, and (3) optimistic methods, that (1) E(n, p) ⩽ n(1 - p/2)n-1, (2) E(n, p) ⩽ (1 - (1 - p)n)/p, and (3) 1 + ((1 - p)/p)ln(p(n - 1) + 1) ⩽ E(n, p) ⩽ 1 + (1/p)ln(p(n - 1) + 1). Thus, for fixed p, as n ↣ ∞), (1) E ↣ 0 for standard locking methods, (2) E ⩽ 1/p for strict priority scheduling methods, and (3) E ↣ ∞ for optimistic methods. Also found are bounds on E in the case where conflicts are analyzed so as to maximize E. The predictions of the random graph model are confirmed by simulations of an abstract transaction processing system. In practice, though, there is a price to pay for the increased effective level of concurrency of methods (2) and (3): using these methods there is more wasted work (i.e., more steps executed by transactions that are later aborted). In response to this problem, three new concurrency control methods suggested by the random graph model analysis are developed. Two of these, called (a) running priority and (b) older or running priority, are shown by the simulation results to perform better than the previously known methods (l)-(3) for relatively large n or large p, in terms of achieving a high effective level of concurrency at a comparatively small cost in wasted work.

Concurrency control for high contention environments

Future transaction processing systems may have substantially higher levels of concurrency due to reasons which include: (1) increasing disparity between processor speeds and data access latencies, (2) large numbers of processors, and (3) distributed databases. Another influence is the trend towards longer or more complex transactions. A possible consequence is substantially more data contention, which could limit total achievable throughput. In particular, it is known that the usual locking method of concurrency control is not well suited to environments where data contention is a significant factor. Here we consider a number of concurrency control concepts and transaction scheduling techniques that are applicable to high contention environments, and that do not rely on database semantics to reduce contention. These include access invariance and its application to prefetching of data, approximations to essential blocking such as wait depth limited scheduling, and phase dependent control. The performance of various concurrency control methods based on these concepts are studied using detailed simulation models. The results indicate that the new techniques can offer substantial benefits for systems with high levels of data contention.

On coupling multi-systems through data sharing

The demand for larger transaction rates and the inability of single-system-based transaction processors to keep up with demand have resulted in the growth of multi-processor-based database systems. The focus here is on coupling in a locally distributed system through multi-system data sharing in which all systems have direct access to the data. This paper addresses the following questions; i) How does a workload running on a single system today perform if migrated to a multi-system? ii) What are the multi-system locking design issues that limit multi-system performance and what is the maximum number of systems that may be effectively coupled? iii) Can alternative locking designs increase the number of systems that may be effectively coupled? Our analysis is based on traces from large mainframe systems running IBMs IMS database management system. We have developed a hierarchical modeling methodology that starts by synthesizing a multi-system IMS lock trace and a reference trace from single-system traces. The multisystem traces are used in trace-driven simulations to predict lock contention and database I/O increase in multi-system environment and to generate workload parameters. These parameters are used in event-driven simulation models to examine the overall performance under different system structures. Performance results are presented for realistic system parameters to determine the performance impact of various design parameters. Lock contention is found to be the critical factor in determining the coupling effectiveness and the effect of alternative locking design to reduce lock contention is studied. The limit on coupling is explored and the analysis indicates that, for this workload, on the order of 6 to 12 systems may be effectively coupled through data sharing, depending on system structure and locking design.

Parallel compression with cooperative dictionary construction

It is often desirable to compress or decompress relatively small blocks of data at high bandwidth and low latency (for example, for data fetches across a high speed network). Sequential compression may not satisfy the speed requirement, while simply splitting the block into smaller subblocks for parallel compression yields poor compression performance due to small dictionary sizes. We consider an intermediate approach, where multiple compressors jointly construct a dictionary. The result is parallel speedup, with compression performance similar to the sequential case.

Integrated concurrency-coherency controls for multisystem data sharing

The authors propose an integrated control mechanism and analyze the performance gain due to its use. An extension to the data sharing system structure is examined in which a shared intermediate memory is used for buffering and for early commit processing. Read-write-synchronization and write-serialization problems arise. The authors show how the integrated concurrency protocol can be used to overcome both problems. A queueing model is used to quantify the performance improvement. Although using intermediate memory as a buffering device produces a moderate performance benefit, the analysis shows that more substantial gains can be realized when this technique is combined with the use of an integrated concurrency-coherency control protocol. >

Progressive search and retrieval in large image archives

In this paper, we describe the architecture and implementation of a framework to perform content-based search of an image database, where content is specified by the user at one or more of the following three abstraction levels: pixel, feature, and semantic. This framework incorporates a methodology that yields a computationally efficient implementation of image-processing algorithms, thus allowing the efficient extraction and manipulation of user-specified features and content during the execution of queries. The framework is well suited for searching scientific databases, such as satellite-image-, medical-, and seismic-data repositories, where the volume and diversity of the information do not allow the a priori generation of exhaustive indexes, but we have successfully demonstrated its usefulness on still-image archives.

Algorithms and data structures for compressed-memory machines

An overview of a set of algorithms and data structures developed for compressed-memory machines is given. These include 1) very fast compression and decompression algorithms, for relatively small fixed-size lines, that are suitable for hardware implementation; 2) methods for storing variable-size compressed lines in main memory that minimize overheads due to directory size and storage fragmentation, but that are simple enough for implementation as part of a system memory controller; 3) a number of operating system modifications required to ensure that a compressed-memory machine never runs out of memory as the compression ratio changes dynamically. This research was done to explore the feasibility of computer architectures in which data are decompressed/compressed on cache misses/writebacks. The results led to and were implemented in IBM Memory Expansion Technology (MXT), which for typical systems yields a factor of 2 expansion in effective memory size with generally minimal effect on performance.

Analysis of steady-state segment storage utilizations in a log-structured file system with least-utilized segment cleaning

The steady-state distribution of storage utilizations of segments in a log-structured file system with least-utilized (greedy) segment cleaning is found using analytic methods. Furthermore, it is shown that as the number of segments increases, this distribution approaches a limiting continuous distribution, which is also derived. These results could be useful for preliminary performance analysis of LFS-type system designs prior to the development of detailed simulation models or actual implementation.

Access invariance and its use in high contention environments

Various factors suggest that data contention may be of increasing significance in transaction processing systems. One approach to this problem is to run transactions twice, the first time without making any changes to the database. Benefits may result either from data prefetching during the first execution or from determining the locks required for purposes of scheduling. Consideration is given to various concurrency control methods based on this notion, and properties required for these methods to be useful are formalized. Performance results based on detailed simulation models suggest that such policies offer potential benefits for some system configurations.<<ETX>>