Paul R. Wilson

Dynamic Storage Allocation: A Survey and Critical Review

Dynamic memory allocation has been a fundamental part of most computer systems since roughly 1960, and memory allocation is widely considered to be either a solved problem or an insoluble one. In this survey, we describe a variety of memory allocator designs and point out issues relevant to their design and evaluation. We then chronologically survey most of the literature on allocators between 1961 and 1995. (Scores of papers are discussed, in varying detail, and over 150 references are given.)

Hoard: a scalable memory allocator for multithreaded applications

Parallel, multithreaded C and C++ programs such as web servers, database managers, news servers, and scientific applications are becoming increasingly prevalent. For these applications, the memory allocator is often a bottleneck that severely limits program performance and scalability on multiprocessor systems. Previous allocators suffer from problems that include poor performance and scalability, and heap organizations that introduce false sharing. Worse, many allocators exhibit a dramatic increase in memory consumption when confronted with a producer-consumer pattern of object allocation and freeing. This increase in memory consumption can range from a factor of P (the number of processors) to unbounded memory consumption.This paper introduces Hoard, a fast, highly scalable allocator that largely avoids false sharing and is memory efficient. Hoard is the first allocator to simultaneously solve the above problems. Hoard combines one global heap and per-processor heaps with a novel discipline that provably bounds memory consumption and has very low synchronization costs in the common case. Our results on eleven programs demonstrate that Hoard yields low average fragmentation and improves overall program performance over the standard Solaris allocator by up to a factor of 60 on 14 processors, and up to a factor of 18 over the next best allocator we tested.

Uniprocessor Garbage Collection Techniques

We survey basic garbage collection algorithms, and variations such as incremental and generational collection. The basic algorithms include reference counting, mark-sweep, mark-compact, copying, and treadmill collection. Incremental techniques can keep garbage collection pause times short, by interleaving small amounts of collection work with program execution. Generational schemes improve efficiency and locality by garbage collecting a smaller area more often, while exploiting typical lifetime characteristics to avoid undue overhead from long-lived objects.

Design of the opportunistic garbage collector

The Opportunistic Garbage Collector (OGC) is a generational garbage collector for stock hardware and operating systems. While incorporating important features of previous systems, the OGC includes several innovations. A new bucket brigade heap organization supports advancement thresholds between one and two scavenges, using only two or three spaces per generation, and without requiring per-object counts. Opportunistic scavenging decouples scavenging from the filling of available memory, in order to hide potentially disruptive scavenge pauses and improve efficiency. Card marking efficiently records which small areas of the heap may contain pointers into younger generations, and is supported by a refinement of the crossing map technique, to enable scanning of arbitrary cards.

The memory fragmentation problem: solved?

We show that for 8 real and varied C and C++ programs, several conventional dynamic storage allocators provide near-zero fragmentation, once we account for overheads due to implementation details such as headers, alignment, etc. This substantially strengthens our previous results showing that the memory fragmentation problem has generally been misunderstood, and that good allocator policies can provide good memory usage for most programs. The new results indicate that for most programs, excellent allocator policies are readily available, and efficiency of implementation is the major challenge. While we believe that our experimental results are state-of-the-art and our methodology is superior to most previous work, more work should be done to identify and study unusual problematic program behaviors not represented in our sample.

Texas: An Efficient, Portable Persistent Store

Texas is a persistent storage system for C++, providing high performance while emphasizing simplicity, modularity and portability. A key component of the design is the use of pointer swizzling at page fault time, which exploits existing virtual memory features to implement large address spaces efficiently on stock hardware, with little or no change to existing compilers. Long pointers are used to implement an enormous address space, but are transparently converted to the hardware-supported pointer format when pages are loaded into virtual memory.

Effective “static-graph” reorganization to improve locality in garbage-collected systems

Several researchers have attempted to improve locality in garbage-collected heaps by changing the traversal algorithm used by a copying garbage collector. Unfortunately, these studies met with small success. We hypothesized that the disappointing results of these previous studies were due to two flaws in the traversal algorithms tested. They failed to group data structures in a manner reflecting their hierarchical organization, and more importantly, they ignored the dkastrous grouping effects caused by reaching data structures from a linear traversal of hash tables (i.e., in pseudo-random order). To test this hypothesis, we modified the garbage collector of a Lisp system (specifically, the Scheme-48 system) to avoid both problems in reorganizing the system heap image. We implemented our “hierarchical decomposition” algorithm (a cousin of Moon’s “approximately depth-fiist” algorithm) that is quite efficient on stock hardware. We aleo changed the collector to traverse global variable bindings in the order of their creation rather than in the memory order imposed by hash tables. The effects of these changes confirm our hypothesis. Some improvement comes from the basic traversal algorithm, and a greater effect results from the special treatment of hash tables. Initial page faults are reduced significantly and repeated page faults are reduced tremendously (by roughly an order of magnitude). In addition, improved measures of static locality (such as the percentage of on-page pointers) indicate that heap data can be cheaply and effectively compressed, and this may allow more effective paging and prefetching strategies; we suggest a level of “compressed in-RAM storage”, with price and performance between those of RAM and disk. These results call for a reinterpretation of previous studies, and for further studies of static-graph traversal techniques. They reduce the attractiveness of the exotic hardware needed for dynamically reorganizing objects within pages, and show that good locality can easily be achieved by straightforward and efficient techniques on stock hardware. Permission to copy without fee all or part of this material is granted provided that the copies ere not mede or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. @ 1991 ACM 0-89791-428-7/91/0005/01 77...

EELRU: simple and effective adaptive page replacement

1.50 Proceedings of the ACM SIGPLAN ‘91 Conference on Programming Language Design and Implementation. Toronto, Ontario, Canada, June 26-28, 1991.

Object Type Directed Garbage Collection To Improve Locality

Despite the many replacement algorithms proposed throughout the years, approximations of Least Recently Used (LRU) replacement are predominant in actual virtual memory management systems because of their simplicity and efficiency. LRU, however, exhibits well-known performance problems for regular access patterns of size larger than the main memory. In this paper we present Early Eviction LRU (EELRU): an adaptive replacement algorithm based on the principle of detecting when the LRU algorithm underperforms (i.e., when the fetched memory pages are often the ones evicted lately). In simulations, EELRU proves to be quite effective for many memory sizes and several applications, often decreasing paging by over 30% for programs with large-scale reference patterns and by over lOSo for programs with small-scale patterns. Additionally, the algorithm is very robust, rarely underperforming LRU. Our experiments are mostly with traces from the recent research literature to allow for easy comparison with previous results.

Caching considerations for generational garbage collection

Most garbage collected systems have excessive need for RAM to achieve reasonable performance without too much paging. The reason for such poor locality is the way data are organized in the heap. Conventional organization approaches such as breadth-first ordering do not always bring objects in the same active working set together. When such co-active objects are distributed throughout the heap (on different memory pages), high paging costs will result from accessing objects during execution. To alleviate such poor ordering, researchers have tried many different approaches: depth-first ordering, dynamic reorganization, object creation ordering, and hierarchical decomposition. Each of these approaches has its associated costs, effectiveness, and limitations. This paper presents a new ordering approach to improve locality. By paying a little attention to object type and format, effective heuristics can be derived to group co-active objects together. To investigate this idea, a number of such object type directed grouping techniques are incorporated into a Scheme-48 system. Page fault reduction of up to an order of magnitude was observed.