Sridhar Seshadri

The Architecture of the Dalí Main-Memory Storage Manager

Performance needs of many database applicationsdictate that the entire database be stored in main memory.The dali system is a main memory storage manager designed toprovide the persistence, availability and safety guarantees one typically expects from a disk-resident database, while at the same time providing very high performance by virtue of being tuned to support in-memory data.User processes map the entire database into their address space andaccess data directly, thus avoiding expensive remote procedure calls andbuffer manager interactionstypical of accesses in disk-resident commercial systems available today.dali recovers the database to a consistent state in the case of system as well as process failures. It alsoprovides unique concurrency control and memory protection features, aswell as ordered and unordered index structures. Both object-oriented and relational database management systems have beenimplemented on top of dali. dali provides access to multiple layers ofapplication programming interface, including its low-level recovery,concurrency control and indexing components as well as its high-levelrelational component. Finally, various features of dali can be tailored tothe needs of an application to achieve high performance–for example,concurrency control and logging can be turned off if not desired, enablingdali to efficiently support applications that requirenon-persistent memory-resident data to be shared by multiple processes.

Distributed multi-level recovery in main-memory databases

In this paper we present recovery techniques for distributed main-memory databases, specifically for client-server and shared-disk architectures. We present a recovery scheme for client-server architectures which is based on shipping log records to the server, and two recovery schemes for shared-disk architectures—one based on page shipping, and the other based on broadcasting of the log of updates. The schemes offer different tradeoffs, based on factors such as update rates.Our techniques are extensions to a distributed-memory setting of a centralized recovery scheme for main-memory databases, which has been implemented in the Dalì main-memory database system. Our centralized as well as distributed-memory recovery schemes have several attractive features—they support an explicit multi-level recovery abstraction for high concurrency, reduce disk I/O by writing only redo log records to disk during normal processing, and use per-transaction redo and undo logs to reduce contention on the system log. Further, the techniques use a fuzzy checkpointing scheme that writes only dirty pages to disk, yet minimally interferes with normal processing—all but one of our recovery schemes do not require updaters to even acquire a latch before updating a page. Our log shipping/broadcasting schemes also support concurrent updates to the same page at different sites.