Kazuichi Oe

Managing Non-Volatile Memory in Database Systems

Non-volatile memory (NVM) is a new storage technology that combines the performance and byte addressability of DRAM with the persistence of traditional storage devices like flash (SSD). While these properties make NVM highly promising, it is not yet clear how to best integrate NVM into the storage layer of modern database systems. Two system designs have been proposed. The first is to use NVM exclusively, i.e., to store all data and index structures on it. However, because NVM has a higher latency than DRAM, this design can be less efficient than main-memory database systems. For this reason, the second approach uses a page-based DRAM cache in front of NVM. This approach, however, does not utilize the byte addressability of NVM and, as a result, accessing an uncached tuple on NVM requires retrieving an entire page. In this work, we evaluate these two approaches and compare them with in-memory databases as well as more traditional buffer managers that use main memory as a cache in front of SSDs. This allows us to determine how much performance gain can be expected from NVM. We also propose a lightweight storage manager that simultaneously supports DRAM, NVM, and flash. Our design utilizes the byte addressability of NVM and uses it as an additional caching layer that improves performance without losing the benefits from the even faster DRAM and the large capacities of SSDs.

On-the-Fly Automated Storage Tiering with Caching and both Proactive and Observational Migration

A hybrid storage system consisting of SSDs and HDDs improves the performance of user IOs by moving IO concentrated areas from the HDDs to the SSDs. It is important for the system to identify such areas and move them to the SSDs immediately. We studied the IO concentrations of operational storage servers and found one type that they continue for up to an hour within a narrow region of storage volume. The narrow region is either the same continued area or the area which moves to the neighboring after 3 minutes in average had passed. Moreover, it appeared at random logical block addresses (LBAs). We decide to call such IO concentrations Wandering IO Concentrated Areas (WIOCAs). To handle these concentrations, we developed an on-the-fly automated storage tiering (OTF-AST) with a caching system and both proactive and observational migration features. This method dynamically migrates the WIOCAs to the SSD of the OTF-AST and hands over the other IO concentrations to cache. The proactive migration feature migrates candidate WIOCAs to an area that will likely become one in the near future. On the other hand, the observational migration feature determines the HDD overhead by checking the user response time and migrates the sub-LUNs that were but are no longer parts of a WIOCA from the SSD to the HDD when the HDD overhead is low. Experimental results showed that our method was 113% faster than Facebook FlashCache on one of the Microsoft Research Cambridge workloads.

A Hybrid Storage System Composed of On-the-Fly Automated Storage Tiering (OTF-AST) and Caching

In a previous workload study, we found a new type of I/O concentration approximately 6 GB in size that continues for more than 10 minutes and then moves on to another area. We also found that the conventional hybrid storage method, static tiering and caching, is not sufficient for this I/O concentration. The on-the-fly automated storage tiering (OTF-AST) designed for this I/O concentration cannot handle other I/O concentrations well, so we have developed a hybrid storage system composed of both OTF-AST and caching. In the proposed system, the OTF-AST dynamically migrates the new I/O concentration to SSD and the caching handles the other I/O concentrations. Experimental results showed that the hybrid storage system surpasses the basic OTF-AST system with an average response time 27% better for one of a typical samba workload in which read is not predominant and I/O concentration lasts for 20 minutes.

Analysis of storage workloads of input-output access locality and designing of hybrid storage system

Hybrid storage systems consisting of SSDs and HDDs improve the performance of input-output (IO) access by migrating IO concentrations from HDDs to SSDs. Therefore, it is important for the system to identify such concentrations and to immediately migrate them to SSDs. This study analyzes the IO access logs of shared filesystems, the Microsoft (MS) Exchange Server, and the Virtual Desktop Infrastructure (VDI) from the viewpoint of access locality. This study illustrates common features of the IO access locality of the portions of storage volume and elapsed time. Moreover, we examine the selection of a hybrid storage system for these features. We have determined that on-the-fly automated storage tiering is suitable for both shared filesystems and the MS Exchange Server and caching is suitable for VDI.