Srinivasan Narayanamurthy

Model building for dynamic multi-tenant provider environments

Increasingly, storage vendors are finding it difficult to leverage existing white-box and black-box modeling techniques to build robust system models that can predict system behavior in the emerging dynamic and multi-tenant data centers. White-box models are becoming brittle because the model builders are not able to keep up with the innovations in the storage system stack, and black-box models are becoming brittle because it is increasingly difficult to a priori train the model for the dynamic and multi-tenant data center environment. Thus, there is a need for innovation in system model building area. In this paper we present a machine learning based blackbox modeling algorithm called M-LISP that can predict system behavior in untrained region for these emerging multitenant and dynamic data center environments. We have implemented and analyzed M-LISP in real environments and the initial results look very promising. We also provide a survey of some common machine learning algorithms and how they fare with respect to satisfying the modeling needs of the new data center environments.

WORMStore: A Specialized Object Store for Write-Once Read-Many Workloads

The recent increase in interest for batch analytics has resulted in extensive use of distributed frameworks such as Hadoop and Dryad. Batch analytics-as the name suggests, perform many computations on large volumes of data. That is, large quantities of data are ingested once and read many times mostly in large chunks, which is characterized as write-once read-many (WORM) workload. The storage part of these distributed frameworks (say, HDFS in Hadoop) use file systems such as ext4 or XFS as native object stores to store objects as files in individual nodes of the distributed system. These general purpose file systems were designed with broader goals such as POSIX-compliance, optimal performance for a wide range of file size, user friendliness, etc. However, most of these features are not required for a native object store in distributed file systems. WORM Store is a light weight object store that is designed exclusively for use in distributed systems for WORM workload. WORM Store provides interesting advantages such as the ability to pre-fetch large objects, small metadata to data ratio, media aware data/metadata placement, etc. As WORM Store is log-structured, it provides the ability to recover upon failure. Our experiments show that WORM Store provides a 28% increase in the read throughput per node in a Hadoop cluster.