Shripad Nadgowda

iSAN: Storage Area Network Management Modeling Simulation

Storage management plays an important role in ensuring the service level agreements (availability, reliability, performance etc..) that are critical to the operation of resilient business IT infrastructure. Also, storage resource management (SRM) is becoming the largest component in the overall cost of ownership of any large enterprise IT environment. Considering these functional requirements and business opportunities, several SRM suites have cropped up in the market place to provide uniform and interoperable management. But, development and test of these suites require access to huge set of heterogeneous multi- vendor storage area network (SAN) devices like fiber channel switches, storage subsystems, tape libraries, servers etc... Its almost impractical for a SRM Suite software manufacturer to own and manage these huge varieties of devices. Management modules of SAN devices have become logically independent components with the emergence of CIM and SMI-S. In this paper, we propose a framework and implementation named iSAN (imitation storage area network) that models the management module of the devices. Our tool can be used to perform a) simulation of management module ranging from individual device to large scale multi-vendor heterogeneous SAN for enterprise b) what-ifanalysis of enterprise IT environment before modeling the changes. This tool provides efficiency and cost- effectiveness with respect to development, test of SRM suites and planning of IT environment by removing their dependence on high cost SAN hardware. We have implemented this tool iSAN, and our experiment results show that one can attain the above mentioned functional objectives and bring significant productivity to enterprise IT environment management.

Towards Auto-remediation in Services Delivery: Context-Based Classification of Noisy and Unstructured Tickets

Service interactions account for major source of revenue and employment in many modern economies, and yet the service operations management process remains extremely complex. Ticket is the fundamental management entity in this process and resolution of tickets remains largely human intensive. A large portion of these human executed resolution tasks are repetitive in nature and can be automated. Ticket description analytics can be used to automatically identify the true category of the problem. This when combined with automated remediation actions considerably reduces the human effort. We look at monitoring data in a big provider’s domain and abstract out the repeatable tasks from the noisy and unstructured human-readable text in tickets. We present a novel approach for automatic problem determination from this noisy and unstructured text. The approach uses two distinct levels of analysis, (a) correlating different data sources to obtain a richer text followed by (b) context based classification of the correlated data. We report on accuracy and efficiency of our approach using real customer data.

Voyager: Complete Container State Migration

Due to the small memory footprint and fast startup times offerred by container virtualization, made ever more popular by the Docker platform, containers are seeing rapid adoption as a foundational capability to build PaaS and SaaS clouds. For such container clouds, which are fundamentally different from VM clouds, various cloud management services need to be revisited. In this paper, we present our Voyager - just-in-time live container migration service, designed in accordance with the Open Container Initiative (OCI) principles. Voyager is a novel filesystem-agnostic and vendor-agnostic migration service that provides consistent full-system migration. Voyager combines CRIU-based memory migration together with the data federation capabilities of union mounts to minimize migration downtime. With a union view of data between the source and target hosts, Voyager containers can resume operation instantly on the target host, while performing disk state transfer lazily in the background.

12MAP: Cloud Disaster Recovery Based on Image-Instance Mapping

Virtual machines (VMs) in a cloud use standardized ‘golden master’ images, standard software catalog and management tools. This facilitates quick provisioning of VMs and helps reduce the cost of managing the cloud by reducing the need for specialized software skills. However, knowledge of this similarity is lost post-provisioning, as VMs could experience different changes and may drift away from one another. In this work, we propose the 12MAP system, which maintains a mapping between each instance and the golden master image from which it was created, consisting of a record of all changes to the instance since provisioning. We motivate that this mapping can aid several cloud management activities such as disaster recovery, system administration, and troubleshooting. We build a host-based disaster recovery solution based on 12MAP, which is ideally suited for low cost cloud VMs that do not have access to dedicated block-based storage recovery solutions. Our solution deduplicates changes across VMs and needs to replicate only the unique changes, significantly reducing replication traffic on end hosts. We demonstrate that 12MAP is able to deliver on tight recovery time and recovery point objectives of the order of minutes with low overhead. Compared to state-of-the-art host-based recovery solutions, 12MAP is able to save 50-87% network bandwidth on the primary data center.

Comparing Scaling Methods for Linux Containers

Linux containers are shaping the new era of building applications. With their low resource utilization overhead and lightweight images, they present an appealing model to package and run applications. The faster boot time of containers compared to virtual/physical machines makes them ideal for auto-scaling and on-demand provisioning. Several methods can be used to spawn new containers. In this paper we compare three different methods in terms of start time, resource consumption and post-start performance. We discuss the applicability, advantages and shortcomings of each method, and conclude with our recommendations.

Optimizing Service Delivery with Minimal Runtimes

In this paper, we argue that deploying applications inside minimal runtime environments, which only contain the files necessary and sufficient for the application to run, can cut down the operating costs, specifically the costs for ensuring the application security and compliance. We identify a way to deliver minimal runtimes as Docker containers built from scratch. We describe a use case where minimal runtimes simplify the service maintenance operations, by reducing the number of updates for fixing security vulnerabilities.

LVD: lean virtual disks

In this work, we present Lean Virtual Disks (LVD), a new virtual disk format for virtualized servers. LVD transparently consolidates duplicate blocks across virtual machines to create a lean disk image, leading to a merged datapath for all virtual machines. This merged datapath allows efficient storage usage, reduction in disk I/O (read/write) by eliminating I/O for same content across VMs and efficient host cache utilization. LVD is motivated by clouds, where VMs are created from golden masters and use standardized middleware and management tools leading to high content similarity. We implement LVD as an extension of QCow2 and study its ability to improve common data center system management activities as well as improving application performance of popular I/O benchmark workloads. We observed that LVD reduced disk space and disk I/O by 70%, making applications run faster by 25% on an average.

Columbus: Filesystem Tree Introspection for Software Discovery

Software discovery is a key management function to ensure that systems are free of vulnerabilities, comply with licensing requirements, and support advanced search for systems containing given software. Today, software is predominantly discovered through querying package management tools, or using rules that check for file metadata or contents. These approaches are inadequate as not every software is installed through package managers, and agile development practices lead to frequent deployment of software. Other approaches to software discovery use machine learning methods requiring training phase, or require maintaining knowledge bases. Columbus uses the knowledge of the software packaging practices that evolved over time, and uses the information embedded in the file system impression created by a software package to discover it. Columbus is able to discover software in 92% of all official Docker images. Further, Columbus can be used in problem diagnosis and drift detection situations to compare two different systems, or to determine the evolution of a system overtime.

The less server architecture for cloud functions

Serverless execution model is becoming an increasingly prominent choice to host data processing applications. In this event-based model, predefined stateless functions are triggered for execution on an event when new data becomes available. We make two critical observations in this framework - first, data sources in the serverless framework are largely-consistent, causing lot of data duplication, and second, stateless functions are deterministic, i.e., they compute same result on multiple applications of duplicate data. Leveraging these two insights in this work, we introduce Sanity, a novel storage de-duplication framework for serverless platform that not only performs data de-duplication but also de-duplicates events to avoid redundant execution of stateless functions. This helps improve overall throughput of the serverless platform through execution of mostly unique functions. We present the design of Sanity for a continuous container vulnerability check use-case and discuss its performance implications.

C2P: Co-operative Caching in Distributed Storage Systems

Distributed storage systems (e.g. clustered filesystems - HDFS, GPFS and Object Stores - Openstack swift ) often partition sequential data across storage systems for performance ( data striping) or protection (Erasure-Coding) . This partitioning leads to logically correlated data being stored on different physical storage devices, which operate autonomously. This un-coordinated operation may lead to inefficient caching, where different devices may cache segments that belong to different working sets. From an application perspective, caching is effective only if all segments needed by it at a given point in time are cached and a single missing segment may lead to high application latency. In this work, we present C2P: a middleware for co-operative caching in distributed storage. C2P uses an event-based architecture to co-ordinate caching across the storage devices and ensures that all devices cache correlated segments. We have implemented C2P as a caching middleware for hosted Openstack Swift Object Store. Our experiments show 4-6% improved cache hit and 3-5% reduced disk IO with minimal resource overheads.