Tudor Marian

Integrated Approach to Data Center Power Management

Energy accounts for a significant fraction of the operational costs of a data center, and data center operators are increasingly interested in moving toward low-power designs. Two distinct approaches have emerged toward achieving this end: the power-proportional approach focuses on reducing disk and server power consumption, while the green data center approach focuses on reducing power consumed by support-infrastructure like cooling equipment, power distribution units, and power backup equipment. We propose an integrated approach, which combines the benefits of both. Our solution enforces power-proportionality at the granularity of a rack or even an entire containerized data center; thus, we power down not only idle IT equipment, but also their associated support-infrastructure. We show that it is practical today to design data centers to power down idle racks or containers—and in fact, current online service trends strongly enable this model. Finally, we show that our approach combines the energy savings of power-proportional and green data center approaches, while performance remains unaffected.

NetSlices: scalable multi-core packet processing in user-space

Modern commodity operating systems do not provide developers with user-space abstractions for building high-speed packet processing applications. The conventional raw socket is inefficient and unable to take advantage of the emerging hardware, like multi-core processors and multi-queue network adapters. In this paper we present the NetSlice operating system abstraction. Unlike the conventional raw socket, NetSlice tightly couples the hardware and software packet processing resources, and provides the application with control over these resources. To reduce shared resource contention, NetSlice performs domain specific, coarse-grained, spatial partitioning of CPU cores, memory, and NICs. Moreover, it provides a streamlined communication channel between NICs and user-space. Although backward compatible with the conventional socket API, the NetSlice API also provides batched (multi-) send / receive operations to amortize the cost of protection domain crossings. We show that complex user-space packet processors—like a protocol accelerator and an IPsec gateway—built from commodity components can scale linearly with the number of cores and operate at 10Gbps network line speeds.

Empirical characterization of uncongested optical lambda networks and 10GbE commodity endpoints

High-bandwidth, semi-private optical lambda networks carry growing volumes of data on behalf of large data centers, both in cloud computing environments and for scientific, financial, defense, and other enterprises. This paper undertakes a careful examination of the end-to-end characteristics of an uncongested lambda network running at high speeds over long distances, identifying scenarios associated with loss, latency variations, and degraded throughput at attached end-hosts. We use identical fast commodity source and destination platforms, hence expect the destination to receive more or less what we send. We observe otherwise: degraded performance is common and easily provoked. In particular, the receiver loses packets even when the sender employs relatively low data rates. Data rates of future optical network components are projected to outpace clock speeds of commodity end-host processors, hence more and more end-to-end applications will confront the same issue we encounter. Our work thus poses a new challenge for those hoping to achieve dependable performance in higher-end networked settings.

Tempest: Soft state replication in the service tier

Soft state in the middle tier is key to enabling scalable and responsive three tier service architectures. While soft-state can be reconstructed upon failure, replicating it across multiple service instances is critical for rapid fail-over and high availability. Current techniques for storing and managing replicated soft state require mapping data structures to different abstractions such as database records, which can be difficult and introduce inefficiencies. Tempest is a system that provides programmers with data structures that look very similar to conventional Java Collections but are automatically replicated. We evaluate Tempest against alternatives such as in-memory databases and we show that Tempest does scale well in real world service architectures.

Exact temporal characterization of 10 Gbps optical wide-area network

We design and implement a novel class of highly precise network instrumentation and apply this tool to perform the first exact packet-timing measurements of a wide-area network ever undertaken, capturing 10 Gigabit Ethernet packets in flight on optical fiber. Through principled design, we improve timing precision by two to six orders of magnitude over existing techniques. Our observations contest several common assumptions about behavior of wide-area networks and the relationship between their input and output traffic flows. Further, we identify and characterize emergent packet chains as a mechanism to explain previously observed anomalous packet loss on receiver endpoints of such networks.

Maelstrom: transparent error correction for communication between data centers

The global network of data centers is emerging as an important distributed systems paradigm-commodity clusters running high-performance applications, connected by high-speed “lambda” networks across hundreds of milliseconds of network latency. Packet loss on long-haul networks can cripple applications and protocols: A loss rate as low as 0.1% is sufficient to reduce TCP/IP throughput by an order of magnitude on a 1-Gb/s link with 50-ms one-way latency. Maelstrom is an edge appliance that masks packet loss transparently and quickly from intercluster protocols, aggregating traffic for high-speed encoding and using a new forward error correction scheme to handle bursty loss.

Fmeter: extracting indexable low-level system signatures by counting kernel function calls

System monitoring tools serve to provide operators and developers with an insight into system execution and an understanding of system behavior under a variety of scenarios. Many system abnormalities leave a significant impact on the system execution which may arise out of performance issues, bugs, or errors. Having the ability to quantify and search such behavior in the system execution history can facilitate new ways of looking at problems. For example, operators may use clustering to group and visualize similar system behaviors. We propose a monitoring system that extracts formal, indexable, low-level system signatures using the classical vector space model from the field of information retrieval and text mining. We drive an analogy between the representation of kernel function invocations with terms within text documents. This parallel allows us to automatically index, store, and later retrieve and compare the system signatures. As with information retrieval, the key insight is that we need not rely on the semantic information in a document. Instead, we consider only the statistical properties of the terms belonging to the document (and to the corpus), which enables us to provide both an efficient way to extract signatures at runtime and to analyze the signatures using statistical formal methods. We have built a prototype in Linux, Fmeter, which extracts such low-level system signatures by recording all kernel function invocations. We show that the signatures are naturally amenable to formal processing with statistical methods like clustering and supervised machine learning.

Scalable Multicast Platforms for a New Generation of Robust Distributed Applications

As distributed systems scale up and are deployed into increasingly sensitive settings, demand is rising for a new generation of communications middleware in support of application-level critical-computing uses. Ricochet, Tempest and Quicksilver are multicast-based systems developed to respond to this need. Ricochet and Quicksilver are multicast platforms; both are exceptionally scalable and support fault-tolerance properties that match closely with the needs of high-availability applications. Ricochet was designed to support time-critical applications replicated for scalability on data centers and clusters. These are typically coded in Java and run under Linux. Tempest is layered over Ricochet and automates most tasks of programming services for data centers. In contrast, Quicksilver focuses on high throughput and is targeted towards very large deployments of desktop computing systems, in support of publish-subscribe, event notification or media dissemination applications. In this paper we offer an overview of the systems and some of the new systems embeddings that, we believe, make them far easier to use than was the case in prior multicast platforms.

Towards Weakly Consistent Local Storage Systems

Heterogeneity is a fact of life for modern storage servers. For example, a server may spread terabytes of data across many different storage media, ranging from magnetic disks, DRAM, NAND-based solid state drives (SSDs), as well as hybrid drives that package various combinations of these technologies. It follows that access latencies to data can vary hugely depending on which media the data resides on. At the same time, modern storage systems naturally retain older versions of data due to the prevalence of log-structured designs and caches in software and hardware layers. In a sense, a contemporary storage system is very similar to a small-scale distributed system, opening the door to consistency/performance trade-offs. In this paper, we propose a class of local storage systems called StaleStores that support relaxed consistency, returning stale data for better performance. We describe several examples of StaleStores, and show via emulations that serving stale data can improve access latency by between 35% and 20X. We describe a particular StaleStore called Yogurt, a weakly consistent local block storage system. Depending on the applications consistency requirements (e.g. bounded staleness, mono-tonic reads, read-my-writes, etc.), Yogurt queries the access costs for different versions of data within tolerable staleness bounds and returns the fastest version. We show that a distributed key-value store running on top of Yogurt obtains a 6X speed-up for access latency by trading off consistency and performance within individual storage servers.

Instrumentation for exact packet timings in networks

We design and implement a novel class of highly precise network instrumentation, capable of the first-ever capture of exact packet timings of network traffic. Our instrumentation — combining real-time physics test equipment with off-line postprocessing software — prevents interference with the system under test, provides reproducible measurements by eliminating non-deterministic error, and uses transparent and ubiquitous lab equipment and open-source software for ease of replication. We use our technique to perform in-situ observations of 10 Gigabit Ethernet packets in flight on optical fiber, showing improvements in timing precision of two to six orders of magnitude over existing methods of measurement, which generally employ software on commodity computer endpoints of network paths.