Aled Edwards

User-space protocols deliver high performance to applications on a low-cost Gb/s LAN

Two important questions in high-speed networking are firstly, how to provide Gbit/s networking at low cost and secondly, how to provide a flexible low-level network interface so that applications can control their data from the instant it arrives. We describe some work that addresses both of these questions. The Jetstream Gbit/s LAN is an experimental, low-cost network interface that provides the services required by delay-sensitive traffic as well as meeting the performance needs of current applications. Jetstream is a combination of traditional shared-medium LAN technology and more recent ATM cell- and switch-based technology. Jetstream frames contain a channel identifier so that the network driver can immediately associate an incoming frame with its application. We have developed such a driver that enables applications to control how their data should be managed without the need to first move the data into the applications address space. Consequently, applications can elect to read just a part of a frame and then instruct the driver to move the remainder directly to its destination. Individual channels can elect to receive frames that have failed their CRC, while applications can specify frame-drop policies on a per-channel basis. Measured results show that both kernel- and user-space protocols can achieve very good throughput: applications using both TCP and our own reliable byte-stream protocol have demonstrated throughputs in excess of 200 Mbit/s. The benefits of running protocols in user-space are well known- the drawback has often been a severe penalty in the performance achieved. In this paper we show that it is possible to have the best of both worlds.

Afterburner (network-independent card for protocols)

Many current implementations of protocols such as the Transmission Control Protocol/Internet Protocol (TCP/IP) are inefficient because data are often accessed more frequently than necessary. Three techniques that reduce the need for memory bandwidth are proposed. The techniques are copy-on-write, page remapping, and single-copy. Afterburner, a network-independent card that provides the services that are necessary for a single-copy protocol stack, is described. The card has 1 MByte of local buffers and provides a simple interface to a variety of network link adapters, including HIPPI and asynchronous transfer mode (ATM). Afterburner can support transfers to and from the link adapter card at rates up to 1 Gbit/s. An implementation of TCP/IP that uses the features provided by Afterburner to reduce the movement of data to a single copy is discussed. Measurements of the end-to-end performance of Afterburner and the single-copy implementation of TCP/IP are presented.<<ETX>>

AAL5 at a Gigabit for a Kilobuck

We present a novel LAN that has been designed to meet three criteria: it should be low cost, support standard protocols, and provide high performance. The LAN interconnects many workstations in a ring topology with a link rate of 1 Gbit/s. The packet format is the same as that of a B-ISDN cell except that our packets can have arbitrary length. The network also provides hardware support for ATM Adaptation Layer 5 as weIl as TCP/IP.We have implemented the ring network and we describe a network interface card which provides hardware support for critical functions such as calculating checksums. This card is used in conjunction with a second card that supports a single-copy implementation of the TCP/IP protocols. The application-to-application throughput has been measured at rates in excess of 200 Mbit/s between two workstations.

Rapid Node Reallocation Between Virtual Clusters for Data Intensive Utility Computing

Utility computing achieves efficiencies by dynamically reallocating shared resources between services operating on virtual clusters. These efficiencies can be hard to realize for data intensive applications; newly allocated nodes must be populated with a large amount of data which impedes rapid node reallocation. We describe a data management architecture that uses disk caches on each node to reduce data copying and speed up node reallocation for data intensive applications. Cache consistency management is simplified by extensive use of copy-on-write techniques. A data-driven scheme is then used to select nodes for reallocation between virtual clusters based on the amount of relevant cached data. These nodes are identified using a novel technique of statistically sampling the contents of caches. We demonstrate the benefits of this architecture using our implementation of an efficient block level caching and copy-on-write target for the Linux device-mapper framework

High-speed Storage Nodes for the Cloud

In this paper, we describe an architecture for high-speed storage nodes intended for supporting cloud-based storage I/O intensive applications such as file servers, backup servers and databases. The nodes can host multiple virtual machines each having direct access to a storage array via Single Route I/O Virtualization (SR-IOV). This is done in a way which does not compromise security. We demonstrate that SR-IOV imposes negligible overhead for storage I/O and provides the hosted virtual machines with four times the storage I/O bandwidth than is available from the same hardware when I/O is redirected through the hyper visor. We describe how the storage nodes are incorporated into a heterogeneous Infrastructure as a Service (IaaS) consisting of mixed storage and compute nodes.

Cells: A Self-Hosting Virtual Infrastructure Service

We describe the design and implementation of Cells, a novel multi-tenanted virtual infrastructure service. Cells has the unique property of being self-hosting: it operates its own management system within one of the tenant virtual infrastructures that it manages and therefore benefits from the same security, flexibility and scalability as other tenant services. Cells is also differentiated by its declarative interface for infrastructure configuration, and its fine-grained control of network and storage connections within and between tenant infrastructures.