Ezra Kissel

Efficient data transfer protocols for big data

Data set sizes are growing exponentially, so it is important to use data movement protocols that are the most efficient available. Most data movement tools today rely on TCP over sockets, which limits flows to around 20Gbps on todays hardware. RDMA over Converged Ethernet (RoCE) is a promising new technology for high-performance network data movement with minimal CPU impact over circuit-based infrastructures. We compare the performance of TCP, UDP, UDT, and RoCE over high latency 10Gbps and 40Gbps network paths, and show that RoCE-based data transfers can fill a 40Gbps path using much less CPU than other protocols. We also show that the Linux zero-copy system calls can improve TCP performance considerably, especially on current Intel “Sandy Bridge”-based PCI Express 3.0 (Gen3) hosts.

Comparative Evaluation of Spoofing Defenses

IP spoofing exacerbates many security threats, and reducing it would greatly enhance Internet security. Seven defenses that filter spoofed traffic have been proposed to date; three are designed for end-network deployment, while four assume some collaboration with core routers for packet marking or filtering. Because each defense has been evaluated in a unique setting, the following important questions remain unanswered: 1) Can end networks effectively protect themselves or is core support necessary? 2) Which defense performs best assuming sparse deployment? 3) How to select core participants to achieve best protection with fewest deployment points? This paper answers the above questions by: 1) formalizing the problem of spoofed traffic filtering and defining novel effectiveness measures, 2) observing each defense as selfish (it helps its participants) or altruistic (it helps everyone) and differentiating their performance goals, 3) defining optimal core deployment points for defenses that need core support, and 4) evaluating all defenses in a common and realistic setting. Our results offer a valuable insight into advantages and limitations of the proposed defenses, and uncover the relationship between any spoofing defenses performance and the Internets topology.

Driving Software Defined Networks with XSP

This paper presents the eXtensible Session Protocol (XSP), which provides a control plane for driving Software Defined Networks (SDNs). The XSP model supports proactive, application-driven configuration of dynamic network resources with support for authentication and authorization, within an extensible protocol framework. We describe XSP application use cases in SDNs using OpenFlow enabled network devices as well as dynamic forwarding rule management that can be implemented on existing router platforms.

Evaluating High Performance Data Transfer with RDMA-based Protocols in Wide-Area Networks

The use of zero-copy RDMA is a promising area of development in support of high-performance data movement over wide-area networks. In particular, the emerging RDMA over Converged Ethernet (RoCE) standard enables the InfiniBand transport for use over existing and widely deployed network infrastructure. In this paper, we evaluate the use of RDMA over Ethernet in two deployment scenarios: 1) a gateway approach that adapts standard application connections to an RDMA-based protocol for transmission over wide-area network paths, and 2) the integration of our RDMA implementation into GridFTP, a popular data transfer tool for distributed computing. We evaluate both approaches over a number of wide-area network conditions emulated using a commercial network emulation device, and we analyze the overhead of our RDMA implementations from a systems perspective. Our results show a significant increase in network utilization and performance when using RDMA over high-latency paths with a reduced CPU and memory I/O footprint on our gateways and end host applications.

Efficient wide area data transfer protocols for 100 Gbps networks and beyond

Due to a number of recent technology developments, now is the right time to re-examine the use of TCP for very large data transfers. These developments include the deployment of 100 Gigabit per second (Gbps) network backbones, hosts that can easily manage 40 Gbps, and higher, data transfers, the Science DMZ model, the availability of virtual circuit technology, and wide-area Remote Direct Memory Access (RDMA) protocols. In this paper we show that RDMA works well over wide-area virtual circuits, and uses much less CPU than TCP or UDP. We also characterize the limitations of RDMA in the presence of other traffic, including competing RDMA flows. We conclude that RDMA for Science DMZ to Science DMZ transfers of massive data is a viable and desirable option for high-performance data transfer.

Photon: Remote Memory Access Middleware for High-Performance Runtime Systems

We introduce the Photon RDMA middleware library that enables consistent remote memory access semantics over a number of network interconnect technologies. A primary goal of Photon is to expose a lightweight and flexible network abstraction that minimizes communication and message handling overheads for high-performance applications and runtime systems, in particular those that require the manipulation of objects within a global address space. Both one-sided and rendezvous communication models are supported and asynchronous network progress is exposed at a fine granularity. Photon implements a novel communication pattern called put-with-completion (PWC) that optimizes a completion notification path with variable size data for realizing active message-driven computation. The results of our performance evaluation show that our PWC model is comparable, and often improves upon, existing one-sided RDMA libraries in message latency and throughput metrics.

Scalable integrated performance analysis of multi-gigabit networks

Monitoring and managing multi-gigabit networks requires dynamic adaptation to end-to-end performance characteristics. This paper presents a measurement collection and analysis framework that automates the troubleshooting of end-to-end network bottlenecks. We integrate real-time host, application, and network measurements with a common representation (compatible with perfSONAR) within a flexible and scalable architecture. Our measurement architecture is supported by a light-weight eXtensible Session Protocol (XSP), which enables context-sensitive adaptive measurement collection. We evaluate the ability of our system to analyze and detect bottleneck conditions over a series of high-speed and I/O intensive bulk data transfer experiments and find that the overhead of the system is very low and that we are able to detect and understand a variety of bottlenecks.

Using phoebus data transfer accelerator in cloud environments

The quality of data exchange in cloud computing applications relies on the connection performance between user clients and their cloud storage providers, and is often dependent on the wide area network (WAN) properties among data centers. For certain classes of applications, it can be crucial to provide an end-to-end solution that accelerates large data transfers and improves overall user experience. The development and deployment of WAN optimization technology has been investigated for improving application performance in heterogeneous, multi-domain environments. WAN optimization devices and services implement a number of approaches for performance improvement, and one key insight is that in contrast to traditional end-to-end TCP connections, middleboxes that segment and optimize transport-layer connections can improve the performance of wide area data transfers. In the context of dynamic cloud computing environments, there is an obvious target for implementations of WAN optimization as Network Function Virtualization (NFV), where the flexibility of virtualized cloud environments can be exploited. This paper describes recent developments and experimentation of our Phoebus WAN accelerator framework. We introduce a software suite that includes new Phoebus clients that operate with the Phoebus Gateway network. We test and discuss virtualizing Phoebus Gateways to provide acceleration services in cloud data transfers. Use cases and performance evaluations are conducted on FutureGrid and Internet2 testbeds, and we demonstrate the effectiveness of a virtualized Phoebus deployment.

Design and Implementation of a Unified Network Information Service

A holistic view of the network is key to the successful operation of many distributed, cloud-based, and service-oriented computing architectures. Supporting network-aware applications and application-driven networks requires a detailed representation of network resources, including multi-layer topologies, associated measurement data, and in-the-network service location and availability information. The rapid development of increasingly configurable and dynamic networks has increased the demand for information services that can accurately and efficiently store and expose the state of the network. This work introduces our Unified Network Information Service (UNIS), designed to represent physical and virtual networks and services. We describe the UNIS network data model and its RESTful interface, which provide a common interface to topology, service, and measurement resources. In addition, we describe the security mechanisms built into the UNIS framework. Our analysis of the UNIS implementation shows significant performance and scalability gains over an existing and widely-deployed topology, service registration, and lookup information service architecture.

Network-Managed Virtual Global Address Space for Message-driven Runtimes

Maintaining a scalable high-performance virtual global address space using distributed memory hardware has proven to be challenging. In this paper we evaluate a new approach for such an active global address space that leverages the capabilities of the network fabric to manage addressing, rather than software at the endpoint hosts. We describe our overall approach, design alternatives, and present initial experimental results that demonstrate the effectiveness and limitations of existing network hardware.