Mitchell Gusat

Current issues in packet switch design

Addressing the ever growing capacity demand for packet switches, current research focuses on scheduling algorithms or buffer bandwidth reductions. Although these topics remain relevant, our position is that the primary design focus for systems beyond 1 Tb/s must be shifted to aspects resulting from packaging disruptions. Based on trends such as increased link rates and improved CMOS technologies, we derive new design factors for such switch fabrics. For instance, we argue that the packet round-trip transmission time within the fabric has become a major design parameter. Furthermore, we observe that high-speed fabrics have become extremely dependent on serial I/O technology that is both high speed and high density. Finally, we conclude that in developing the architecture, packaging constraints must be put first and not as an afterthought, which also applies to solving the tremendous power consumption challenges.

A four-terabit packet switch supporting long round-trip times

This 4-TBPS packet switch uses a combined input- and crosspoint-queued (CICQ) structure with virtual output queuing at the ingress to achieve the scalability of input-buffered switches, the performance of output-buffered switches, and low latency.

Designing a Crossbar Scheduler for HPC Applications

A crucial part of any high-performance computing (HPC) system is its interconnection network. Corning and IBM are jointly developing a demonstration interconnect based on optical cell switching with electronic control. The Corning-IBM joint optical shared memory supercomputer interconnect system (Osmosis) project explores the opportunity to advance the role of optical-switching technologies in such systems. Key innovations in the scheduler architecture directly address the main HPC requirements: low latency, high throughput, efficient multicast support, and high reliability

Design issues in next-generation merchant switch fabrics

Packet-switch fabrics with widely varying characteristics are currently deployed in the domains of both communications and computer interconnection networks. For economical reasons, it would be highly desirable that a single switch fabric could accommodate the needs of a variety of heterogeneous services and applications from both domains. In this paper, we consider the current requirements, technological trends, and their implications on the design of an ASIC chipset for a merchant switch fabric. We then identify the architecture upon which such a suitable and generic switch fabric could be based, and we present the general characteristics of an implementation of this switching fabric within the bounds of current state-of-the-art technology. To our knowledge, this is the first attempt to design a chipset that can be used for both communications and computer interconnection networks.

A four-terabit single-stage packet switch with large round-trip time support

We present the architecture and practical VLSI implementation of a 4-Tb/s single-stage switch. It is based on a combined input- and crosspoint-queued structure with virtual output queuing at the ingress, which has the scalability of input-buffered switches and the performance of output-buffered switches. Our system handles the large fabric-internal transmission latency that results from packaging up to 256 line cards into multiple racks. We provide the justification for selecting this architecture and compare it with other current solutions. With an ASIC implementation, we show that a single-stage multi-terabit buffered crossbar approach is viable today.

Short and Fat: TCP Performance in CEE Datacenter Networks

One of the consequential new features of emerging datacenter networks is lossless ness, achieved by means of Priority Flow Control (PFC). Despite PFCs key role in the datacenter and its increasing availability -- supported by virtually all Converged Enhanced Ethernet (CEE) products -- its impact remains largely unknown. This has motivated us to evaluate the sensitivity of three widespread TCP versions to PFC, as well as to the more involved Quantized Congestion Notification (QCN) congestion management mechanism. As datacenter workloads we have adopted several representative commercial and scientific applications. For evaluation we employ an accurate Layer 2 CEE network simulator coupled with a TCP implementation extracted from FreeBSD v9. A somewhat unexpected outcome of this investigation is that PFC significantly improves TCP performance across all tested configurations and workloads, hence our recommendation to enable PFC whenever possible. In contrast, QCN can help or harm depending on its parameter settings, which are currently neither adaptive nor universal for datacenters. To the best of our knowledge this is the first performance evaluation of TCP performance in lossless CEE networks.

Control path implementation for a low-latency optical HPC switch

A crucial part of any high-performance computing system is its interconnection network. In the OSMOSIS project, Corning and IBM are jointly developing a demonstrator interconnect based on optical cell switching with electronic control. Starting from the core set of requirements, we present the system design rationale and show how it impacts the practical implementation. Our focus is on solving the technical issues related to the electronic control path, and we show that it is feasible at the targeted design point.

Reducing memory size in buffered crossbars with large internal flow control latency

A buffered crossbar supporting P priorities and a flow control latency of RT packets between the input adapter and the crossbar requires a memory of order O(N/sup 2/*P*RT) packets in the crossbar to support any traffic pattern without blocking. We propose a new priority elevation mechanism that reduces the memory requirements to O(N/sup 2/*RT) for large values of RT. Our analysis shows that our mechanism has no drawback on the usual performance metrics and that it only introduces a small priority unfairness and worst-case blocking of less than RT packet times. We further show that an optimized system using a crosspoint memory of size 2RT has an unfairness of less than 0.02% affected packets at 95% loading with an average burst size of 30 packets.

Adaptive routing for Convergence Enhanced Ethernet

A significant drive to consolidate data center networks on a single infrastructure is taking place. 10-Gigabit Ethernet is one of the contenders to fulfill the role of universal data center interconnect. One of the key features missing from conventional Ethernet is congestion management; this void is being filled by the standardization work of the IEEE 802.1Qau working group. However, the schemes under consideration react to congestion only at the sources by reducing the transmission rates of “hot” flows, i.e., those detected as contributing to congestion. This approach ignores a crucial aspect of many data center networks, namely, that there typically are multiple paths between any pair of end nodes. Before reducing transmission rates, it would make sense to look for an alternative, uncongested path first. Here, we propose an adaptive routing scheme that builds—in a fully transparent way—on top of the existing 802.1Qau schemes, by snooping the congestion notification frames to modify the routing behavior of the switching nodes. We demonstrate how this can lead to significant performance improvements by taking full advantage of path diversity.

Speculative Flow Control for High-Radix Datacenter Interconnect Routers

High-radix switches are desirable building blocks for large computer interconnection networks, because they are more suitable to convert chip I/O bandwidth into low latency and low cost than low-radix switches. Unfortunately, most existing switch architectures do not scale well to a large number of ports. For example, the complexity of the buffered crossbar architecture scales quadratically with the number of ports. Compounded with support for long round-trip times and many virtual channels, the overall buffer requirements limit the feasibility of such switches to modest port counts. Compromising on the buffer sizing leads to a drastic increase in latency and reduction in throughput, as long as traditional credit flow control is employed at the link level. We propose a novel link-level flow control protocol that enables high-performance scalable routers based on the increasingly popular buffered crossbar architecture to scale to higher port counts without sacrificing performance. By combining credited and speculative transmission, this scheme achieves reliable delivery, low latency, and high throughput, even with crosspoint buffers that are significantly smaller than the round-trip time.