Moray McLaren

HyperX: topology, routing, and packaging of efficient large-scale networks

In the push to achieve exascale performance, systems will grow to over 100,000 sockets, as growing cores-per-socket and improved single-core performance provide only part of the speedup needed. These systems will need affordable interconnect structures that scale to this level. To meet the need, we consider an extension of the hypercube and flattened butterfly topologies, the HyperX, and give an adaptive routing algorithm, DAL. HyperX takes advantage of high-radix switch components that integrated photonics will make available. Our main contributions include a formal descriptive framework, enabling a search method that finds optimal HyperX configurations; DAL; and a low cost packaging strategy for an exascale HyperX. Simulations show that HyperX can provide performance as good as a folded Clos, with fewer switches. We also describe a HyperX packaging scheme that reduces system cost. Our analysis of efficiency, performance, and packaging demonstrates that the HyperX is a strong competitor for exascale networks.

A Nanophotonic Interconnect for High-Performance Many-Core Computation

Silicon nanophotonics holds the promise of revolutionizing computing by enabling parallel architectures that combine unprecedented performance and ease of use with affordable power consumption. Here we describe the results of a detailed multiyear design study of dense wavelength division multiplexing (DWDM) on-chip and off-chip interconnects and the device technologies that could improve computing performance by a factor of 20 above industry projections over the next decade.

The role of optics in future high radix switch design

For large-scale networks, high-radix switches reduce hop and switch count, which decreases latency and power. The ITRS projections for signal-pin count and per-pin bandwidth are nearly flat over the next decade, so increased radix in electronic switches will come at the cost of less per-port bandwidth. Silicon nanophotonic technology provides a long-term solution to this problem. We first compare the use of photonic I/O against an all-electrical, Cray YARC inspired baseline. We compare the power and performance of switches of radix 64, 100, and 144 in the 45, 32, and 22 nm technology steps. In addition with the greater off-chip bandwidth enabled by photonics, the high power of electrical components inside the switch becomes a problem beyond radix 64. We propose an optical switch architecture that exploits high-speed optical interconnects to build a flat crossbar with multiple-writer, single-reader links. Unlike YARC, which uses small buffers at various stages, the proposed design buffers only at input and output ports. This simplifies the design and enables large buffers, capable of handling ethernet-size packets. To mitigate head-of-line blocking and maximize switch throughput, we use an arbitration scheme that allows each port to make eight requests and use two grants. The bandwidth of the optical crossbar is also doubled to to provide a 2x internal speedup. Since optical interconnects have high static power, we show that it is critical to balance the use of optical and electrical components to get the best energy efficiency. Overall, the adoption of photonic I/O allows 100,000 port networks to be constructed with less than one third the power of equivalent all-electronic networks. A further 50% reduction in power can be achieved by using photonics within the switch components. Our best optical design performs similarly to YARC for small packets while consuming less than half the power, and handles 80% more load for large message traffic.

QSNET/sup II/: defining high-performance network design

QSNET/sup II/ optimizes interprocessor communication in systems built from standard server building blocks. Its short-message processing unit permits fast injection of small messages, providing ultra-low latency and scalability to thousands of nodes. Thus, in a sense, the high-performance network in a cluster computer is the computer because it largely defines achievable performance, widening the range of the applications a cluster can efficiently execute, as well as defining its scalability, fault tolerance, system software, and overall usability.

Photonic Architectures for High-Performance Data Centers

Over the next decade, significant progress must be made in research on computer architectures that enable unprecedented improvements in the efficiency of large-scale computing systems, particularly to support applications that require exascale algorithmic performance. Here, we review the performance requirements for both high-performance computing systems and data centers, and show that it will be critical to exploit photonic devices for interconnect applications to meet these expectations. In the long term, CMOS-compatible fabrication technologies promise a “Moores Law for photonics” that could completely change the economics of integrated optics and high-performance computing for defense, security, scientific, and consumer applications.

A High-Speed Optical Multi-Drop Bus for Computer Interconnections

Buses have historically provided a flexible communications structure in computer systems. However, signal integrity constraints of high-speed electronics have made multi-drop electrical busses infeasible. Instead, we propose an optical data bus for computer interconnections. It has two sets of optical waveguides, one as a fan-out and the other as a fan-in, that are used to interconnect different modules attached to the bus. A master module transmits optical signals which are received by all the slave modules attached to the bus. Each slave module in turn sends data back on the bus to the master module. Arrays of lasers, photodetectors, waveguides, microlenses, beamsplitters and Tx/Rx integrated circuits are used to realize the optical data bus. With 1 mW of laser power, we are able to interconnect 8 different modules at 10 Gb/s per channel. An aggregate bandwidth of over 25 GB/s is achievable with 10 bit wide signaling paths.

A High-Speed Optical Multidrop Bus for Computer Interconnections

Signal integrity constraints of high-speed electronics have made multidrop electrical buses infeasible. This high-speed alternative uses hollow metal waveguides and pellicle beam splitters that interconnect modules attached to the bus. With 1 mw of laser power, the bus can interconnect eight modules at 10 gbps per channel and achieves an aggregate bandwidth of more than 25 gbytes per second with 10-bit-wide signaling paths.

Optical high radix switch design

Networking consumes up to 33 percent of modern data center power. Network switches are the key source of inefficiency: a switch traversal costs an order of magnitude more than a link traversal. The authors propose a new high-radix switch architecture that uses emerging integrated optical interconnect technology to reduce switch power. They tailor every component of a switch to best exploit optical technology and improve switch scalability and energy efficiency.

Low cost, injection molded 120 Gbps optical backplane

A low cost, blind mate, injection molded optical backplane is presented. The optical backplane consists of a 12 channel optical broadcast bus operating at 10 Gbps/channel with six blind mate optical output ports spaced 1U apart.

A nanophotonic interconnect for high-performance many-core computation

We describe the results of a design study of DWDM on-chip and off-chip nanophotonic interconnects and device technologies that could improve computing performance by a factor of 20 above industry projections over the next decade.