Richard Robert Grzybowski

Optical-packet-switched interconnect for supercomputer applications [Invited]

Feature Issue on Optical Interconnection Networks (OIN). We describe a low-latency, high-throughput scalable optical interconnect switch for high-performance computer systems that features a broadcast-and-select architecture based on wavelength- and space-division multiplexing. Its electronic control architecture is optimized for low latency and high use. Our demonstration system will support 64 nodes with a line rate of 40 Gbit/s per node and operate on fixed-length packets with a duration of 51.2 ns using burst-mode receivers. We address the key system-level requirements and challenges for such applications.

Designing a Crossbar Scheduler for HPC Applications

A crucial part of any high-performance computing (HPC) system is its interconnection network. Corning and IBM are jointly developing a demonstration interconnect based on optical cell switching with electronic control. The Corning-IBM joint optical shared memory supercomputer interconnect system (Osmosis) project explores the opportunity to advance the role of optical-switching technologies in such systems. Key innovations in the scheduler architecture directly address the main HPC requirements: low latency, high throughput, efficient multicast support, and high reliability

Optical interconnection networks: The OSMOSIS project

OSMOSIS is an optical packet switching interconnection network for high-performance computing systems. It aims at delivering sustained high bandwidth, very low latency, and cost-effective scalability. We describe its system and control architecture.

Viable opto-electronic HPC interconnect fabrics

We address the problem of how to exploit optics for ultrascale High Performance Computing interconnect fabrics. We show that for high port counts these fabrics require multistage topologies regardless of whether electronic or optical switch components are used. Also, per stage electronic buffers remain indispensable for maintaining throughput, lossless-ness and packet sequence. Although the notion of true all-optical packet switching is not yet viable, we show that appropriate use of optical switching technology offers power and scaling advantages that can be leveraged economically, and propose a hybrid opto-electronic HPC interconnect fabric architecture that combines the strength of electronics in processing and storing information with the strength of optics in switching and transporting high bandwidths. Using Semiconductor Optical Amplifier technology, we are building a prototype demonstrator switch that we believe solves all the technical challenges. Having reached this threshold now enables commercialization of this technology, which we are currently pursuing.

The OSMOSIS Optical Packet Switch for Supercomputers: Enabling Technologies and Measured Performance

The OSMOSIS project explores the role of optics in large-scale interconnection networks for high-performance computing (HPC) systems. Its main objectives are solving the technical challenges to meet the stringent HPC requirements of high bandwidth, low latency, low error rates, and cost-effective scalability. We discuss the technologies and architectural innovations that enabled us to build a demonstration system meeting these targets. We demonstrate the optical performance for the 64 ports @ 40 Gb/s data paths across the semiconductor optical amplifier based optical crossbar, and report on the implementation of the electronic central controller.

Extraordinary laser-induced swelling of oxide glasses

We describe a novel process of laser-assisted fabrication of surface structures on doped oxide glasses with heights reaching 10 - 13% of the glass thickness. This effect manifests itself as a swelling of the irradiated portion of the glass, which occurs in a wide range of glass compositions. The extent of such swelling depends on the glass base composition. Doping with Fe, Ti, Co, Ce, and other transition metals allows for adjusting the absorption of the glass and maximizing the feature size. In the case of bumps grown on borosilicate glasses, we observe reversible glass swelling and the bump height can increase or decrease depending on whether the consecutive laser pulse has higher or lower energy compared with the previous one. The hypothetical mechanism includes laser heating of glass, glass melting, and directional flow. We review several potential applications of such glass swelling.

Optimization of a supercomputer optical interconnect architecture

We present a novel, optimal architecture for a NxN supercomputer optical interconnect, composed of arrayed waveguide grating multiplexers/demultiplexers and a minimum total number, of the order of N In N, of semiconductor optical amplifiers used as on-off gates.

Laser texturing of doped borosilicate glasses

We describe a novel process of laser-assisted fabrication of surface structures on doped oxide glasses with heights reaching 10 - 13% of the glass thickness. This effect manifests itself as a swelling of the irradiated portion of the glass, and occurs in a wide range of glass compositions. The extent of such swelling depends on the glass base composition. Doping with Fe, Ti, Co, Ce, and other transition metals allows for adjusting the absorption of the glass and maximizing the feature size. In the case of bumps grown on borosilicate glasses, we observe reversible glass swelling and the bump height can increase or decrease depending on whether the consecutive laser pulse has higher or lower energy compared with the previous one. To understand the hypothetical mechanism, which includes laser heating of glass, glass melting, and directional flow, we explored density, refractive index, fictive temperature, and phase separation dynamics.

Influence of Transmission Impairments on the OSMOSIS HPC Optical Interconnect Architecture

We examine the impact of transmission impairments on the performance of the optical supercomputer interconnect architecture, initially proposed in the context of the optical shared memory supercomputer interconnect system (OSMOSIS) project. We study two versions of the aforementioned optical interconnect that differ in terms of the number of semiconductor optical amplifiers (SOAs) used as ON-OFF gates. For practical reasons related to packet arbitration, the size of the crossbar switch of the optical interconnect in this study is limited to 64 ports. The switch is based on a broadcast-and-select architecture and employs DWDM in conjunction with 10 Gb/s intensity modulation/direct detection per wavelength channel. We show, both by experiment and by simulation, that the minimization of the number of SOAs in the optical switch by taking advantage of the cyclic routing capability of optical arrayed waveguide multiplexers/demultiplexers leads to negligible performance deterioration compared to conventional wavelength-space switches that are prohibitive slower and do not use any inherent gain properties like in OSMOSIS.

A scaleable optical interconnect for low-latency cell switching in high- performance computing systems

An optical cell switch for interconnecting massively parallel nodes offers the potential for reduction in size, power consumption, and cost of high-performance computing (HPC) interconnects. We designed an architecture based on a broadcast and select approach that is highly flexible in terms of supported ports and can easily scale from a 16x16 to a 2048x2048 port switch by exploiting both wavelength multiplexing and fiber multiplexing. The optical system is designed for 40 Gb/s operation, but the full 160 Gb/s switching likely required of a commercial system can be supported by this architecture. At the core of the switch is an array of semiconductor optical amplifiers (SOAs), which provide fast switching (~1 ns), high extinction ratio (>40 dB) for cross-talk reduction, and optical gain (15 dB typical). The full optical switch consists of a 2-stage broadcast and select design for fiber select and color select, leading to a bufferless low-latency crossbar cell switch. A switch system demonstrator with 8 full optical paths has been implemented and used for performance characterization. A fast 40 Gb/s cell receiver was developed and proven to support up to 9 dB dynamic range. System demonstration measurements have shown that a raw BER of 10-15 is achievable. Optical cross-talk is negligible and does not degrade the system performance. The system design and verification experiments demonstrate that a scaleable 40 Gb/s switch for massively parallel systems is feasible and offers the potential for significant size, power consumption, and cost reduction by applying the scalability of an optical solution to a HPC system.