David M. Calhoun

Silicon Photonics for Exascale Systems

With the extraordinary growth in parallelism at all system scales driven by multicore architectures, computing performance is increasingly determined by how efficiently high-bandwidth data is communicated among the numerous compute resources. High-performance systems are especially challenged by the growing energy costs dominated by data movement. As future computing systems aim to realize the Exascale regime-surpassing 1018 operations per second-achieving energy efficient high-bandwidth communication becomes paramount to scaled performance. Silicon photonics offers the possibility of delivering the needed communication bandwidths to match the growing computing powers of these highly parallel architectures with extremely scalable energy efficiency. However, the insertion of photonic interconnects is not a one-for-one replacement. The lack of practical buffering and the fundamental circuit switched nature of optical data communications require a holistic approach to designing system-wide photonic interconnection networks. New network architectures are required and must include arbitration strategies that incorporate the characteristics of the optical physical layer. This paper reviews the recent progresses in silicon photonic based interconnect devices along with the system level requirements for Exascale. We present a co-design approach for building silicon photonic interconnection networks that leverages the unique optical data movement capabilities and offers a path toward realizing future Exascale systems.

Scaling silicon photonic switch fabrics for data center interconnection networks

With the rapidly increasing aggregate bandwidth requirements of data centers there is a growing interest in the insertion of optically interconnected networks with high-radix transparent optical switch fabrics. Silicon photonics is a particularly promising and applicable technology due to its small footprint, CMOS compatibility, high bandwidth density, and the potential for nanosecond scale dynamic connectivity. In this paper we analyze the feasibility of building silicon photonic microring based switch fabrics for data center scale optical interconnection networks. We evaluate the scalability of a microring based switch fabric for WDM signals. Critical parameters including crosstalk, insertion loss and switching speed are analyzed, and their sensitivity with respect to device parameters is examined. We show that optimization of physical layer parameters can reduce crosstalk and increase switch fabric scalability. Our analysis indicates that with current state-of-the-art devices, a high radix 128 × 128 silicon photonic single chip switch fabric with tolerable power penalty is feasible. The applicability of silicon photonic microrings for data center switching is further supported via review of microring operations and control demonstrations. The challenges and opportunities for this technology platform are discussed.

Single Microring-Based

Realizing small-footprint and energy-efficient optical switching fabrics is of crucial importance to solve the data movement challenges faced by optical interconnection networks. This letter demonstrates silicon photonic 2 × 2 full crossbar switching functionality based on a single microring. The ultracompact device is shown to successfully switch data channels from two input ports simultaneously. Data channels in both the multiple and the same wavelength switching experiments are measured to be error-free. Simulation shows that by optimizing some of the microring parameters crosstalk could be reduced. This letter confirms the applicability of a single microring as a 2 × 2 switch element for on-chip optical interconnects.

2\times 2

Integrated photonics offers the possibility of compact, low energy, bandwidth-dense interconnects for large port count spatial optical switches, facilitating flexible and energy efficient data movement in future data communications systems. To achieve widespread adoption, intimate integration with electronics has to be possible, requiring switch design using standard microelectronic foundry processes and available devices. We report on the feasibility of a switch fabric comprised of ubiquitous silicon photonic building blocks, opening the possibility to combine technologies, and materials towards a new path for switch fabric design. Rather than focus on integrating all devices on a single silicon chip die to achieve large port count optical switching, this work shifts the focus towards innovative packaging and integration schemes. In this work, we demonstrate 1×8 and 8×1 microring-based silicon photonic switch building blocks with software control, providing the feasibility of a full 8×8 architecture composed of silicon photonic building blocks. The proposed switch is fully non-blocking, has path-independent insertion loss, low crosstalk, and is straightforward to control. We further analyze this architecture and compare it with other common switching architectures for varying underlying technologies and radices, showing that the proposed architecture favorably scales to very large port counts when considering both crosstalk and architectural footprint. Separating a switch fabric into functional building blocks via multiple photonic integrated circuits offers the advantage of piece-wise manufacturing, packaging, and assembly, potentially reducing the number of optical I/O and electrical contacts on a single die.

Silicon Photonic Crossbar Switches

Face-to-face comparison of major large scale HPC interconnects.Review of challenges and solutions for increased use of optics in HPC.Description of optical switching, in terms of principle, limitations, technical requirements and benefits. Large-scale high performance computing is permeating nearly every corner of modern applications spanning from scientific research and business operations, to medical diagnostics, and national security. All these communities rely on computer systems to process vast volumes of data quickly and efficiently, yet progress toward increased computing power has experienced a slowdown in the last number of years. The sheer cost and scale, stemming from the need for extreme parallelism, are among the reasons behind this stall. In particular, very large-scale, ultra-high bandwidth interconnects, essential for maintaining computation performance, represent an increasing portion of the total cost budget.Photonic systems are often cited as ways to break through the energy-bandwidth limitations of conventional electrical wires toward drastically improving interconnect performance. This paper presents an overview of the challenges associated with large-scale interconnects, and reviews how photonic technologies can contribute to addressing these challenges. We review some important aspects of photonics that should not be underestimated in order to truly reap the benefits of cost and power reduction.

Modular architecture for fully non-blocking silicon photonic switch fabric

We review the requirements and expectations of future Supercomputer architectures, analyse how transparent optical networking might contribute to these future systems scalable performance, and develop the most immediate challenges for optical system developers in this context.

Optical interconnects for extreme scale computing systems

A scalable and flexible memory interconnect is a key component for a many-core architecture to take full advantage of the high-bandwidth of multiple memory stacks. In this paper, we discuss both technological and architectural challenges of these processor-to-memory interconnects, and focus on two important issues of many-core memory accesses: traffic hotspots and non-uniform memory access (NUMA). We propose a reconfigurable Silicon photonic memory interconnect based on 2.5D stacking that can direct memory traffic to any memory interface on the processor, thus alleviating the two aforementioned effects in addition to providing high bandwidth. Simulations based on a 16-core 4-memory model show that the proposed architecture can lead to up to 2× STREAM speedup over fixed connections in both hotspot and NUMA scenarios. We also demonstrate the proposed architecture using a four-port Silicon photonic demultiplexer and a 4×4 synthesizable on-chip fabric called OpenSoC. The FPGA-emulated system demonstrates dynamic memory rewiring through wavelength routing, and achieves a reconfiguration time of 5 microseconds.

Toward transparent optical networking in exascale computers

A wealth of high-bandwidth and energy-efficient silicon photonic devices have been demonstrated in recent years. These represent promising solutions for high-performance computer systems that need to distribute extremely large amounts of data in an energy-efficient manner. Chip-scale optical interconnects that employ novel silicon photonics devices can potentially leapfrog the performance of traditional electronic-interconnected systems. However, the benefits of silicon photonics at a system level have yet to be realized. This chapter reviews methodologies for integrating silicon photonic interconnect technologies with computing systems, including implementation challenges associated with device characteristics. A fully functional co-integrated hardware–software system needs to encompass device functionality, control schema, and software logic seamlessly. Each layer, ranging from individual device characterization, to higher layer control of multiple devices, to arbitration of networks of devices, and ultimately to encapsulation of subsystems to create the entire computing system is explored. Finally, results and implications at each level of the system stack are presented.

Silicon photonic memory interconnect for many-core architectures

We present an optical programmable network architecture to enable agile and efficient iterative multicasting for cluster computing framework. Support for multiple multicast groups and dynamic group reassignment are experimentally demonstrated.

Hardware–Software Integrated Silicon Photonics for Computing Systems

We present an architecture towards accelerating compute element operations on a fully arbitrated silicon photonic (SiP) system. An 8×8 SiP network is controlled in a distributed fashion, with connectivity consisting of two 10Gbps wavelength division multiplexed data links that are arbitrated out-of-band.