Madeleine Glick

ProjecToR: Agile Reconfigurable Data Center Interconnect

We explore a novel, free-space optics based approach for building data center interconnects. It uses a digital micromirror device (DMD) and mirror assembly combination as a transmitter and a photodetector on top of the rack as a receiver (Figure 1). Our approach enables all pairs of racks to establish direct links, and we can reconfigure such links (i.e., connect different rack pairs) within 12 us. To carry traffic from a source to a destination rack, transmitters and receivers in our interconnect can be dynamically linked in millions of ways. We develop topology construction and routing methods to exploit this flexibility, including a flow scheduling algorithm that is a constant factor approximation to the offline optimal solution. Experiments with a small prototype point to the feasibility of our approach. Simulations using realistic data center workloads show that, compared to the conventional folded-Clos interconnect, our approach can improve mean flow completion time by 30-95% and reduce cost by 25-40%.

TCP flow classification and bandwidth aggregation in optically interconnected data center networks

Optical functionality is being used to realize new data center architectures that minimize electronic switching overheads, pushing the processing to the edge of the network. A challenge in optically interconnected data center networks is to identify the large, bandwidth hungry flows (i.e., elephants) and efficiently establish the optical circuits. Moreover, the amount of optical resources to be provisioned during the network planning phase is a critical design problem. Flow classification accuracy affects the efficiency of optical circuits. Optical channel bandwidth, on the other hand, directly relates to the additive-increase, multiplicative-decrease congestion control mechanism of the transmission control protocol and affects the effective bandwidth allocated to elephant flows. In this paper, we simultaneously investigate the impact of two important mechanisms on data center network performance: traffic flow classification accuracy and optical bandwidth aggregation (i.e., the consolidation of several low-capacity channels into a single high-capacity one by employing advanced modulation formats for short-reach communications). We develop a discrete-event simulator for a hybrid data center network, enabling the tuning of flow classification parameters. Our simulations indicate that data center performance is highly sensitive to the aggregation level.We could observe up to a 74.5% improvement in network throughput only due to consolidating the optical channel bandwidth. We further noticed that the role of flow classification becomes more pronounced with higher bandwidth per wavelength as well as with more hot-spot traffic. Compared to a random classification benchmark, adaptive flow classification could lead to throughput improvements as large as 54.7%.

Design Methodology for Optimizing Optical Interconnection Networks in High Performance Systems

Modern high performance computers connect hundreds of thousands of endpoints and employ thousands of switches. This allows for a great deal of freedom in the design of the network topology. At the same time, due to the sheer numbers and complexity involved, it becomes more challenging to easily distinguish between promising and improper designs. With ever increasing line rates and advances in optical interconnects, there is a need for renewed design methodologies that comprehensively capture the requirements and expose trade-offs expeditiously in this complex design space. We introduce a systematic approach, based on Generalized Moore Graphs, allowing one to quickly gauge the ideal level of connectivity required for a given number of end-points and traffic hypothesis, and to collect insight on the role of the switch radix in the topology cost. Based on this approach, we present a methodology for the identification of Pareto-optimal topologies. We apply our method to a practical case with 25,000 nodes and present the results.

All-optical graphical models for probabilistic inference

Considering that high performance electronic computation has become extremely efficient, for an optical hardware accelerator to be relevant, it must solve a type or a set of problems where its electronic counterpart is still struggling in term of size, energy, or time. We have identified one such challenge as the minimization of large scale Ising Hamiltonians when the number of particles is on the order of a million. Here we discuss an algorithmic approach based on probabilistic inference using graphical model and message passing.

Optical implementation of probabilistic graphical models

We are investigating the use of optics to solve highly connected graphical models by probabilistic inference, and more specifically the sum-product message passing algorithm. We are examining the fundamental limit of size and power requirement according to the best multiplexing strategy we have found. For a million nodes, and an alphabet of a hundred, we found that the minimum size for the optical implementation is 10mm3, and the lowest bound for the power is 200 watts for operation at the shot noise limit. The various functions required for the algorithm to be operational are presented and potential implementations are discussed. These include a vector matrix multiplication using spectral hole burning, a logarithm carried out with two photon absorption, an exponential performed with saturable absorption, a normalization executed with an thermo-optics interferometer, and a wavelength remapping accomplished with a pump-probe amplifier.

Scheduling and control in hybrid data centers

We examine the scheduling issues in hybrid electrical/optical data center (DC) networks, considering several implementation requirements. For flexible and programmable resource provisioning, centralized software-defined network (SDN) control can be best complemented with distributed, fast, and accurate flow classification based on machine learning (ML).

Recent advances in optical technologies for data centers: a review

Modern data centers increasingly rely on interconnects for delivering critical communications connectivity among numerous servers, memory, and computation resources. Data center interconnects turned to optical communications almost a decade ago, and the recent acceleration in data center requirements is expected to further drive photonic interconnect technologies deeper into the systems architecture. This review paper analyzes optical technologies that will enable next-generation data center optical interconnects. Recent progress addressing the challenges of terabit/s links and networks at the laser, modulator, photodiode, and switch levels is reported and summarized.

The role of integrated photonics in datacenter networks

Datacenter networks are not only larger but with new applications increasing the east-west traffic and the introduction of the spine leaf architecture there is an urgent need for high bandwidth, low cost, energy efficient interconnects. This paper will discuss the role integrated photonics can have in achieving datacenter requirements. We will review the state of the art and then focus on advances in optical switch fabrics and systems. The optical switch is of particular interest from the integration point of view. Current MEMS and LCOS commercial solutions are relatively large with relatively slow reconfiguration times limiting their use in packet based datacenter networks. This has driven the research and development of more highly integrated silicon photonic switch fabrics, including micro ring, Mach-Zehnder and MEMS device designs each with its own energy, bandwidth and scalability, challenges and trade-offs. Micro rings show promise for their small footprint, however they require an energy efficient means to maintain wavelength and thermal control. Latency requirements have been traditionally less stringent in datacenter networks compared to high performance computing applications, however with the increasing numbers of servers communicating within applications and the growing size of the warehouse datacenter, latency is becoming more critical. Although the transparent optical switch fabric itself has a minimal additional latency, we must also take account of any additional latency of the optically switched architecture. Proposed optically switched architectures will be reviewed.

Optical circuit granularity impact in TCP-dominant hybrid data center networks

Hybrid networking, based on electronic packet switching and optical circuit switching, has been proposed to resolve the existing switching bottlenecks in data centers in an energy-efficient and cost-effective fashion. We consider the problem of resource provisioning in hybrid data centers in terms of optical circuit switching capacity and granularity. The number of fibers connected to server racks, the number of wavelengths per fiber, and the ratio of capacity provided by the optical circuit-switched portion of the network to that of the electronic packet-switched portion are crucial design parameters to be optimized during the data center planning phase. These parameters in conjunction with the additive-increase, multiplicative-decrease (AIMD) congestion control mechanism of the Transmission Control Protocol (TCP) pose a significant impact on data center network performance. In this paper, we examine the combined impact of optical bandwidth settings and TCP dynamics using event-driven simulations. Our analysis reveals the strong dependence of overall network throughput on channel capacity (i.e., the bit rate per wavelength channel) and points to the advantages of optical bandwidth consolidation employing higher-order modulation formats.

Machine learning based adaptive flow classification for optically interconnected data centers

We optimize flow placement for a hybrid network implementing an adaptive neural network classifier. We predict elephant flows with high accuracy on anonymized university network traffic. We also demonstrate the capability to perform highly complex actions at 40 Gbps using less than 5% of co-processor capacity. This shows that it is possible to implement intelligent actions such as a neural network in a data center using fully programmable NICs without handicapping the server CPU.