Manoj Wadekar

Enhanced Ethernet for Data Center: Reliable, Channelized and Robust

Data Centers today deploy multiple fabrics for LAN, SAN and IPC traffic. This introduces significant cost, complexity and management challenges. Hence there is strong demand for consolidating these multiple fabrics. Ethernet is default fabric used primarily for LAN traffic and it is low-cost, well-known, widely available, and broadly compatible. So, it makes an ideal candidate for such IO consolidation. However, current Ethernet suffers from certain deficiencies that need to be addressed. Legacy Ethernet suffers from 3 main issues (a) Lack of congestion management mechanism -which results in packet drops when network is congested. This also results in unpredictable latencies for the traffic through these congested networks, (b) No standard mechanism for traffic differentiation on the link which will protect all the traffic types on consolidated link. Only mechanism available is 802. lp strict prioritization without any bandwidth assurance, (c) Poor utilization of available bandwidth in the network. Spanning Tree Protocol tends to remove links that can cause loops in the network. This results in lower bisectional bandwidth. Underlying Layer 2 protocols do not allow any multipathing techniques that are available in other technologies like Fiber Channel, Infiniband etc. This paper presents a framework of solution that includes multiple enhancements to Ethernet network -that will improve legacy Ethernet significantly for Data Center deployment. Some of these enhancements are being pursued in IEEE and IETF for standardization. IEEE 802.1Qau Task Force is focusing on improving congestion management capabilities of Ethernet IEEE 802.1Qat and IETF-TRILL is focusing on improving bisectional bandwidth by allowing L2 multipathing capabilities for complex and large DC deployments. Other efforts include mechanisms for better traffic differentiation and improved link level flow control for data center -Ethernet. This paper will elaborate more on congestion management related work.

An architecture for congestion management in Ethernet clusters

Interconnects for clusters and bladed systems must deliver efficient throughput, low latency, low delay variations and minimal frame drops. The primary technical issues hindering Ethernet adoption for cluster and blade system interconnects are the current methods Ethernet switches use for dealing with congestion, which can happen frequently under cluster and blade system workloads. The common response to congestion is to drop frames and the common method of avoiding the need to drop frames is to utilize very large switch buffers. In this paper, we propose a three-level approach to dealing with congestion that provides efficient throughput, low latency, low delay variations, and can eliminate frame drops, even with very modest sized switch buffers. The approach employs three levels of congestion management: 1) improved link level transient congestion control; 2) oversubscription control at layer 2 subnet ingresses, and 3) end-to-end oversubscription control by the higher layer protocols. We present compelling simulation results showing the incremental benefits provided by each level.