Madhusudan K. Iyengar

Challenges of data center thermal management

The need for more performance from computer equipment in data centers has driven the power consumed to levels that are straining thermal management in the centers. When the computer industry switched from bipolar to CMOS transistors in the early 1990s, low-power CMOS technology was expected to resolve all problems associated with power and heat. However, equipment power consumption with CMOS has been rising at a rapid rate during the past 10 years and has surpassed power consumption from equipment installed with the bipolar technologies 10 to 15 years ago. Data centers are being designed with 15-20-year life spans, and customers must know how to plan for the power and cooling within these data centers. This paper provides an overview of some of the ongoing work to operate within the thermal environment of a data center. Some of the factors that affect the environmental conditions of data-communication (datacom) equipment within a data center are described. Since high-density racks clustered within a data center are of most concern, measurements are presented along with the conditions necessary to meet the datacom equipment environmental requirements. A number of numerical modeling experiments have been performed in order to describe the governing thermo-fluid mechanisms, and an attempt is made to quantify these processes through performance metrics.

Comparative Analysis of Different Data Center Airflow Management Configurations

Increase in computing power resulting from high performance microprocessors, packages, and modules and the deployment of high heat load computer rack units in high density configurations, has escalated the thermal challenges in today’s data center systems. One of the key issues is the location of hot recirculation regions in the room and the mixing of hot rack exhaust air with the cold supply air. Along with many factors such as the rack heat load and the cooling capacity of the supply air, the data center thermal management architecture plays an important role in determining the reliability of the electronic equipment and the general thermal performance of the data center. There are several candidate configurations available for the air ducting designs for data centers. The overall energy efficiency of the system is highly dependant upon the selection of the specific configuration. This paper will summarize the results of a broad numerical study carried out to assess the effectiveness of different data center configurations. The numerical modeling is performed using a commercial computational fluid dynamics (CFD) code based on finite volume approach. The configurations studied include different combinations of raised floor and ceiling supply and return vent location subject to specific constraints. The performance of the data center has been characterized on the basis of average and maximum mean region rack inlet air temperature. Among the seven different configurations compared, the raised floor/ceiling return type configuration is found to be the most effective configuration for the given set of constraints and assumptions.Copyright

Server liquid cooling with chiller-less data center design to enable significant energy savings

This paper summarizes the concept design and hardware build efforts as part of a US Department of Energy cost shared grant, two year project (2010-2012) that was undertaken to develop highly energy efficient, warm liquid cooled servers for use in chiller-less data centers. Significant savings are expected in data center energy, refrigerant and make up water use. The technologies being developed include liquid cooling hardware for high volume servers, advanced thermal interface materials, and dry air heat exchanger (chiller-less with all year “economizer”) based facility level cooling systems that reject the Information Technology (IT) equipment heat load directly to the outside ambient air. Substantial effort has also been devoted towards exploring the use of high volume manufacturable components and cost optimized cooling designs that address high volume market design points. Demonstration hardware for server liquid cooling and data center economizer based cooling has been built and is operational for a 15 kW rack fully populated with liquid cooled servers. This design allows the use of up to 45 °C liquid coolant to the rack. Data collection has commenced to document the system thermal performance and energy usage using sophisticated instrumentation and data collection software methodologies. The anticipated benefits of such energy-centric configurations are significant energy savings at the data center level of as much as 30% and energy-proportional cooling in real time based on IT load and ambient air temperatures. The objective of this project is to reduce the cooling energy to 5% or less of a comparable typical air cooled chiller based total data center energy. Additional energy savings can be realized by reducing the IT power itself through reduced server fan power and potentially less leakage power due to lower device temperatures on average for most locations. This paper focuses on the server liquid cooling, the rack enclosure with heat exchanger cooling and liquid distribution, and the data center level cooling infrastructure. A sample of recently collected energy-efficiency data is also presented to provide experimental validation of the concept demonstrating cooling energy use to be less than 3.5% of the IT power for a hot summer day in New York.

Uncovering energy-efficiency opportunities in data centers

The combination of rapidly increasing energy use of data centers (DCs), which is triggered by dramatic increases in IT (information technology) demands, and increases in energy costs and limited energy supplies has made the energy efficiency of DCs a central concern from both a cost and a sustainability perspective. This paper describes three important technology components that address the energy consumption in DCs. First, we present a mobile measurement technology (MMT) for optimizing the space and energy efficiency of DCs. The technology encompasses the interworking of an advanced metrology technique for rapid data collection at high spatial resolution and measurement-driven modeling techniques, enabling optimal adjustments of a DC environment within a target thermal envelope. Specific example data demonstrating the effectiveness of MMT is shown. Second, the static MMT measurements obtained at high spatial resolution are complemented by and integrated with a real-time sensor network. The requirements and suitable architectures for wired and wireless sensor solutions are discussed. Third, an energy and thermal model analysis for a DC is presented that exploits both the high-spatial-resolution (but static) MMT data and the high-timeresolved (but sparse) sensor data. The combination of these two data types (static and dynamic), in conjunction with innovative modeling techniques, provides the basis for extending the MMT concept toward an interactive energy management solution.

Analytical Modeling for Thermodynamic Characterization of Data Center Cooling Systems

The increasingly ubiquitous nature of computer and internet usage in our society has driven advances in semiconductor technology, server packaging, and cluster level optimizations in the IT industry. Not surprisingly this has an impact on our societal infrastructure with respect to providing the requisite energy to fuel these power hungry machines. Cooling has been found to contribute about a third of the total data center energy consumption and is the focus of this study. In this paper we develop and present physics based models to allow the prediction of the energy consumption and heat transfer phenomenon in a data center. These models allow the estimation of the microprocessor junction and server inlet air temperatures for different flows and temperature conditions at various parts of the data center cooling infrastructure. For the case study example considered, the chiller energy use was the biggest fraction of about 41 % and was also the most inefficient. The room air conditioning was the second largest energy component and was also the second most inefficient. A sensitivity analysis of plant and chiller energy efficiencies with chiller set point temperature and outdoor air conditions is also presented.

Least-energy optimization of air-cooled heat sinks for sustainable development

The development of heat sinks for microelectronic applications, which are compatible with sustainable development, involves the achievement of a subtle balance between a superior thermal design, minimum material consumption, and minimum pumping power. This presentation explores the potential for the least-energy optimization of natural and forced convection cooled rectangular plate heat sinks. The results are evaluated in terms of a heat sink coefficient of performance, relating the cooling capability to the energy invested in the fabrication and operation of the heat sink, and compared to the entropy generation minimization methodology (EGM).

Methods and techniques for measuring and improving data center best practices

Here we present a novel, measurement-based method for characterizing and improving the energy efficiency of a data center (DC). The technique not only yields a clear set of measurement-based best practices metrics, but also provides clear guidance to a DC manager to substantially improve the DC energy efficiency. We describe a technology which exploits the superiority of fast and massive parallel data collection using the Mobile Measurement Technology (MMT) [1] to drive towards quantitative, measurement-driven DC best practices implementation. A large representative raised-floor DC is mapped by the MMT methodology readily yielding the DC layout, very detailed 3D temperature distributions, flows and other relevant physical parameters of the specific facility. The data is used to calculate key metrics (horizontal and vertical hotspots, targeted air flow, plenum temperature, air conditioning unit utilization and flow levels). These metrics provide insights into the sources of energy inefficiencies of the current DC setup, and systematically guide DC managers to improve various best practice aspects in the specific DC. It is shown that significant energy reductions can be achieved utilizing the above described best practices methodology.

Reducing energy usage in data centers through control of Room Air Conditioning units

Information Technology data centers consume a large amount of electricity in the US and world-wide. Cooling has been found to contribute about one third of this energy use. The two primary contributors to the data center cooling energy use are the refrigeration chiller and the Computer Room Air Conditioning units (CRACs). There have been recent changes in specifications for the data center environmental envelopes as mandated by ASHRAE (American Society for Heating Refrigeration and Air Conditioning Engineers), one of which specifically pertains to the upper and lower bound of air temperatures at the inlet to servers that are housed in data center rooms. These changes have been put in place in part to address the desire for greater cooling energy efficiency of these facilities. This paper primarily focuses on the methodologies to reduce the energy usage of room air conditioning devices by exploiting these recent changes in standards for the equipment environmental envelope. A 22000 square foot (6706 m2) data center with 739 kilo Watt of IT load is used as a representative example for numerical CFD analyses using a commercial software package to demonstrate methodologies to reduce the cooling energy use of Information Technology data centers. Several test case simulations are used to enable the calculation of room level air temperature fields for varying design conditions such as different numbers of operational CRACs or the volumetric air flow rate setting of the CRACs. Computation of cooling energy is carried out using available vendor equipment information. The relationship between the reduction in energy usage in CRAC units and the server inlet air temperatures are quantified and a systematic methodology for CRAC shut off is proposed. The relative magnitude of reduction in data center cooling energy use from shutting off CRACs or reducing CRAC motor speeds is also compared with scenarios involving increases in refrigeration chiller plant water temperature set point.

Experimental and computational study of perforated floor tile in data centers

Current CFD simulation studies of large data centers cannot model the detailed geometries of the perforated tiles due to grid size limitation. These studies often assume that the tile flow can be modeled as constant velocity based on a fully open tile. In this case, mass flux is enforced at the expense of under-predicting momentum flux; the error in momentum flux can be as high as a factor of four for a 25% open perforated tile. Since jet entrainment is a strong function of its initial momentum flux, this error can be significant with respect to predicting the mixing of the surrounding room air into the tile flow. Combined experimental and computational studies were carried out to quantify the importance of the detailed tile geometry, and it was found that proper prediction of the mixing process must account for the tile opening patterns. Suggestions of how to model the floor perforated tiles in data center CFD simulations are then presented.

Least-material optimization of vertical pin-fin, plate-fin, and triangular-fin heat sinks in natural convective heat transfer

A least-material optimization of pin-fin, plate-fin, and triangular-fin array geometries is performed, by extending the use of least-material single fin analysis to multiple fin arrays. The heat dissipation from vertical fin arrays in natural convection is calculated using the Nusselt number correlation by Aihara et al. (Int. J. Heat and Mass Transfer vol. 33, no. 6, pp. 1223-32, 1990) for pin-fins, by Bar-Cohen and Rohsenow (Trans IEEE CHMT vol. 6, pp. 154-8, 1983) for rectangular plate fins, and by Karagiozis et al (Air, vol. 116, pp. 105-11, 1994) for triangular plate fins. Comparisons of the thermal capability of the three different array geometries are carried out on the basis of total heat dissipation and material-specific volumetric heat dissipation. Manufacturability constraints of these heat sinks, and their effects on the final design, are briefly discussed.