Robert E. Simons

Review of cooling technologies for computer products

This paper provides a broad review of the cooling technologies for computer products from desktop computers to large servers. For many years cooling technology has played a key role in enabling and facilitating the packaging and performance improvements in each new generation of computers. The role of internal and external thermal resistance in module level cooling is discussed in terms of heat removal from chips and module and examples are cited. The use of air-cooled heat sinks and liquid-cooled cold plates to improve module cooling is addressed. Immersion cooling as a scheme to accommodate high heat flux at the chip level is also discussed. Cooling at the system level is discussed in terms of air, hybrid, liquid, and refrigeration-cooled systems. The growing problem of data center thermal management is also considered. The paper concludes with a discussion of future challenges related to computer cooling technology.

Server liquid cooling with chiller-less data center design to enable significant energy savings

This paper summarizes the concept design and hardware build efforts as part of a US Department of Energy cost shared grant, two year project (2010-2012) that was undertaken to develop highly energy efficient, warm liquid cooled servers for use in chiller-less data centers. Significant savings are expected in data center energy, refrigerant and make up water use. The technologies being developed include liquid cooling hardware for high volume servers, advanced thermal interface materials, and dry air heat exchanger (chiller-less with all year “economizer”) based facility level cooling systems that reject the Information Technology (IT) equipment heat load directly to the outside ambient air. Substantial effort has also been devoted towards exploring the use of high volume manufacturable components and cost optimized cooling designs that address high volume market design points. Demonstration hardware for server liquid cooling and data center economizer based cooling has been built and is operational for a 15 kW rack fully populated with liquid cooled servers. This design allows the use of up to 45 °C liquid coolant to the rack. Data collection has commenced to document the system thermal performance and energy usage using sophisticated instrumentation and data collection software methodologies. The anticipated benefits of such energy-centric configurations are significant energy savings at the data center level of as much as 30% and energy-proportional cooling in real time based on IT load and ambient air temperatures. The objective of this project is to reduce the cooling energy to 5% or less of a comparable typical air cooled chiller based total data center energy. Additional energy savings can be realized by reducing the IT power itself through reduced server fan power and potentially less leakage power due to lower device temperatures on average for most locations. This paper focuses on the server liquid cooling, the rack enclosure with heat exchanger cooling and liquid distribution, and the data center level cooling infrastructure. A sample of recently collected energy-efficiency data is also presented to provide experimental validation of the concept demonstrating cooling energy use to be less than 3.5% of the IT power for a hot summer day in New York.

The evolution of IBM high performance cooling technology

This paper provides a perspective and review of the evolution of high performance cooling technology that has been developed and used in IBM medium and large-scale computers over the past 25 years. Package cooling technology and its evolution, leading to the development of the Thermal Conduction Module (TCM) is described. The development of air cooling technology is discussed; along with enhancements using turbulators, air-to-liquid heat exchangers, and impinging flow. The development of water cooling and direct liquid immersion technology is also covered.

The evolution of water cooling for IBM large server systems: Back to the future

This paper provides a technical perspective and review of water cooling technology as implemented through 5 generations of IBMs high performance computing systems from the S360/91 to the recently announced IBM Power 575 supercomputing system. The use of hybrid air-to-water cooling and then indirect (cold plate) water cooling in earlier IBM systems is described. Attention is given to how and why water-cooling was implemented to provide the required cooling capability while maintaining ease of serviceability at the module level. Also discussed is the use of a Cooling Distribution Unit (CDU) to control cooling system water temperature, distribute water to multiple racks and serve as a buffer between system water and customer facility water. Rising microprocessor power dissipation, increased heat loads at the data center level, and demands for increased cooling energy efficiency are presented as driving the need for the reintroduction of water cooling. The introduction of the rear door heat exchanger to respond to the challenge of rising heat loads at the data center level is discussed. Finally, the IBM Power 575 water cooling system is described. Included in the discussion are the coolant flow architecture and the incorporation of Modular Water cooling Units (MWUs) within the server frame replacing the remote CDU concept.

Jet impingement boiling of a dielectric coolant in narrow gaps

An experimental investigation into the effect of the chip-to-orifice gap was conducted for jet impingement boiling of FC-72 at the back surface of a 6.5 mm square thermal chip on a 28 mm square ceramic substrate. Four different types of jets were used, all of them employing a single 0.50 mm diameter orifice. A fixed position jet impingement pin with the orifice centered on a flat face measuring 7.62 mm on an edge was used as a base case. This jet supported a heat flux of about 120 W/cm/sup 2/ at a reasonable flowrate. Variable position jet impingement pistons were shown to perform as well as, but no better than the base case. Additional tests were conducted to investigate performance with variation in the chip-to-orifice gap. Thermal performance was found to be insensitive to the gap at large spacing, but below some specific gap it degraded with decreasing gap. A sudden jump in the chip temperature was discovered to occur at a specific gap. This gap length was found to vary with jet flowrate.<<ETX>>

Experimental characterization of an energy efficient chiller-less data center test facility with warm water cooled servers

Typical data centers utilize approximately 50% of the total IT energy in cooling of the server racks. We present a chillerless data center where server-level cooling is achieved through a combination of warm water cooling hardware and re-circulated air; eventual heat rejection to ambient air is achieved using a closed secondary liquid loop to ambient-air heat exchanger (dry-cooler). Several experiments were carried out to characterize the individual pieces of equipment and data center thermal performance and energy consumption. A 22+ hour experimental run was also carried out with results indicating an average cooling energy use of 3.5% of the total IT energy use, with average ambient air temperatures of 23.8°C and average IT power use of 13.14 kW.

Experimental investigation of an enhanced thermosyphon heat loop for cooling of a high performance electronics module

This paper discusses the investigation of module cooling utilizing an enhanced thermosyphon heat loop as an alternative to direct air cooling or liquid-to-air cooling with forced convection of the liquid. Using water as the working fluid in the thermosyphon, experiments were conducted to investigate the effects of fill volume, heat load, and condenser air flow rate on overall thermosyphon performance in terms of thermal resistance. Enhancement of evaporator performance using fins was also investigated and the results are reported in the paper.

Impact of operating conditions on a chiller-less data center test facility with liquid cooled servers

Data center cooling can constitute a significant portion of the total data center energy usage, with typical cooling energy expenditures approximately 50% of the IT energy use. Much of this energy consumption occurs at the refrigeration/chiller plant and in the Computer Room Air Handlers (CRAH) that cool and condition the air used to cool the electronics racks. To reduce cooling energy use, a data center test facility was designed and constructed to reduce cooling energy use to less than 5% of the total IT energy use through a combination of warm water cooling of the electronics and liquid-side economization. Several data center operating conditions, such as changes in liquid and air flow rates, heat exchanger arrangements and addition of propylene glycol were investigated to determine their impact on the energy consumption and thermal performance of the key cooling equipment. Day long runs collected from summer and fall days are also reported to illustrate the impact of external weather conditions and loop operating conditions on the thermal performance and energy consumption of the dual-loop data center test facility. The work presented highlights the impact of various operating conditions in influencing the cooling energy use and improving data center energy efficiency in chiller-less, ambient-air cooled data center designs using water cooled servers.

Microelectronics cooling and SEMI-THERM: a look back

For the occasion of the 10th anniversary of the SEMI-THERM conference, this paper provides a look back at some of the developments that have taken place since its founding. Topics covered include thermal measurement, thermal characterization, thermal analysis and modeling, air cooling, water cooling, and immersion cooling.<<ETX>>

Extreme energy efficiency using water cooled servers inside a chiller-less data center

The paper summarizes part of a project that was undertaken to develop highly energy efficient warm liquid cooled servers for use in chiller-less data centers that could save significant data center energy use and reduce data center refrigerant and make up water usage. One of the key concepts developed as part of this project is the Dual-Enclosure-Liquid-Cooling (DELC), which comprises of a 100% liquid cooled server cabinet and an outdoor dry cooler unit for heat rejection to the ambient and this configuration is used to reject the Information Technology (IT) equipment heat load directly to the outside ambient air without the use of a chiller. Demonstration hardware for server liquid cooling and a chiller-less data center was built and is operational for a 15 kW rack fully populated with liquid cooled servers which has been designed for use for up to 45°C liquid coolant to the rack. The anticipated benefits of such energy-centric configurations are significant energy savings of as much as 25% at the data center level. This paper builds on recent work that focused on the server liquid cooling, the rack enclosure with heat exchanger cooling and liquid distribution, and the data center level cooling infrastructure and which also presented sample data from experiments in support of the DELC concept. This paper presents experimental data related to the novel data center loop in a new manner, which is used to create a simplified thermodynamic model using curve-fit of surfaces of heat exchanger approach temperatures and power use of cooling devices. The model is validated using experimental data for a 22 hour test that was conducted in August of 2011. Subsequent to model validation, the simplified model is then used to make projections for DELC prototype performance (thermal and energy) under different conditions including different simple control schemes and weather conditions in the US. Weather data from nine different US cities is analyzed for a single day in August and realizable energy and energy cost savings over traditional chiller based data center cooling designs are presented. The results show that the new innovative data center cooling configuration presented could reduce cooling energy use to be less than 3.5% of the IT power for most US locations even in warm summer times of the year.