Kevin P. Moran

System cooling design for the water-cooled IBM Enterprise System/9000 processors

The high operating speed and corresponding high chip heat fluxes in the IBM Enterprise System/9000™ water-cooled mainframe processors are made possible by improvements in component- and system-level cooling. The heart of the closed-loop water-cooling system is a coolant distribution frame (CDF) common to all water-cooled processors. The CDF provides a controlled water temperature of 21.7°C to the central electronic complex (CEC) at water flow rates up to 245 liters per minute (lpm) and rejects heat loads of up to 63 kW for the largest processor. The water flow provides cooling to multichip thermal conduction modules (TCMs), to power supplies, and to air-to-water heat exchangers that provide preconditioned air to channel and memory cards. As many as 121 chips are mounted on a TCM glass-ceramic substrate, with chip powers reaching 27 W or a heat flux of 64 W/cm 2 . A separate cold plate was developed to cool these modules. The power supplies with high heat densities are primarily cooled by water which flows through a unique separable cold plate designed for ease of serviceability of the power supply. Although water cooling is utilized for components with high heat fluxes, air cooling is employed for elements of the system with lower power densities. For cards cooled by forced air, careful trade-off studies among acoustical power, chip reliability, and high availability were required. The acoustic noise emissions of all the fans and blowers were determined, and a system model was constructed to measure the noise radiated from each frame in the system. The data were used to design top covers and other components to ensure that the system could meet its thermal/acoustical requirements. A closed-loop frame in which all the heat was rejected to water was developed to meet these requirements.

Dual-tapered piston (DTP) module cooling for IBM Enterprise System/9000 system

The water-cooled thermal conduction modules (TCMs) in the IBM Enterprise System/9000™ (ES/9000™) systems require a fourfold thermal improvement over TCMs in the 3090™ system. An examination of the thermal/mechanical tolerance relationships among the chips, substrate, and cooling hardware showed that a cylindrical piston would not meet this requirement. The piston was redesigned with a cylindrical center section and a taper on each end. This shape minimizes the gap between the piston and “hat” while retaining intimate contact between the piston face and chip surface during all assembly conditions. Numerical and analytical models demonstrate that this new piston shape, coupled with improved conductivity of the cooling hardware materials, exceeds ES/9000 system needs. These models were verified by tests conducted on single-site and full-scale modules in the laboratory and by tests on actual ES/9000 systems.