David J. Delia

System cooling design for the water-cooled IBM Enterprise System/9000 processors

The high operating speed and corresponding high chip heat fluxes in the IBM Enterprise System/9000™ water-cooled mainframe processors are made possible by improvements in component- and system-level cooling. The heart of the closed-loop water-cooling system is a coolant distribution frame (CDF) common to all water-cooled processors. The CDF provides a controlled water temperature of 21.7°C to the central electronic complex (CEC) at water flow rates up to 245 liters per minute (lpm) and rejects heat loads of up to 63 kW for the largest processor. The water flow provides cooling to multichip thermal conduction modules (TCMs), to power supplies, and to air-to-water heat exchangers that provide preconditioned air to channel and memory cards. As many as 121 chips are mounted on a TCM glass-ceramic substrate, with chip powers reaching 27 W or a heat flux of 64 W/cm 2 . A separate cold plate was developed to cool these modules. The power supplies with high heat densities are primarily cooled by water which flows through a unique separable cold plate designed for ease of serviceability of the power supply. Although water cooling is utilized for components with high heat fluxes, air cooling is employed for elements of the system with lower power densities. For cards cooled by forced air, careful trade-off studies among acoustical power, chip reliability, and high availability were required. The acoustic noise emissions of all the fans and blowers were determined, and a system model was constructed to measure the noise radiated from each frame in the system. The data were used to design top covers and other components to ensure that the system could meet its thermal/acoustical requirements. A closed-loop frame in which all the heat was rejected to water was developed to meet these requirements.