Peter W. Kelly

An overview of the IBM zEnterprise EC12 processor cooling system

On September 19, 2012 IBM announced its latest System z Enterprise Class zServer, the IBM zEnterprise EC12 (zEC12). This server uses a 96 mm glass ceramic substrate to interconnect processors and related cache chips on a multi-chip module (MCM). In rare applications, the power in these MCMs can exceed 2000W, well beyond air cooling capability. This paper describes a new cooling methodology IBM employs in zEC12 to cool its processor MCMs. From the IBM S/390 G4, which first shipped in 1997, through z196 which is EC12s enterprise class predecessor, IBMs high end System z servers have utilized vapor compression refrigeration to cool its processor MCMs. In zEC12, the thermal solution employs an air to water heat exchanger to provide this function. This paper discusses the technical details of this cooling system. Thermal performance of each component of the cooling path from processor core to ambient, as well as comparison to prior cooling approaches in terms of temperatures, reliability, and energy efficiency will be reviewed. In summary, this technology shows considerable promise for cooling this class of server.

Thermal-mechanical Co-design of Cold Plate, Second Level Thermal Interface Material (TIM2) and Heat Spreaders for Optimal Thermal Performance for High-end Processor Cooling

Cooling high-end system processors has become increasingly more challenging due to the increase in both total power and peak power density in processor cores. Junction peak temperature at worst case corner conditions often establish the limits on the maximum supportable circuit speed as well as processor chip yield. While significant progress has been made in cooling technology (e.g., cold plate design and thermal interface materials at first and the second level package), a systematic approach is needed to optimize the entire thermal and mechanical stack to achieve the overall (optimal) thermal performance objectives. The necessity and importance of this is due to the thermal and mechanical design interdependencies contained with the overall stack. This paper reports an in-depth study of the thermal-mechanical interactions associated with the cold plate, second level thermal interface material (TIM2) and heat spreaders. Thermal test results are reported for different cold plate designs and TIM2 pad sizes. Thermal and mechanical modeling results are provided to quantify the TIM2 thermal performance as a function of the TIM2 mechanical stress, the TIM2 dimensions and cold plate design. As described via both modeling and testing results, an optimal TIM2 pad size results as a trade-off between heat transfer area for conduction and TIM2 compressive pressure. In addition, pressure sensitive film study results are also provided revealing that heat spreader design affects the module level and TIM2 thermal performance. Results from this set of work clearly demonstrate the close interactions between cooling hardware in the stack hence the importance of thermal-mechanical co-design to achieve optimal thermal performance for the high-end processors.

Modeling blower flow characteristics and comparing to measurements

Thermal management for high performance electronic systems such as servers and I/O boxes has become increasingly challenging due to the ever growing demand for higher computing performance and packaging density. Proper modeling, design and characterization of the cooling system have become critical to the overall system performance, reliability and energy consumption. Air moving devices such as blowers (centrifugal fans) are key components of the air-cooled electronic systems. This paper focuses on the flow characteristics of blowers and the impact on the system air flow distribution. To capture the flow characteristics, different levels of numerical modeling methodology are considered using a commercially available tool. It is demonstrated that a simple compact model, typically used in system level models, is not sufficient to resolve the air flow distribution near the exhaust. A more detailed model which includes the actual geometry of the blower blades resolves the body forces using MRF and predicts the flow distribution with better agreement with measurement data. Comparing the different modeling methodologies for systems of different impedance characteristics, a general guideline is subsequently proposed.

Air-water hybrid cooling for computer servers: A case study for optimum cooling energy allocation

Air-water hybrid cooling offers flexible design choices for computer systems with components of different thermal management needs. On one hand, water cooling enables the continuous growth of CPU performance and increasing packaging density. High performance cold plates such as microchannels have been successfully implemented for water cooling in previous high-end systems. When coupled with an air-water heat exchanger or radiator, the water loop becomes a closed one with no need for facility chilled water. This significantly reduces the complexity to deploy the server in the data center. On the other hand, for components with less thermal demand, traditional air-cooling technology is adequate with low cost, high availability and better serviceability. For the computer system as a whole, an air-water hybrid cooling system may be optimized. Such a hybrid system typically requires pumps to drive the water loops, air-movers to drive air through the radiator and blowers or fans to drive the air flow for component cooling. It is the focus of this paper to study the optimum allocation of energy between the pumps and air-movers for a given total cooling energy budget and overall load. The goals are to achieve better overall thermal performance and to reduce the cooling energy consumption. To this end models for each cooling block are established based on test data. These include the air-water heat exchanger, pumps, blowers, and cold plates. These models are linked together to predict the overall thermal system operating points for different application scenarios. A parametric study is then conducted to define the near optimum allocation of cooling energy for these scenarios that meets the thermal design objectives. Additionally, sub-threshold leakage for the CPU is taken into account to enhance the model since temperature provides positive feedback. It is shown through modeling that additional performance enhancement is possible with judicious allocation of cooling energy for a given overall energy budget. It is argued in this paper that overall energy efficiency can be improved significantly through intelligent data driven energy allocation.