Allan C. Vandeventer

Mechanical packaging, power, and cooling design for the IBM z13

The system-level packaging of the IBM z13™ supports the implementation of a new drawer-based Central Processor Complex (CPC). Departing from previous IBM z Systems™ designs, the introduction of distributed land-grid-array (LGA) attached single-chip modules (SCMs) required new mechanical, power, and cooling designs to address specified performance requirements and to provide enhanced reliability, availability, and serviceability (RAS) attributes. Building upon the designs created for the IBM zEnterprise® BC12 (zBC12), new CPC drawer and frame mechanical designs were created to significantly increase overall packaging density. Similar to its predecessor, the IBM zEnterprise EC12 (zEC12), the z13 utilizes water-cooling of the processors, but in contrast to the single input and return flow used to cool the multi-chip module (MCM) in the zEC12, the z13 accomplishes its processor cooling using a flexible hose internal manifold design that provides parallel input and return fluid flow to each SCM. The use of flexible hose also enabled SCM field replacement, new to high-end IBM z Systems. A new internal cooling loop unit and an updated external (building-chilled) modular water-conditioning unit were designed utilizing customized water delivery manifold systems to feed the common CPC drawer design. Revised power delivery and service control structures were also created to address the distributed nature of the z13 system design.

Thermal-mechanical Co-design of Cold Plate, Second Level Thermal Interface Material (TIM2) and Heat Spreaders for Optimal Thermal Performance for High-end Processor Cooling

Cooling high-end system processors has become increasingly more challenging due to the increase in both total power and peak power density in processor cores. Junction peak temperature at worst case corner conditions often establish the limits on the maximum supportable circuit speed as well as processor chip yield. While significant progress has been made in cooling technology (e.g., cold plate design and thermal interface materials at first and the second level package), a systematic approach is needed to optimize the entire thermal and mechanical stack to achieve the overall (optimal) thermal performance objectives. The necessity and importance of this is due to the thermal and mechanical design interdependencies contained with the overall stack. This paper reports an in-depth study of the thermal-mechanical interactions associated with the cold plate, second level thermal interface material (TIM2) and heat spreaders. Thermal test results are reported for different cold plate designs and TIM2 pad sizes. Thermal and mechanical modeling results are provided to quantify the TIM2 thermal performance as a function of the TIM2 mechanical stress, the TIM2 dimensions and cold plate design. As described via both modeling and testing results, an optimal TIM2 pad size results as a trade-off between heat transfer area for conduction and TIM2 compressive pressure. In addition, pressure sensitive film study results are also provided revealing that heat spreader design affects the module level and TIM2 thermal performance. Results from this set of work clearly demonstrate the close interactions between cooling hardware in the stack hence the importance of thermal-mechanical co-design to achieve optimal thermal performance for the high-end processors.

Air-water hybrid cooling for computer servers: A case study for optimum cooling energy allocation

Air-water hybrid cooling offers flexible design choices for computer systems with components of different thermal management needs. On one hand, water cooling enables the continuous growth of CPU performance and increasing packaging density. High performance cold plates such as microchannels have been successfully implemented for water cooling in previous high-end systems. When coupled with an air-water heat exchanger or radiator, the water loop becomes a closed one with no need for facility chilled water. This significantly reduces the complexity to deploy the server in the data center. On the other hand, for components with less thermal demand, traditional air-cooling technology is adequate with low cost, high availability and better serviceability. For the computer system as a whole, an air-water hybrid cooling system may be optimized. Such a hybrid system typically requires pumps to drive the water loops, air-movers to drive air through the radiator and blowers or fans to drive the air flow for component cooling. It is the focus of this paper to study the optimum allocation of energy between the pumps and air-movers for a given total cooling energy budget and overall load. The goals are to achieve better overall thermal performance and to reduce the cooling energy consumption. To this end models for each cooling block are established based on test data. These include the air-water heat exchanger, pumps, blowers, and cold plates. These models are linked together to predict the overall thermal system operating points for different application scenarios. A parametric study is then conducted to define the near optimum allocation of cooling energy for these scenarios that meets the thermal design objectives. Additionally, sub-threshold leakage for the CPU is taken into account to enhance the model since temperature provides positive feedback. It is shown through modeling that additional performance enhancement is possible with judicious allocation of cooling energy for a given overall energy budget. It is argued in this paper that overall energy efficiency can be improved significantly through intelligent data driven energy allocation.