John G. Torok

Packaging design of the IBM system z10 enterprise class platform central electronic complex

The IBM System z10™ Enterprise Class mainframe addresses the modern data center requirements for minimizing floor space while increasing computing power efficiency. These objectives placed challenges on the z10™ packaging design as a result of significantly increased demand on system packaging density, power delivery, and logic and power cooling efficiency compared with the recent IBM System z9® and z990 mainframe generations. Several innovations were implemented to successfully meet these challenges: a more powerful multichip module (MCM) that delivers denser computing capability and a 64-way system; a vertically mated processor unit (PU) book structure that achieves a more efficient thermal implementation and a higher signal bandwidth between processors; and a PU book-centric dc-dc power delivery design that is more efficient. This paper presents the key elements to achieve this design: the novel mechanical load transmission paths and the connector technologies for the MCM, PU book, I/O, and power regulation components; an innovative cooling and thermal design that includes component-level tolerance of failures; and improved power delivery and power code developments to maximize the overall z10 compute efficiency.

Packaging the IBM eServer z990 central electronic complex

The z990 eServerTM central electronic complex (CEC) houses four multichip-module-based processor units instead of one, as in the previous-generation z900 eServer. The multichip module (MCM) input/output pin density in z990 processor units is more than twice that of the MCMs in z900 processor units. This increase in packaging density and the consequent tripling of the current drawn by the processor units were accommodated by the first-time use of land grid array (LGA) MCM-to-board interconnections in an IBM zSeries® eServer. This was done by using innovative refrigeration cooling of the MCM with air cooling as backup, and by a new mechanical packaging and power distribution scheme. This paper describes the mechanical engineering of the CEC cage, the LGA MCM-to-board interconnection scheme, and the mechanical isolation of the MCM evaporator-heat-sink mass from the LGA contacts. The paper also describes the electrical power and the cooling solutions implemented to meet the more demanding requirements of the denser CEC package.

Mechanical packaging, power, and cooling design for the IBM z13

The system-level packaging of the IBM z13™ supports the implementation of a new drawer-based Central Processor Complex (CPC). Departing from previous IBM z Systems™ designs, the introduction of distributed land-grid-array (LGA) attached single-chip modules (SCMs) required new mechanical, power, and cooling designs to address specified performance requirements and to provide enhanced reliability, availability, and serviceability (RAS) attributes. Building upon the designs created for the IBM zEnterprise® BC12 (zBC12), new CPC drawer and frame mechanical designs were created to significantly increase overall packaging density. Similar to its predecessor, the IBM zEnterprise EC12 (zEC12), the z13 utilizes water-cooling of the processors, but in contrast to the single input and return flow used to cool the multi-chip module (MCM) in the zEC12, the z13 accomplishes its processor cooling using a flexible hose internal manifold design that provides parallel input and return fluid flow to each SCM. The use of flexible hose also enabled SCM field replacement, new to high-end IBM z Systems. A new internal cooling loop unit and an updated external (building-chilled) modular water-conditioning unit were designed utilizing customized water delivery manifold systems to feed the common CPC drawer design. Revised power delivery and service control structures were also created to address the distributed nature of the z13 system design.

Thermal-mechanical Co-design of Cold Plate, Second Level Thermal Interface Material (TIM2) and Heat Spreaders for Optimal Thermal Performance for High-end Processor Cooling

Cooling high-end system processors has become increasingly more challenging due to the increase in both total power and peak power density in processor cores. Junction peak temperature at worst case corner conditions often establish the limits on the maximum supportable circuit speed as well as processor chip yield. While significant progress has been made in cooling technology (e.g., cold plate design and thermal interface materials at first and the second level package), a systematic approach is needed to optimize the entire thermal and mechanical stack to achieve the overall (optimal) thermal performance objectives. The necessity and importance of this is due to the thermal and mechanical design interdependencies contained with the overall stack. This paper reports an in-depth study of the thermal-mechanical interactions associated with the cold plate, second level thermal interface material (TIM2) and heat spreaders. Thermal test results are reported for different cold plate designs and TIM2 pad sizes. Thermal and mechanical modeling results are provided to quantify the TIM2 thermal performance as a function of the TIM2 mechanical stress, the TIM2 dimensions and cold plate design. As described via both modeling and testing results, an optimal TIM2 pad size results as a trade-off between heat transfer area for conduction and TIM2 compressive pressure. In addition, pressure sensitive film study results are also provided revealing that heat spreader design affects the module level and TIM2 thermal performance. Results from this set of work clearly demonstrate the close interactions between cooling hardware in the stack hence the importance of thermal-mechanical co-design to achieve optimal thermal performance for the high-end processors.

SEISMIC TESTING AND ANALYSIS OF MAIN FRAME COMPUTER STRUCTURE

Designing a mainframe computer structure that can withstand seismic events requires significant testing and analysis. The computer frame and anchorage system must have adequate strength and stiffness to counteract earthquake-induced forces, thereby preventing human injury and potential system damage. However, this same frame and anchorage system must also meet the requirement of ensuring continued system operation by limiting overall displacement of the structure to accepted levels. Therefore, the test and analysis scope need to include the mainframe structure and its anchorage attachment to the building’s concrete floor via a raised floor structure. This paper discusses the numerical modeling and its verification to quantify the robustness of a high end computer server structure subjected to a severe seismic event. The frame of the computer is the structure where components are installed (e.g., the central processing unit, the input-output drawer, the power supply component, etc.). The dynamic response of this structure is highly related to the weight of the components, the assembly’s inherent natural frequency, and the location of the structure’s center of gravity. The natural frequency of various mainframe configurations were analyzed and measured by either changing the weights (i.e., adding or eliminating components) or by changing the structural stiffness (e.g., adding reinforcement brackets). The main objective of the modeling was to ensure structural integrity following a seismic test of a functional server system. Finite element analysis (FEA) was employed as part of the overall frame’s structural robustness design verification, whereby the simulated modal analysis results were compared to both the experimental modal data of the frame structure as well as measured swept sine data. This design study builds toward the objective of constructing a verified model of the server frame and components, which lead to a guideline for implementing optimized reinforcement. As part of the verification, the mainframe structures were subjected to horizontal table vibration tests simulating the loads and environmental conditions endured during seismic events. During experimental verification, the dynamic responses were recorded and analyzed in both the time and frequency domains.

Analysis and Evaluation Methods Associated with the Application of Compliant Thermal Interface Materials in Multi-chip Electronic Board Assemblies

Increased demands on large scale server system packaging density have driven the need for new, more challenging electronic component cooling solutions. One such application required the development of a large form-factor printed circuit board assembly with multiple power transformer devices to be cooled via a common heat spreader. Thermally coupling the multiplicity of devices to the heat spreader was completed using a compliant thermal interface material. Given the mechanical tolerance range, the strain rate dependency of the interface material and the mechanical load limitations of the electronic devices, finite element analysis and empirical evaluation techniques were applied to ensure the anticipated interface gaps were established and that the initial and residual mechanical loading effects were understood. A characterization of the thermal interface material’s mechanical properties was completed for analysis input. Coupling this input with the geometric and stiffness properties of the assembly’s structural elements provided predictions of both the initial as well as the residual mechanical assembly loads. Once completed, experiments using pressure sensitive film and piezoresistive film load cells were completed to correlate with the acquired analytical predictions.