Thomas Brunschwiler

3D-ICE: fast compact transient thermal modeling for 3D ICs with inter-tier liquid cooling

Three dimensional stacked integrated circuits (3D ICs) are extremely attractive for overcoming the barriers in interconnect scaling, offering an opportunity to continue the CMOS performance trends for the next decade. However, from a thermal perspective, vertical integration of high-performance ICs in the form of 3D stacks is highly demanding since the effective areal heat dissipation increases with number of dies (with hotspot heat fluxes up to 250W/cm2) generating high chip temperatures. In this context, inter-tier integrated microchannel cooling is a promising and scalable solution for high heat flux removal. A robust design of a 3D IC and its subsequent thermal management depend heavily upon accurate modeling of the effects of liquid cooling on the thermal behavior of the IC during the early stages of design. In this paper we present 3D-ICE, a compact transient thermal model (CTTM) for the thermal simulation of 3D ICs with multiple inter-tier microchannel liquid cooling. The proposed model is compatible with existing thermal CAD tools for ICs, and offers significant speed-up (up to 975x) over a typical commercial computational fluid dynamics simulation tool while preserving accuracy (i.e., maximum temperature error of 3.4%). In addition, a thermal simulator has been built based on 3D-ICE, which is capable of running in parallel on multicore architectures, offering further savings in simulation time and demonstrating efficient parallelization of the proposed approach.

Forced convective interlayer cooling in vertically integrated packages

The heat removal capability of area-interconnect-compatible interlayer cooling in vertically integrated, high-performance chip stacks was characterized with de-ionized water as coolant. Correlation-based predictions and computational fluid dynamic modeling of cross-flow heat-removal structures show that the coolant temperature increase due to sensible heat absorption limits the cooling performance at hydraulic diameters les 200 mum. An experimental investigation with uniform and double-side heat flux at Reynolds numbers les 1000 and heat transfer areas of 1 cm2 was carried out to identify the most efficient interlayer heat-removal structure. Parallel plate, microchannel, pin fin, and their combinations with pins using in-line and staggered configurations with round and drop-like shapes at pitches ranging from 50 to 200 mum and fluid structure heights of 100 to 200 mum were tested. A hydrodynamic flow regime transition responsible for a local junction temperature minimum was observed for pin fin inline structures. The experimental data was extrapolated to predict maximal heat flux in chip stacks with a 4-cm2 heat transfer area. The performance of interlayer cooling strongly depends on this parameter, and drops from >200 W/cm2 at 1 cm2 and >50 mum interconnect pitch to <100 W/cm2 at 4 cm2.

Direct Liquid Jet-Impingment Cooling With Micron-Sized Nozzle Array and Distributed Return Architecture

We demonstrate submerged single-phase direct liquid-jet-impingement cold plates that use arrays of jets with diameters in the range of 31 to 126 mum and cell pitches from 100 to 500 mum for high power-density microprocessor cooling applications. Using parallel inlet and outlet manifolds, a distributed return concept for easy scaling to 40,000 cells on an area of 4 cm was implemented. Pressure drops < 0.1 bar at 2.5 1/min flow rate have been reached with a hierarchical tree-like double-branching manifold. Experiments were carried out with water jets having Reynolds numbers smaller than 900 at nozzle to heater gaps ranging between 3 to 300 mum. We identified four flow regimes, namely, pinch-off, transition, impingement, and separation, with different influences on heat-removal and pressure-drop characteristics. Parametric analysis resulted in an optimal heat-removal rate of 420 W/cm2 using water as a coolant. For a near optimal design with a gap to inlet diameter ratio of 1.2, we measured a heat-transfer coefficient of 8.7 W/cm2 K and a junction to inlet fluid unit thermal resistance of 0.17 Kcm2 /W (720 mum chip), which is equivalent to a 370 W/cm2 cooling performance at a junction to inlet fluid temperature rise of 63 degC, a pressure drop of 0.35 bar, and a flow rate of 2.5 1/min

Energy-efficient variable-flow liquid cooling in 3D stacked architectures

Liquid cooling has emerged as a promising solution for addressing the elevated temperatures in 3D stacked architectures. In this work, we first propose a framework for detailed thermal modeling of the microchannels embedded between the tiers of the 3D system. In multicore systems, workload varies at runtime, and the system is generally not fully utilized. Thus, it is not energy-efficient to adjust the coolant flow rate based on the worst-case conditions, as this would cause an excess in pump power. For energy-efficient cooling, we propose a novel controller to adjust the liquid flow rate to meet the desired temperature and to minimize pump energy consumption. Our technique also includes a job scheduler, which balances the temperature across the system to maximize cooling efficiency and to improve reliability. Our method guarantees operating below the target temperature while reducing the cooling energy by up to 30%, and the overall energy by up to 12% in comparison to using the highest coolant flow rate.

Experimental Investigation of an Ultrathin Manifold Microchannel Heat Sink for Liquid-Cooled Chips

We report an experimental investigation of a novel, high performance ultrathin manifold microchannel heat sink. The heat sink consists of impinging liquid slot-jets on a structured surface fed with liquid coolant by an overlying two-dimensional manifold. We developed a fabrication and packaging procedure to manufacture prototypes by means of standard microprocessing. A closed fluid loop for precise hydrodynamic and thermal characterization of six different test vehicles was built. We studied the influence of the number of manifold systems, the width of the heat transfer microchannels, the volumetric flow rate, and the pumping power on the hydrodynamic and thermal performance of the heat sink. A design with 12.5 manifold systems and 25 μm wide microchannels as the heat transfer structure provided the optimum choice of design parameters. For a volumetric flow rate of 1.3 l/min we demonstrated a total thermal resistance between the maximum heater temperature and fluid inlet temperature of 0.09 cm 2 K/W with a pressure drop of 0.22 bar on a 2 ×2 cm 2 chip. This allows for cooling power densities of more than 700 W/cm 2 for a maximum temperature difference between the chip and the fluid inlet of 65 K. The total height of the heat sink did not exceed 2 mm, and includes a 500 μm thick thermal test chip structured by 300 μm deep microchannels for heat transfer. Furthermore, we discuss the influence of elevated fluid inlet temperatures, allowing possible reuse of the thermal energy, and demonstrate an enhancement of the heat sink cooling efficiency of more than 40% for a temperature rise of 50 K.

On the Cooling of Electronics With Nanofluids

Nanofluids have been proposed to improve the performance of microchannel heat sinks. In this paper, we present a systematic characterization of aqueous silica nanoparticle suspensions with concentrations up to 31 vol %. We determined the particle morphology by transmission electron microscope imaging and its dispersion status by dynamic light scattering measurements. The thermophysical properties of the fluids, namely, their specific heat, density, thermal conductivity, and dynamic viscosity were experimentally measured. We fabricated microchannel heat sinks with three different channel widths and characterized their thermal performance as a function of volumetric flow rate for silica nanofluids at concentrations by volume of 0%, 5%, 16%, and 31%. The Nusselt number was extracted from the experimental results and compared with the theoretical predictions considering the change of fluids bulk properties. We demonstrated a deviation of less than 10% between the experiments and the predictions. Hence, standard correlations can be used to estimate the convective heat transfer of nanofiuids. In addition, we applied a one-dimensional model of the heat sink, validated by the experiments. We predicted the potential of nanofluids to increase the performance of microchannel heat sinks. To this end, we varied the individual thermophysical properties of the coolant and studied their impact on the heat sink performance. We demonstrated that the relative thermal conductivity enhancement must be larger than the relative viscosity increase in order to gain a sizeable performance benefit. Furthermore, we showed that it would be preferable to increase the volumetric heat capacity of the fluid instead of increasing its thermal conductivity.

Toward zero-emission data centers through direct reuse of thermal energy

We have tested hot water data center cooling by directly reusing the generated thermal energy in neighborhood heating systems. First, we introduce high-performance liquid cooling devices with minimal thermal resistance in order to cool a computer system board. This cooling is performed with water at a temperature as high as 608C, thereby eliminating the chillers and their electrical power consumption, and enabling direct reuse of the heat. We collect 85% of the board heat using microscale liquid coolers for CPUs (central processing units), interfaces, and dc (direct current) converters. With our concept, data centers can be cooled in all climate zones throughout the year without a pre-cooled heat carrier. Second, we analyze how the supply of heat and financial payback from customers reduce the total cost of ownership. With 5,000 district heating systems satisfying 9.7% of the thermal demand of Europe, ample opportunities exist for data centers to become heat providers, thereby reducing the associated carbon dioxide emission. Finally, we show how our concept can be developed within 5 years into a zero-emission data center and that such investments are economically viable and ecologically beneficial given increasing energy prices. With such data centers, the IT (information technology) industry can assume a key role in greatly reducing carbon dioxide emissions and global warming by replacing energy-intensive processes with more efficient, digitally assisted processes.

Toward five-dimensional scaling: how density improves efficiency in future computers

We address integration density in future computers based on packaging and architectural concepts of the human brain: a dense 3-D architecture for interconnects, fluid cooling, and power delivery of energetic chemical compounds transported in the same fluid with little power needed for pumping. Several efforts have demonstrated that by vertical integration, memory proximity and bandwidth are improved using efficient communication with low-complexity 2-D arrays. However, power delivery and cooling do not allow integration of multiple layers with dense logic elements. Interlayer cooled 3-D chip stacks solve the cooling bottlenecks, thereby allowing stacking of several such stacks, but are still limited by power delivery and communication. Electrochemical power delivery eliminates the electrical power supply network, freeing valuable space for communication, and allows scaling of chip stacks to larger systems beyond exascale device count and performance. We find that historical efficiency trends are related to density and that current transistors are small enough for zetascale systems once communication and supply networks are simultaneously optimized. We infer that biological efficiencies for information processing can be reached by 2060 with ultracompact space-filled systems that make use of brain-inspired packaging and allometric scaling laws.

Heat-removal performance scaling of interlayer cooled chip stacks

Interlayer cooling is the only heat removal concept which scales with the number of active tiers in a vertically integrated chip stack. In this work, we numerically and experimentally characterize the performance of a three tier chip stack with a footprint of 1cm2. The implementation of 100µm pitch area array interconnect compatible heat transfer structures results in a maximal junction temperature increase of 54.7K at 1bar pressure drop with water as coolant for 250W/cm2 hot-spot and 50W/cm2 background heat flux. The total power removed was 390W which corresponds to a 3.9kW/cm3 volumetric heat flow.

Energy-Efficient Multiobjective Thermal Control for Liquid-Cooled 3-D Stacked Architectures

3-D stacked systems reduce communication delay in multiprocessor system-on-chips (MPSoCs) and enable heterogeneous integration of cores, memories, sensors, and RF devices. However, vertical integration of layers exacerbates temperature-induced problems such as reliability degradation. Liquid cooling is a highly efficient solution to overcome the accelerated thermal problems in 3-D architectures; however, it brings new challenges in modeling and run-time management for such 3-D MPSoCs with multitier liquid cooling. This paper proposes a novel design-time/run-time thermal management strategy. The design-time phase involves a rigorous thermal impact analysis of various thermal control variables. We then utilize this analysis to design a run-time fuzzy controller for improving energy efficiency in 3-D MPSoCs through liquid cooling management and dynamic voltage and frequency scaling (DVFS). The fuzzy controller adjusts the liquid flow rate dynamically to match the cooling demand of the chip for preventing overcooling and for maintaining a stable thermal profile. The DVFS decisions increase chip-level energy savings and help balance the temperature across the system. Our controller is used in conjunction with temperature-aware load balancing and dynamic power management strategies. Experimental results on 2-tier and 4-tier 3-D MPSoCs show that our strategy prevents the system from exceeding the given threshold temperature. At the same time, we reduce cooling energy by up to 63% and system-level energy by up to 21% in comparison to statically setting a flow rate setting to handle worst-case temperatures.