Timothy J. Chainer

Server liquid cooling with chiller-less data center design to enable significant energy savings

This paper summarizes the concept design and hardware build efforts as part of a US Department of Energy cost shared grant, two year project (2010-2012) that was undertaken to develop highly energy efficient, warm liquid cooled servers for use in chiller-less data centers. Significant savings are expected in data center energy, refrigerant and make up water use. The technologies being developed include liquid cooling hardware for high volume servers, advanced thermal interface materials, and dry air heat exchanger (chiller-less with all year “economizer”) based facility level cooling systems that reject the Information Technology (IT) equipment heat load directly to the outside ambient air. Substantial effort has also been devoted towards exploring the use of high volume manufacturable components and cost optimized cooling designs that address high volume market design points. Demonstration hardware for server liquid cooling and data center economizer based cooling has been built and is operational for a 15 kW rack fully populated with liquid cooled servers. This design allows the use of up to 45 °C liquid coolant to the rack. Data collection has commenced to document the system thermal performance and energy usage using sophisticated instrumentation and data collection software methodologies. The anticipated benefits of such energy-centric configurations are significant energy savings at the data center level of as much as 30% and energy-proportional cooling in real time based on IT load and ambient air temperatures. The objective of this project is to reduce the cooling energy to 5% or less of a comparable typical air cooled chiller based total data center energy. Additional energy savings can be realized by reducing the IT power itself through reduced server fan power and potentially less leakage power due to lower device temperatures on average for most locations. This paper focuses on the server liquid cooling, the rack enclosure with heat exchanger cooling and liquid distribution, and the data center level cooling infrastructure. A sample of recently collected energy-efficiency data is also presented to provide experimental validation of the concept demonstrating cooling energy use to be less than 3.5% of the IT power for a hot summer day in New York.

Thermal proximity imaging of hard-disk substrates

We discuss a new measurement technique called thermal proximity sensing (TPS) and its application to the inspection of magnetic hard-disk surfaces. This method uses the magneto-resistive (MR) readback element of a hard-disk file in a novel thermally sensitive mode to actively sense the air-bearing gap spacing while the disk spins under the slider. We present model calculations that describe the origin of the thermal sensitivity and its range of applicability. By collecting height information as a function of position, we have obtained high-resolution images of disk topography with sensitivity in the subnanometer range. Initial results of the technique, obtained on a test stand, showed an exceptional imaging ability for surface features. We have also obtained useful surface structure data from in situ measurements of disk surface defects in an operating hard-disk file.

Experimental characterization of an energy efficient chiller-less data center test facility with warm water cooled servers

Typical data centers utilize approximately 50% of the total IT energy in cooling of the server racks. We present a chillerless data center where server-level cooling is achieved through a combination of warm water cooling hardware and re-circulated air; eventual heat rejection to ambient air is achieved using a closed secondary liquid loop to ambient-air heat exchanger (dry-cooler). Several experiments were carried out to characterize the individual pieces of equipment and data center thermal performance and energy consumption. A 22+ hour experimental run was also carried out with results indicating an average cooling energy use of 3.5% of the total IT energy use, with average ambient air temperatures of 23.8°C and average IT power use of 13.14 kW.

Thermal Characterization of Non-Raised Floor Air Cooled Data Centers Using Numerical Modeling

Data center equipment almost always represents a high expenditure capital investment to the customer, and is often operated without any down time. Data com equipment is typically designed to operate at a rack air inlet temperature of between 10 and 35°C, and a violation of this specification can diminish electronic device reliability and even lead to failure in the field. Thus, it is of paramount importance, from a reliability perspective, to sufficiently understand these systems. A representative non-raised floor data center system was numerically modeled and the data generated from a parametric study was analyzed. The model constitutes a half symmetry section of a 40 rack data center that is arranged in a cold aisle-hot aisle fashion. The effect of several input variables, namely, rack heat load, rack flow rate, rack temperature rise, diffuser flow rate, diffuser location, diffuser height, diffuser pitch, ceiling height, hot exhaust air return vent location, and non-uniformity in rack heat load, was studied. Temperature data was collected at several locations at the inlet to the racks. Statistical analysis was carried out to describe trends in the data.Copyright

A self-servowrite clocking process

A high-speed process for servowriting hard-disk assemblies (HDAs) without an external clock head has been developed. This robust process achieves servo-pattern alignment accuracy comparable to or better than that achieved with conventional clock-head based servowriters at a substantially reduced capital and process cost.

Integrated, variable-reluctance magnetic minimotor

The use of lithography and electroplating to fabricate variable-reluctance, nearly planar, integrated minimotors with 6-mm-diameter rotors on silicon wafers is described. The motors consist of six electroplated Permalloy® horseshoe-shaped cores that surround the rotor. Copper coils are formed around each core. The Permalloy and copper electroplating baths, electroplating seed layers, and through-mask plating techniques are similar to those used to fabricate inductive thin-film heads. High-aspect-ratio optical lithography or X-ray lithography was used to form the various resist layers. The rotors were fabricated separately, released from the substrate, and then slipped onto the shaft, which was plated as part of the stator fabrication process. The fabrication processes for stator and rotor are described in this paper, and initial minimotor operation data are presented.

Impact of operating conditions on a chiller-less data center test facility with liquid cooled servers

Data center cooling can constitute a significant portion of the total data center energy usage, with typical cooling energy expenditures approximately 50% of the IT energy use. Much of this energy consumption occurs at the refrigeration/chiller plant and in the Computer Room Air Handlers (CRAH) that cool and condition the air used to cool the electronics racks. To reduce cooling energy use, a data center test facility was designed and constructed to reduce cooling energy use to less than 5% of the total IT energy use through a combination of warm water cooling of the electronics and liquid-side economization. Several data center operating conditions, such as changes in liquid and air flow rates, heat exchanger arrangements and addition of propylene glycol were investigated to determine their impact on the energy consumption and thermal performance of the key cooling equipment. Day long runs collected from summer and fall days are also reported to illustrate the impact of external weather conditions and loop operating conditions on the thermal performance and energy consumption of the dual-loop data center test facility. The work presented highlights the impact of various operating conditions in influencing the cooling energy use and improving data center energy efficiency in chiller-less, ambient-air cooled data center designs using water cooled servers.

Embedded Two-Phase Cooling of Large 3D Compatible Chips With Radial Channels

Thermal performance for embedded two phase cooling using dielectric coolant (R1234ze) is evaluated on a ∼20 mm × 20 mm large die. The test vehicles incorporate radial expanding channels with embedded pin fields suitable for through-silicon-via (TSV) interconnects of multi-die stacks. Power generating features mimicking those anticipated in future generations of processor chips with 8 cores are included. Initial results show that for the types of power maps anticipated, critical heat fluxes in “core” areas of at least 350 W/cm2 with at least 20 W/cm2 “background” heating in rest of the chip area can be achieved with less than 30 °C temperature rise over the inlet coolant temperature. These heat fluxes are significantly higher than those seen for relatively long parallel channel devices of similar base channel dimensions. Experimental results of flow rate, pressure drop, “device,” and coolant temperature are also provided for these test vehicles along with details of the test facility developed to properly characterize the test vehicles.Copyright

Experimental investigation of water cooled server microprocessors and memory devices in an energy efficient chiller-less data center

Understanding and improving the thermal management and energy efficiency of data center cooling systems is of growing importance from a cost and sustainability perspective. Toward this goal, warm liquid cooled servers were developed to enable highly energy efficient chiller-less data centers that utilize only “free” ambient environment cooling. This approach greatly reduces cooling energy use, and could reduce data center refrigerant and make up water usage. In one exemplary experiment, a rack having such liquid cooled servers was tested on a hot summer day (~32°C) with CPU exercisers and memory exercisers running on every server to provide steady heat dissipation from the processors and from the DIMMs, respectively. Compared to a typical air cooled rack, significantly lower DIMM temperatures and CPU thermal values were observed.

A flexural in-line actuator for magnetic recording disk drives

An actuator for magnetic recording disk drives based on flexures and coupled to a voice coil motor (VCM) was developed for a 3.5-in disk drive. The design and performance of this system is described. This high-performance flexural actuator has been shown to have unique advantages over conventional designs, demonstrated in a magnetic recording disk file prototype. >