Mark E. Steinke

A Computational Study to Compare the Data Center Cooling Energy Spent Using Traditional Air Cooled and Hybrid Water Cooled Servers

High performance datacenters that are being built and operated to ensure optimized compute density for high performance computing (HPC) workloads are constrained by the requirement to provide adequate cooling for the servers. Traditional methods of cooling dense high power servers using air cooling imposes a large cooling and power burden on datacenters. Airflow optimization of the datacenter is a constraint subject to a high energy penalty when dense power hungry racks each capable of consuming 30 to 40 kW are populated in a dense datacenter environment. The work documented using a simulation model (TileFlow) in this paper demonstrates the challenges associated with a standard air cooled approach in a HPC datacenter. Alternate cooling approaches to traditional air cooling are simulated as a comparison to traditional air cooling. These include models using a heat exchanger assisted rack cooling solution with conventional chilled water and, a direct to node cooling model simulated for the racks.These three distinct data center models are simulated at varying workloads and the resulting data is presented for typical and maximal inlet temperatures to the racks. For each cooling solution an estimate of the energy spend for the servers is determined based on the estimated PUEs of the cooling solutions chosen.Copyright

Thermodynamic Characterization of a Direct Water Cooled Server Rack Running Synthetic and Real High Performance Computing Work Loads

High performance computing server racks are being engineered to contain significantly more processing capability within the same computer room footprint year after year. The processor density within a single rack is becoming high enough that traditional, inefficient air-cooling of servers is inadequate to sustain HPC workloads. Experiments that characterize the performance of a direct water-cooled server rack in an operating HPC facility are described in this paper. Performance of the rack is reported for a range of cooling water inlet temperatures, flow rates and workloads that include actual and worst-case synthetic benchmarks. Power and temperature measurements of all processors and memory components in the rack were made while extended benchmark tests were conducted throughout the range of cooling variables allowed within an operational HPC facility. Synthetic benchmark results were compared with those obtained on a single server of the same design that had been characterized thermodynamically. Neither actual nor synthetic benchmark performances were affected during the course of the experiments, varying less than 0.13 percent. Power consumption change in the rack was minimal for the entire excursion of coolant temperatures and flow rates. Establishing the characteristics of such a highly energy efficient server rack in situ is critical to determine how the technology might be integrated into an existing heterogeneous, hybrid cooled computing facility — i.e., a facility that includes some servers that are air cooled as well as some that are direct water cooled.Copyright

Thermodynamic Characterization of a Server Optimized for High Performance Computing Using Water Cooling

Energy efficiency is an essential element of server design for high performance computers. Traditional HPC servers or nodes that are air cooled enable efficiency by using optimized system design elements that include efficient heat sink design for critical components such as CPUs, Memory, Networking and Disk Subsystems. In addition, airflow optimization is enabled via critical component placement decisions as well as fan and cooling algorithms that have an objective to optimize airflow and maximize system performance. Critical elements that cannot be avoided in traditional air cooled servers are computer center level management of both the airflow requirements and the exhaust heat flux of the servers. An alternative approach shown in this paper uses a novel water cooled design that enables both extreme energy efficiency for heat extraction of the server heat load and allows for lower device operating temperatures for the critical components. Experimental data documented in this paper illustrates the advantages of using non-chilled water to cool the server, allowing 85 to 90 percent of the server heat load to be extracted by water while allowing inlet water temperatures up to 45 degrees Celsius. A comparison is made of the energy consumption needed to cool the server components for both the air cooled and water cooled systems. The base system used for the comparison uses identical system electronics and firmware. The server thermal data shown in the paper include thermal behavior at idle, typical and maximum power consumption states for the server. The data documents the range of boundary conditions that can be tolerated for water cooled server solutions and the comparative advantages of using this technology.Copyright

Experimental Investigation of a Cost Effective Skived Microchannel Cold Plate for Mass Production

A liquid cold plate that utilizes skived microchannels has been developed to gain the benefits of direct liquid cooling, but minimize the expensive cost of such cold plates. The construction, application, and experimental results of the skived cold plate will be presented. Skiving is a mechanical process that cuts thin layers of material. It is an established process for making air cooled heat sinks. In this application, the fin field is skived and placed inside a housing that allows for liquid flow through the resulting fins.The design boundary conditions and parameters will be described and performance per cost metric will be presented and used to evaluate future optimization possibilities. The objective of the present work was to minimize the thermal resistance while maintaining a low manufacturing cost. The design goal was to produce a cold plate that had sufficient thermal performance and the ability to be mass produced at a reasonable cost.The resulting cold plate would also need to support warm water cooling of microprocessors. Warm water is a working fluid that has not been chilled below ambient temperatures. Therefore, the water temperature could be up to 45 degrees Celsius. The cold plate had a thermal resistance less than 0.3 °Ccm2/W. The pressure drop was minimized to lower the required pumping power and was less than 6 kPa at 1.0 liter per minute. Using a skiving process, it is possible to develop a cold plate that delivers good thermal performance and maintains a low production cost.© 2015 ASME