Thomas J. Breen

From Chip to Cooling Tower Data Center Modeling: Influence of Server Inlet Temperature and Temperature Rise Across Cabinet

To achieve reductions in the power consumption of the data center cooling infrastructure, the current strategy in data center design is to increase the inlet temperature to the rack, while the current strategy for energy-efficient system thermal design is to allow increased temperature rise across the rack. Either strategy, or a combination of both, intuitively provides enhancements in the coefficient of performance of the data center in terms of computing energy usage relative to cooling energy consumption. However, this strategy is currently more of an empirically based approach from practical experience, rather than a result of a good understanding of how the impact of varying temperatures and flow rates at rack level influences each component in the chain from the chip level to the cooling tower. The aim of this paper is to provide a model to represent the physics of this strategy by developing a modeling tool that represents the heat flow from the rack level to the cooling tower for an air cooled data center with chillers. This model presents the performance of a complete data center cooling system infrastructure. After detailing the model, two parametric studies are presented that illustrate the influence of increasing rack inlet air temperature, and temperature rise across the rack, on different components in the data center cooling architecture. By considering the total data center, and each components influence on the greater infrastructure, it is possible to identify the components that contribute most to the resulting inefficiencies in the heat flow from chip to cooling tower and thereby identify the components in need of possible redesign. For the data center model considered here it is shown that the strategy of increasing temperature rise across the rack may be a better strategy than increasing inlet temperature to the rack.

From chip to cooling tower data center modeling: Part I Influence of server inlet temperature and temperature rise across cabinet

To achieve reductions in the power consumption of the data center cooling infrastructure, the current strategy in data center design is to increase the inlet temperature to the rack, while the current strategy for energy-efficient system thermal design is to allow increased temperature rise across the rack. Either strategy, or a combination of both, intuitively provides enhancements in the coefficient of performance (COP) of the data center in terms of computing energy usage relative to cooling energy consumption. However, this strategy is currently more of an empirically based approach from practical experience, rather than a result of a good understanding of how the impact of varying temperatures and flow rates at rack level influences each component in the chain from the chip level to the cooling tower. The aim of this paper is to provide a model to represent the physics of this strategy by developing a modeling tool that represents the heat flow from the rack level to the cooling tower for an air cooled data center with chillers. This model presents the performance of a complete data center cooling system infrastructure. After detailing the model, two parametric studies are presented that illustrate the influence of increasing rack inlet air temperature, and temperature rise across the rack, on different components in the data center cooling architecture. By considering the total data center, and each components influence on the greater infrastructure, it is possible to identify the components that contribute most to the resulting inefficiencies in the heat flow from chip to cooling tower and thereby identify the components in need of possible redesign. For the data center model considered here it is shown that the strategy of increasing temperature rise across the rack may be a better strategy than increasing inlet temperature to the rack. Part II of this work expands on this paper with further parametric studies to evaluate the robustness of this data center cooling strategy, with conditions for optimal strategy deployment.

From chip to cooling tower data center modeling: Part II Influence of chip temperature control philosophy

The chiller cooled data center environment consists of many interlinked elements that are usually treated as individual components. This chain of components and their influences on each other must be considered in determining the benefits of any data center design and operational strategies seeking to improve efficiency, such as temperature control fan algorithms. Using the models developed in part I of this work, this paper extends the analysis to include the electronics within the rack through considering the processor heat sink temperature. This has allowed determination of the influence of various cooling strategies on the data center coefficient of performance. The strategy of increasing inlet aisle temperature is examined in some detail and found not to be a robust methodology for improving the overall energy performance of the data center, while tight temperature controls at the chip level consistently provides better performance, yielding more computing per watt of cooling power. These findings are of strong practical relevance for the design of fan control algorithms at the rack level and general operational strategies in data centers. Finally, the impact of heat sink thermal resistance is considered and the potential data center efficiency gains from improved heat sink designs are discussed.

From Chip to Cooling Tower Data Center Modeling: Chip Leakage Power and Its Impact on Cooling Infrastructure Energy Efficiency

The power consumption of the chip package is known to vary with operating temperature, independently of the workload processing power. This variation is commonly known as chip leakage power, typically accounting for ~10% of total chip power consumption. The influence of operating temperature on leakage power consumption is a major concern for the information technology (IT) industry for design optimization where IT system power densities are steadily increasing and leakage power expected to account for up to ~50% of chip power in the near future associated with the reducing package size. Much attention has been placed on developing models of the chip leakage power as a function of package temperature, ranging from simple linear models to complex super-linear models. This knowledge is crucial for IT system designers to improve chip level energy efficiency and minimize heat dissipation. However, this work has been focused on the component level with little thought given to the impact of chip leakage power on entire data center efficiency. Studies on data center power consumption quote IT system heat dissipation as a constant value without accounting for the variance of chip power with operating temperature due to leakage power. Previous modeling techniques have also omitted this temperature dependent relationship. In this paper, we discuss the need for chip leakage power to be included in the analysis of holistic data center performance. A chip leakage power model is defined and its implementation into an existing multiscale data center energy model is discussed. Parametric studies are conducted over a range of system and environment operating conditions to evaluate the impact of varying degrees of chip leakage power. Possible strategies for mitigating the impact of leakage power are also illustrated in this study. This work illustrates that when including chip leakage power in the data center model, a compromise exists between increasing operating temperatures to improve cooling infrastructure efficiency and the increase in heat load at higher operating temperatures due to leakage power.

From Chip to Cooling Tower Data Center Modeling: Influence of Air-Stream Containment on Operating Efficiency

In the drive to enhance data center energy efficiency, much attention has been placed on the prospect of airflow containment in hot-aisle cold-aisle raised floor arrangements. Such containment prevents airflow recirculation, eliminating the mixing effects of the hot and cold air streams that can cause an undesirable temperature rise at the inlet of the equipment racks. The intuitive assessment of the industry has been that the elimination of such mixing effects increases the energy efficiency of the data center cooling system by enabling delivery of air at higher inlet temperatures, thus reducing the amount of infrastructure cooling required. This paper employs an end-to-end modeling approach to analyze the effect of air stream containment in the computer room and its impact on the holistic system efficiency. Dimensionless heat index parameters are employed to characterize the effects of containment, recirculation and mixing within the computer room environment. The extent of recirculation is shown to primarily influence the operation of the rack and CRAC level cooling systems, with the chiller systems also impacted. The overall effect on the complete cooling system performance and data center efficiency requires balancing of these effects. Through this model analysis, it is shown that containment may negatively impact overall energy efficiency in some circumstances, and that recirculation may actually be beneficial to overall energy efficiency under certain system dependent operating thresholds.Copyright

On the Need for Energy Efficiency Metrics That Span Integrated IT-Facility Infrastructures

Infrastructure efficiency metrics, such as Power Usage Effectiveness (PUE) and Data Center Infrastructure Efficiency (DCiE), have gained much traction within the industry for evaluating data center energy efficiency. Gradually, however, as the lines between traditional IT systems in the data center and the facility infrastructure supporting the data center get blurred, adaptations to the usage of infrastructure efficiency metrics will be required. This paper presents three cases where holistic data center energy efficiency does not necessarily track infrastructure efficiency, implicitly emphasizing the need for new metrics that address the IT-facility infrastructure in holistic fashion.Copyright

From Chip to Cooling Tower Data Center Modeling: Chip Leakage Power and its Impact on Cooling Infrastructure Energy Efficiency

The power consumption of the chip package is known to vary with operating temperature, independently of the workload processing power. This variation is commonly known as chip leakage power, typically accounting for ∼10% of total chip power consumption. The influence of operating temperature on leakage power consumption is a major concern for the IT industry for design optimization where IT system power densities are steadily increasing and leakage power expected to account for up to ∼50% of chip power in the near future associated with the reducing package size. Much attention has been placed on developing models of the chip leakage power as a function of package temperature, ranging from simple linear models to complex super-linear models. This knowledge is crucial for IT system designers to improve chip level energy efficiency and minimize heat dissipation. However, this work has been focused on the component level with little thought given to the impact of chip leakage power on entire data center efficiency. Studies on data center power consumption quote IT system heat dissipation as a constant value without accounting for the variance of chip power with operating temperature due to leakage power. Previous modeling techniques have also omitted this temperature dependent relationship. In this paper we discuss the need for chip leakage power to be included in the analysis of holistic data center performance. A chip leakage power model is defined and its implementation into an existing multi-scale data center energy model is discussed. Parametric studies are conducted over a range of system and environment operating conditions to evaluate the impact of varying degrees of chip leakage power. Possible strategies for mitigating the impact of leakage power are also illustrated in this study. This work illustrates that when including chip leakage power in the data center model, a compromise exists between increasing operating temperatures to improve cooling infrastructure efficiency and the increase in heat load at higher operating temperatures due to leakage power.Copyright

Influence of Experimental Uncertainty on Prediction of Holistic Multi-Scale Data Center Energy Efficiency

As the energy footprint of data centers continues to increase, models that allow for “what-if” simulations of different data center design and management paradigms will be important. Prior work by the authors has described a multi-scale energy efficiency model that allows for evaluating the coefficient of performance of the data center ensemble (COPGrand ), and demonstrated the utility of such a model for purposes of choosing operational set-points and evaluating design trade-offs. However, experimental validation of these models poses a challenge because of the complexity involved with tailoring such a model for implementation to legacy data centers, with shared infrastructure and limited control over IT workload. Further, test facilities with dummy heat loads or artificial racks in lieu of IT equipment generally have limited utility in validating end-to-end models owing to the inability of such loads to mimic phenomena such as fan scalability, etc. In this work, we describe the experimental analysis conducted in a special test chamber and data center facility. The chamber, focusing on system level effects, is loaded with an actual IT rack, and a compressor delivers chilled air to the chamber at a preset temperature. By varying the load in the IT rack as well as the air delivery parameters — such as flow rate, supply temperature, etc. — a setup which simulates the system level of a data center is created. Experimental tests within a live data center facility are also conducted where the operating conditions of the cooling infrastructure are monitored — such as fluid temperatures, flow rates, etc. — and can be analyzed to determine effects such as air flow recirculation, heat exchanger performance, etc. Using the experimental data a multi-scale model configuration emulating the data center can be defined. We compare the results from such experimental analysis to a multi-scale energy efficiency model of the data center, and discuss the accuracies as well as inaccuracies within such a model. Difficulties encountered in the experimental work are discussed. The paper concludes by discussing areas for improvement in such modeling and experimental evaluation. Further validation of the complete multi-scale data center energy model is planned.© 2011 ASME