A Framework for Auditing Data Center Energy Usage and Mitigating Environmental Footprint
AA Framework for Auditing Data Center Energy Usage andMitigating Environmental Footprint
Justin A. Gould
Purdue UniversityWest Lafayette, [email protected]
ABSTRACT
Data Science has been a growing field since the 2010s. Betweenself-driving cars, movie and music recommendations, routing algo-rithms, and much more, both private companies and universitiesare making major investments in the field. As the field continues tomature, and we collect more data, the demand to store and analyzethem will continue to increase. This increase in data availability anddemand for analytics will put a strain on data centers and computeclustersβwith implications for both energy costs and emissions.As the world battles a climate crisis, it is prudent for organizationswith data centers to have a framework for combatting increasingenergy costs and emissions to meet demand for analytics work.In this paper, I present a generalized framework for organizationsto audit data centers energy efficiency to understand the resourcesrequired to operate a given data center and effective steps organi-zations can take to improve data center efficiency and lower theenvironmental impact.
KEYWORDS
Energy Efficiency, Energy Audit, Data Center, High-PerformanceComputing, Environment
With the increased demand for access to computational resources,the amount of energy required to operate data centers is expected tocontinue to grow. In 2020, global data center usage was > ππ β [1].According to the Central Intelligence Agencyβs 2020 Fact Book, in2016, the electrical consumption of Poland, Sweden, and Norwaywere 142
ππ β , 127
ππ β , and 126
ππ β , respectively [2]. By 2030,global data center electricity usage is predicted to exceed 2 tril-lion
ππ β [1]. In the midst of a global climate crisis, data centersβemissions and contributions are non-trivial and should be takenseriously to mitigate. The need for a generalized framework fororganizations to audit data centers energy efficiency is great.Data center emissions has implications in the fields of Artifi-cial Intelligence and Machine Learning, tooβas training models,and running inference in a production environment, requires ac-cess to compute resources, such as graphic processing units (GPU).Open AIβs Tom B. Brown et al.βs work on GPT-3 made referenceto energy usage and implications of training models; however, thepaper did not offer solutions to this growing problem [3]. The au-thors reported estimates of the compute resources required to train17 recent state-of-the-art NLP models. The numbers provided arenot accessible to non-subject matter experts and difficult to visu-alize. I took these estimates and converted the power required tothe number of average U.S. homes that can be powered for one year, depicted in Table 1. For the purposes of this analysis, I as-sume 16 . πΊπΉπΏππ / π€ππ‘π‘ , per a conservative cursory search of theTop500 List [4] and a similar analysis only for GPT-3 performedby Matthew Burruss [5], and average U.S. home electrical con-sumption as 900 ππ β per month, per the U.S. Energy InformationAdministration [6].Given the recent popularity of Data Science, and the increaseddemand for computational resources, previous work has been com-pleted in attempting to create a set of guidelines and standards fororganizations to follow to ensure data center efficiency. For exam-ple, the Lawrence Berkeley National Laboratoryβs Center of Exper-tise for Energy Efficiency in Data Centers updated its Data CenterMaster List of Energy Efficiency Measures in September 2020 [7].This document is a collection of best practices for energy efficientdata center operations, split into 7 sections:
Data Center EnergyEfficiency Management , IT Power Distribution Chain , IT Equipment , Lighting , Air Management, Cooling the Data Center Space , and
Cen-tral Cooling Plant . While the report has a large quantity of bestpractices and action items, many of the recommendations providedare resource-intensiveβwhich may put smaller organizations ata disadvantageβand may not provide enough detail to be clearlyunderstood and actionable. The Master Listβs
EEM 1-2.7: Perform anEnergy Audit , for example, does not provide enough detail to carryout an audit. The Master List does, however, link the
DCEE ToolkitβsEnergy Assessment Process Manual [8]. The process manual fallsvictim to the similar shortfall of the Master List, whereby, quantityof content overshadows specificity and accessibility.Another example of recent work is the U.S. Department of En-ergyβs
Best Practices Guide for Energy-Efficient Data Center De-sign [9]. This guide is similar to the Master List, as it walks throughbest practices for various aspects of the data center, such as airmanagement, cooling systems, environmental conditions, and ITsystems. While the design guide offers more specificity than theMaster List, it is not only nearly a decade old, but lacks the di-versity and quantity of best practices provided by the Master List.Most apparent is the absence of guidelines to execute an energyefficiency audit. Furthermore, neither guide provides insight intoorganizational challenges associated with embarking on an audit,such as: creating an audit team, setting goals and benchmarks, andcommunication/goal accountability. I aim to address these impor-tant aspects of the audit process, and close these gaps, in section3. My primary goal for this paper is to leverage, enhance, and com-plement previous work in this space to provide a specific frame-work for an energy audit of a data centerβwith clear suggestionsfor organizations of varying size and resource availability. This is a r X i v : . [ c s . D C ] F e b ustin A. Gould Table 1: Electrical consumption to train state-of-the-art natural language processing models, shown as powering average U.S.homes for 1 year
Model Name GFLOP Consumption Number of Homes
T5-Small . π T5-Base . π T5-Large . π T5-3B π T5-11B . π BERT-Base . π BERT-Large . πΈ RoBERTa-Base . π RoBERTa-Large . π GPT-3 Small . π GPT-3 Medium . π GPT-3 Large . π GPT-3 XL . π GPT-3 2.7B . π GPT-3 6.7B . π GPT-3 13B . π GPT-3 175B . π This section contains the guidelines which comprise both the βfullβand βliteβ data center energy audits. The guidelines are binned intosix categories, following the naming conventions established in theMaster List [7]:
Cooling Air and Air Management , EnvironmentalConditions , Global , IT Equipment , IT Power Distribution Chain , and
Lighting . The appendix offers two tables, depicting the components whichmake up the βfullβ and βliteβ audits. To reiterate: the "full" auditis intended for large organizations with the resources required toundertake a laborious, and potentially expensive, audit. The "lite"audit framework is meant for smaller organizations with fewerresources, to serve as an accessible option to have the ability to gainan understanding of the state of data center energy efficiency. Theaction items of the "lite" audit framework generally require fewerresources and expertise than large-scale equipment replacementspecified in the "full" audit. To conduct a "lite" audit, only completeitems listed in Table 4. To conduct a "full" audit, an organization must complete items all "lite" (Table 4) and "full" (Table 5) options,depicted in Table 6.
All entries in the proceeding subsections will use the followingstructure: β’ Audit Level : At what level (e.g., individual rack, equipmentroom, entire data center, etc.) shall the analysis cover β’ Item Description : Description of what the audit item isand how to perform it. β’ Metric : How to measure the audit item. β’ Desired Output(s) & Goal(s) :(1) Takeaway(s) from the audit item. β’ Action(s) to Take :(1) What action(s) should follow completion of this audit item(i.e., mitigation strategies to improve efficiency). β’ Audit Type : "Full" or "Lite" item.
Items related to efficiently managing airflow and cool air withinthe data center.
π π πΌ ). The Return Temperature Index (
π π πΌ ) [10] is a key indicator ex-plaining how effectively of the equipment room air-managementsystem is operating. The
π π πΌ reflects either net by-pass air (
π π πΌ < π π πΌ > β’ Audit Level : Perform at the equipment room or entire datacenter level. β’ Item Description : Capture the
π π πΌ for either an equipmentroom in a multi-room data center, or roll up to the entiredata center level. β’ Metric : Percentage: (measure of net by-pass air (
π π πΌ < π π πΌ > Framework for Auditing Data Center Energy Usage and Mitigating Environmental Footprint β’ Desired Output(s) & Goal(s) :(1) Identify which rooms or data centers do not have sufficientairflow management (i.e.,
π π πΌ β β’ Action(s) to Take :(1) Adjust room air management to ensure optimal
π π πΌ (100%). β’ Audit Type : Full.
Aisles of server racks should alternate temperature between hotand cold. This can be achieved by ensuring the equipment is in-stalled into the racks to create a front-to-back pattern of airflow,which will bring in cold air from the HVAC-chilled cold aisles andget rid of hot air via the hot aisles located behind the racks [11]. β’ Audit Level : Perform at the individual aisle level. β’ Item Description : Determine how many aisles of racks arealternating hot and cold temperatures. β’ Metric : Percentage of the aisles compliant with the alternat-ing hot and cold aisle scheme:
πΆππππΏπΌπ΄ππΆπΈ = πππππππππ‘ πππ πππ π‘ππ‘ππ πππ πππ . β’ Desired Output(s) & Goal(s) :(1) Identify which rows do not follow the alternating hot andcold aisle scheme. β’ Action(s) to Take :(1) Re-organize server racks and HVAC system to ensure 100%compliance. β’ Audit Type : Full.
To further reduce mixing the cold supply air with the hot exhaustair (e.g., using flexible strip curtains or rigid enclosures). This, inturn, will reduce fan energy and improve chiller efficiency [7].Accordingto the U.S. Department of Energy [9], this can reduce fan energyrequirements by up to 20% β β’ Audit Level : Perform at the individual aisle level. β’ Item Description : Determine how many aisles of racksprovide physical barriers to other aisles. β’ Metric : Percentage of the aisles compliant physical barrierscheme:
πΆππππΏπΌπ΄ππΆπΈ = πππππππππ‘ πππ πππ π‘ππ‘ππ πππ πππ . β’ Desired Output(s) & Goal(s) :(1) Identify which aisles do not follow the physical barrierscheme. β’ Action(s) to Take :(1) With non-compliant aisles, install a barrier (e.g., strip cur-tain, rigid enclosures, etc.) to ensure 100% compliance. β’ Audit Type : Lite.
Unstructured cable (see Figure 1) can restrict airflow to and fromthe rack, allowing for neither cold air to cool the server nor hot airto expel. Use structured cabling (seeFigure 1) to avoid this. β’ Audit Level : Perform at the individual rack level. β’ Item Description : Determine how many racks within thedata center follow acceptable structured cabling best prac-tices. β’ Metric : Percentage of the racks leveraging structured ca-bling:
πΆππππΏπΌπ΄ππΆπΈ = πππππππππ‘ πππππ π‘ππ‘ππ πππππ . β’ Desired Output(s) & Goal(s) :(1) Identify which racks have unstructured cabling.
Figure 1: An example of unstructured cabling (left) andstructured cabling (right) [12]. β’ Action(s) to Take :(1) With non-compliant racks, re-wire to ensure structuredcabling. β’ Audit Type : Lite.
The Minimum Efficiency Reporting Values (MERV) rating is themeasure of an air filterβs ability to capture larger particles between0.3 and 10 microns ( ΞΌ m) [13]. According to the American Societyof Heating (ASHRAE), Refrigerating and Air-Conditioning Engi-neers [14], recommends that any air entering the data center useMERV 11-13 filters, whereas the air continuously present in thedata center can use MERV 8 filters. β’ Audit Level : Perform at the individual filter level. β’ Item Description : Determine how many air filters withinthe data center are compliant π€ .π .π‘.
MERV rating, for anair filterβs given purpose (i.e., external air vs. continuous airinternal to the data center). β’ Metric : Percentage of the compliant air filters:
πΆππππΏπΌπ΄ππΆπΈ = πππππππππ‘ πππ π πππ‘πππ π‘ππ‘ππ πππ π πππ‘πππ . β’ Desired Output(s) & Goal(s) :(1) Identify which air filters are not compliant. β’ Action(s) to Take :(1) With non-compliant air filters, replace with the appropri-ate MERV-rated air filter. β’ Audit Type : Lite.
Items related to environmental factors, such as: data center ambienttemperature, rack cooling, etc.
One of the simplest, yet effective, measures an organization cantake to immediately improve a data centerβs energy efficiency isanalyze the data centerβs ambient temperature. According to astudy conducted by the U.S. Environmental Protection Agency andthe U.S. Department of Energy [12], many data centers set theirtemperatures much lower than the suggested range. Some datacenters in the study were reported to have ambient temperaturesas low as 55Β°F. The suggested range by ASHRAE [9] is depictedin Table 2. The agencies also report that data centers can save upto 4 β
5% in energy costs for every 1Β°F increase in server inlettemperature. Page 3 ustin A. Gould
Table 2: Data center ambient temperature ranges suggested by ASHRAE [9]
Temperature Class 1 and Class 2 Recommended Range Class 1 Allowable Range Class 2 Allowable Range
Low Temperature Limit
High Temperature Limit β’ Audit Level : Perform at the entire data center level. β’ Item Description : Capture hourly data center ambient tem-perature ranges. β’ Metric :(1) Percentage of the time within the suggested ASHRAErange:
πΆππππΏπΌπ΄ππΆπΈ = ππππππ‘ππππ πππππππ‘ π‘πππππππ‘π’π ππππππππ π‘ππ‘ππ πππππππ‘ π‘πππππππ‘π’ππ ππππππππ .(2) Plot the ambient temperature over time, and aim to benear the high temperature range, depicted in Table 2. β’ Desired Output(s) & Goal(s) :(1) Determine if data center follows ASHRAE guidelines fordata center temperature ranges. β’ Action(s) to Take :(1) Raise the data centerβs ambient temperature to not onlybe within the acceptable ASHRAE suggested temperaturerange, but strive to be near the high temperature limit. β’ Audit Type : Lite.
π πΆπΌ ). The Rack Cooling Index (
π πΆπΌ ) [15] is a key indicator explaininghow effectively individual racks are cooled and maintained. The
π πΆπΌ comprises of two metrics:
π πΆπΌ π»πΌ and π πΆπΌ πΏπ , where π πΆπΌ π»πΌ measures the presence or absence of over-temperatures and π πΆπΌ πΏπ measures the presence or absence of under-temperatures. β’ Audit Level : Perform at the individual rack level. β’ Item Description : Capture the
π πΆπΌ π»πΌ and π πΆπΌ πΏπ for a givenrack. β’ Metric : Percentage for both
π πΆπΌ π»πΌ and π πΆπΌ πΏπ . β’ Desired Output(s) & Goal(s) :(1) Identify which rooms or data centers do not have sufficienttemperature management (i.e.,
π πΆπΌ π‘π¦ππ β β’ Action(s) to Take :(1) Adjust rack temperature management to ensure optimal
π πΆπΌ π»πΌ and π πΆπΌ πΏπ (100%). β’ Audit Type : Full.
High-level items to monitor overall efficiency of energy use, suchas; electrical, water, HVAC, airflow, etc.
ππ πΈ ). The Power Usage Effectiveness (
ππ πΈ ) [9] is a standard metric tomeasure the overall efficiency of power used by the data center. β’ Audit Level : Perform at the entire data center level. β’ Item Description : Capture the
ππ πΈ at an organization-defined cadence to track over time. β’ Metric : The ratio of the total power to run the data centerfacility to the total power drawn by all IT equipment:
ππ πΈ = πππ‘ππ πΉππππππ‘π¦ πππ€πππΌπ πΈππ’ππππππ‘ πππ€ππ . β’ Desired Output(s) & Goal(s) : (1) Benchmark
ππ πΈ per data center; overall current stateof data center energy efficiency, per U.S. Department ofEnergy standards, depicted in Table 3. β’ Action(s) to Take :(1) Once an organization has a benchmark understandingof its
ππ πΈ , it can use the non-metric best practices totake actions to improve
ππ πΈ and track it over time. Thisbenchmark can also be used in coΓΆperation with any exist-ing sustainability strategy from the organization, to drivegoals for future
ππ πΈ values (see section 3). β’ Audit Type : Lite.
π·πΆπΌπΈ ). The Data Center Infrastructure Effectiveness (
π·πΆπΌπΈ ) [9] is theinverse of
ππ πΈ -defined as the ratio of the total power drawn by allIT equipment to the total power to run the data center facility. β’ Audit Level : Perform at the entire data center level. β’ Item Description : Capture the
π·πΆπΌπΈ at an organization-defined cadence to track over time. β’ Metric : The ratio of the total power drawn by all IT equip-ment to the total power to run the data center facility (inverseof
ππ πΈ ): π·πΆπΌπΈ = ππ πΈ = πππ‘ππ πΉππππππ‘π¦ πππ€πππΌπ πΈππ’ππππππ‘ πππ€ππ . β’ Desired Output(s) & Goal(s) :(1) Benchmark
π·πΆπΌπΈ per data center; overall current stateof data center energy efficiency, per U.S. Department ofEnergy standards, depicted in Table 3. β’ Action(s) to Take :(1) Once an organization has a benchmark understandingof its
π·πΆπΌπΈ , it can use the non-metric best practices totake actions to improve
π·πΆπΌπΈ and track it over time. Thisbenchmark can also be used in coΓΆperation with any exist-ing sustainability strategy from the organization, to drivegoals for future
π·πΆπΌπΈ values (see section 3). β’ Audit Type : Lite.
πΈπ πΈ ). The Energy Reuse Effectiveness (
πΈπ πΈ ) [9] is the ratio of the totalenergy to run the data center facility minus the reuse energy to thetotal energy drawn by all IT equipment.
πΈπ πΈ value range: 0 β β .For example, an
πΈπ πΈ of 0 means that 100% of the energy broughtinto the data center is reused elsewhere, outside of the data centercontrol volume. β’ Audit Level : Perform at the entire data center level. β’ Item Description : Capture the
πΈπ πΈ at an organization-defined cadence to track over time. β’ Metric : The ratio of the total energy to run the data center fa-cility minus the reuse energy to the total energy drawn by allIT equipment:
πΈπ πΈ = πΆππππππ + πππ€ππ + πΏππβπ‘πππ + πΌπ β π ππ’π π πΈπππππ¦πΌπ πΈππ’ππππππ‘ πΈπππππ¦ . β’ Desired Output(s) & Goal(s) : Page 4
Framework for Auditing Data Center Energy Usage and Mitigating Environmental Footprint
Table 3: U.S. Department of Energy recommendations [9] for standard data center energy efficiency metrics
Metric Name Standard Score Good Score Better Score
Power Usage Effectiveness (PUE)
Data Center Infrastructure Efficiency (DCIE)
HVAC System Effectivenss (HVACSE)
Airflow Efficiency (AE) π / π π π π / π π π ππ / π π π Cooling System Efficiency (CSE) ππ / π‘ππ ππ / π‘ππ ππ / π‘ππ (1) Benchmark πΈπ πΈ per data center; overall current state ofdata center energy efficiency. β’ Action(s) to Take :(1) Once an organization has a benchmark understandingof its
πΈπ πΈ , it can use the non-metric best practices totake actions to improve
πΈπ πΈ and track it over time. Thisbenchmark can also be used in coΓΆperation with any exist-ing sustainability strategy from the organization, to drivegoals for future
πΈπ πΈ values (see section 3). β’ Audit Type : Lite.
π»π π΄πΆππΈ ). HVAC System Effectiveness (
π»π π΄πΆππΈ ) [9] is the ratio of the an-nual IT equipment energy to the annual HVAC system energy. β’ Audit Level : Perform at the entire data center level. β’ Item Description : Capture the
π»π π΄πΆππΈ annually to trackover time. β’ Metric : The ratio of the annual IT equipment energy to theannual HVAC system energy:
π»π π΄πΆππΈ = ππ β / π¦π πΌπ ππ β / π¦π π»ππ΄πΆ . β’ Desired Output(s) & Goal(s) :(1) Benchmark
π»π π΄πΆππΈ per data center; overall current stateof data center energy efficiency, per U.S. Department ofEnergy standards, depicted in Table 3. β’ Action(s) to Take :(1) Once an organization has a benchmark understanding ofits
π»π π΄πΆππΈ , it can use the non-metric best practices totake actions to improve
π»π π΄πΆππΈ and track it over time.This benchmark can also be used in coΓΆperation with anyexisting sustainability strategy from the organization, todrive goals for future
π»π π΄πΆππΈ values (see section 3). β’ Audit Type : Lite. π΄πΈ ). Airflow Efficiency ( π΄πΈ ) [9] is the overall measure of how efficientlyair is moved through the data center. β’ Audit Level : Perform at the individual fan level. β’ Item Description : Capture the π΄πΈ at an organization-definedcadence to track over time; roll up to the entire data centerlevel. β’ Metric : The overall airflow efficiency in terms of the total fanpower required per unit of airflow: π΄πΈ = πππ‘ππ πΉππ πππ€ππ ( π ) πππ‘ππ πΉππ π΄ππ π πππ€ ( ππ π ) . β’ Desired Output(s) & Goal(s) :(1) Benchmark π΄πΈ per data center; overall current state of datacenter energy efficiency, per U.S. Department of Energystandards, depicted in Table 3. β’ Action(s) to Take : (1) Once an organization has a benchmark understanding ofits π΄πΈ , it can use the non-metric best practices to take ac-tions to improve π΄πΈ and track it over time. This benchmarkcan also be used in coΓΆperation with any existing sustain-ability strategy from the organization, to drive goals forfuture π΄πΈ values (see section 3). β’ Audit Type : Lite.
πΆππΈ ). Cooling System Efficiency (
πΆππΈ ) [9] is a metric to measure theefficiency of an HVAC system. β’ Audit Level : Perform at the individual HVAC system level. β’ Item Description : Capture the
πΆππΈ at an organization-defined cadence to track over time; roll up to the entiredata center level. β’ Metric : The ratio of average cooling system power usage( ππ ) to the average data center cooling load (tons): πΆππΈ = π πΆππππππ ππ¦π π‘ππ πππ€ππ ( ππ ) π πΆππππππ πΏπππ ( ππ π ) . β’ Desired Output(s) & Goal(s) :(1) Benchmark
πΆππΈ per data center; overall current state ofdata center energy efficiency, per U.S. Department of En-ergy standards, depicted in Table 3. β’ Action(s) to Take :(1) Once an organization has a benchmark understandingof its
πΆππΈ , it can use the non-metric best practices totake actions to improve
πΆππΈ and track it over time. Thisbenchmark can also be used in coΓΆperation with any exist-ing sustainability strategy from the organization, to drivegoals for future
πΆππΈ values (see section 3). β’ Audit Type : Lite.
Items related to the performance and utilization of servers andother IT equipment.
Running unused operational servers-or any IT equipment-can beexpensive and wasteful of resources. β’ Audit Level : Perform at the entire data center level. β’ Item Description : Determine which equipment are notused within the data center. β’ Metric : Percentage of the equipment unused: π’ππ’π ππ πππ’ππππππ‘ π‘ππ‘ππ πππ’ππππππ‘ . β’ Desired Output(s) & Goal(s) :(1) Identify which operational servers are not in use and con-sider them candidates for retirement. β’ Action(s) to Take : Page 5 ustin A. Gould (1) Determine which unused operational servers to retire, toensure all servers that are drawing energy are in use. β’ Audit Type : Lite.
Understand server performance π€ .π .π‘.
CPU utilization. IgnacioCano et al. [16] performed a study in which 80% of devices in asample of VMs have a maximum resource usage that is < β’ Audit Level : Perform at the individual server level. β’ Item Description : Determine if a server is under- or over-utilized: β Under-utilization [16]: π’π πππ < = β Correct utilization [16]: 51% < = π’π πππ < = β Over-utilization [16]: π’π πππ < = β’ Metric : Percentage of the CPU utilized. β’ Desired Output(s) & Goal(s) :(1) Identify which operational servers are under-utilized orover-utilized. β’ Action(s) to Take :(1) Optimize server performance to avoid under-utilization(poor energy management) and over-utilization (wear andtear).(2) Consolidate under-utilized servers. β’ Audit Type : Full.
Understand how performant current IT equipment is π€ .π .π‘. stan-dard metric [4] of
πΊπΉπΏππ / π . β’ Audit Level : Perform at the individual server level. β’ Item Description : Determine how energy efficient a serveris. β’ Metric : πΊπΉπΏππ / π . β’ Desired Output(s) & Goal(s) :(1) Identify which equipment have a low value of
πΊπΉπΏππ / π . β’ Action(s) to Take :(1) Procure energy efficient servers to replace inefficient equip-ment. β’ Audit Type : Lite.
Items related to the efficiency of systems providing the data centerwith power.
Understand from where power fed to the data center are coming.Over time, non-renewable power generation sources should bephased out in favor of replacing with renewable sources. β’ Audit Level : Perform at the entire data center level. β’ Item Description : Determine if an energy supply for thedata center is renewable or non-renewable, and how muchpower is coming from each source. β’ Metric : Percentage of power from renewable vs. non-renewablesources:
πππ‘ππ ππ β π πππ πππππ€πππππππ‘ππ ππ β ππππ€π β’ Desired Output(s) & Goal(s) : (1) Identify how electrical power is generated for the datacenter. β’ Action(s) to Take :(1) Work with the organizationβs facilities group to create astrategy to phase out non-renewable sources of power infavor of renewable sources. The goal is 100%. β’ Audit Type : Lite.
Items related understanding the impact of lights, lamps, and bulbsthroughout the data center.
A quick and easy way an organization can reduce a data centerβsenvironmental footprint is by looking at the bulbs used in its lightsand lamps. According to the U.S. Department of Energy [17], LEDbulbs required 75% less energy than traditional incandescents. β’ Audit Level : Perform at the entire data center level. β’ Item Description : Determine how many light bulbs in thedata center are LEDs. β’ Metric : Percentage of LED bulbs vs. traditional incandes-cents:
πΏπΈπ· ππ’πππ
πππ‘ππ ππ’πππ β’ Desired Output(s) & Goal(s) :(1) Identify which lamps do not contain LED bulbs. β’ Action(s) to Take :(1) Replace all non-LED bulbs with LED. The goal is 100%. β’ Audit Type : Lite.
Outside of replacing all non-LED bulbs with LEDs, one way anorganization can reduce energy costs within a data center is toenable lights and lamps to operate at a lower level. β’ Audit Level : Perform at the entire data center level. β’ Item Description : Determine how many lamps in the datacenter offer dimming. β’ Metric : Percentage of lamps which support dimming: β πΏππππ ππ π ππ πππππππ
πππ‘ππ πππππ β’ Desired Output(s) & Goal(s) :(1) Identify which lamps do not offer dimming. β’ Action(s) to Take :(1) Install dimming controls on all non-dimming lamps andlights. The goal is 100%. β’ Audit Type : Lite.
This is very closely related to the previous best practice. Sensorsto determine if a person is present or absent, and turning lights offwhen not needed, can help save energy and costs in operating thedata center. β’ Audit Level : Perform at the entire data center level. β’ Item Description : Determine how many lamps in the datacenter offer control via occupancy sensors. β’ Metric : Percentage of lamps which support control via oc-cupancy sensors:
πΏππππ π€ππ‘β π πππ πππ
πππ‘ππ πππππ β’ Desired Output(s) & Goal(s) :(1) Identify which lamps do not offer control via occupancysensors. Page 6
Framework for Auditing Data Center Energy Usage and Mitigating Environmental Footprint
Figure 2: An example of a data centerβs energy efficiencydashboard, from the National Renewable Energy Labora-toryβs ESIF High Performance Computing Data Center [21]. β’ Action(s) to Take :(1) Install occupancy sensors on all non-equipped lamps andlights. The goal is 100%. β’ Audit Type : Lite.
Having a framework to conduct a data center energy efficiency auditis only the beginning in the journey for an organization to make itsdata centers more environmentally friendly. An organization mustalso have a team in place to execute the audit and make changesonce benchmark performance data are collected and a strategy tomake necessary changes to data centers-typically to contributeto an ongoing sustainability campaign (such as Googleβs [18], Mi-crosoftβs [19], or Facebookβs [20] commitments to become moresustainable organizations). The goal of this section is to provide theinformation necessary to help organizations in this space to ensurea successful energy audit, as well as enable the organization post-audit to make the changes necessary to improve energy efficiencyand reduce the environmental impact.
The Lawrence Berkeley National Laboratoryβs Center of Expertisefor Energy Efficiency in Data Centers provides some insight onthe creation of a team to execute some of the best practices forenergy-efficient data centers [7]; however, given the ambiguity ofthe energy audit criterion described, I wanted to provide sugges-tions to complement the Lawrence Berkeley National LaboratoryβsCenter of Expertise for Energy Efficiency in Data Centersβs sugges-tions [8].Roles: β’ Audit Lead : The Audit Lead will coΓΆrdinate with Site Lead(s)and Organization Engagement Lead/Liaison to organize au-dit and communicate results to responsible parties within theorganization. The Audit Lead will also distribute work to theAudit Assessors and meet regularly to track audit progressand understand benchmark performance. β’ Site Lead(s) : This is recommended by the Lawrence Berke-ley National Laboratoryβs Center of Expertise for EnergyEfficiency in Data Centers [7]. The lab defines this role as:"Data center technical representative that will be the primaryperson participating in the assessment." This individual willlead the Site Consulting Engineer(s) in any technical workthat needs to be completed in the data center for the pur-poses of the audit, and will report findings back to the AuditAssessor(s) and Audit Lead. β’ Audit Assessor(s) : This is recommended by the LawrenceBerkeley National Laboratoryβs Center of Expertise for En-ergy Efficiency in Data Centers [7]. The lab defines this roleas: "The energy expert assigned to complete the...assessment;the expert serves as the facilitator for all activities." Depend-ing on the size of the organization, > β’ Site Consulting Engineer(s) : At least one Data Engineerper data center to provide information on infrastructure, in-stalling sensors to collect data, handling operational servers,etc. β’ Data Scientist(s) : At least one Data Scientist to collect,parse, visualize, and communicate data required to success-fully execute the audit. Suggested action: create dashboardsto report, track, and monitor data center performance overtime-not only during the audit to benchmark performance.An example of this dashboard can be seen in Figure 2. β’ Organization Engagement Lead/Liaison : Serves as thepoint of contact between organization management (e.g., forfunding, marketing/communication/organizational public-ity), facilities (e.g., for power generation, data center lightingcontrol, etc.), and sustainability (e.g., for contributing to anorganizationβs ongoing sustainability strategy).
Once the initial data center energy efficiency audit is complete, theorganization will understand the environmental impact its datacenter(s) are having. At this time, it is prudent for the organizationto make changes necessary to improve energy efficiency and re-duce the environmental footprint of the data centers. Outside of thetaking the recommended actions based on the result of the items inthe audit (described in length in the previous section), the organi-zation should continue to track and monitor its energy efficiencyat a regular cadence (quarterly reporting is recommended, per [18];however, a far more granular cadence of collecting these data canenable the organization to leverage techniques like machine learn-ing to optimize the data centerβs parameters to maximize energyefficiency [22]).An important consideration to make is setting annual goals forenergy efficiency improvement, as well as reporting mechanisms tocommunicate these improvements. As seen in the examples fromGoogle [18], Microsoft [19], and Facebook [20], these organizationshave set ambitious environmental goals for the near future to reachcarbon-free energy generation, carbon negativity, and net zeroGHG emissions, respectively. However, smaller organizations-andPage 7 ustin A. Gould higher education institutions-can set similar goals to improve its en-vironmental footprint. For example, Purdue University is workingtoward its Sustainability Master Plan [23], consisting of goals re-lated to energy, water, materials/waste, buildings, and the groundsof the University to reduce the impact operations have on the en-vironment. To demonstrate that a data center energy efficiencyinitiative contributes to a strategy such as this:Sustainability Master Plan Initiatives: β’ Energy [24]: β E-1 β Cut Carbon Emissions in Half : "Reduce...carbon emis-sions by 50% by FY25, with FY11 as the baseline year."(1) Improving data center energy efficiency (decreasing
ππ πΈ , for example) will reduce the required power drawnfrom power sources potentially contributing to an orga-nizationβs carbon emissions. Actions as trivial as dim-ming data center lights when not in use contributes tothis goal.(2) Over time, matriculating a data centerβs power genera-tion sources from non-renewable sources to renewablesources can reduce an organizationβs overall carbonemission footprint. β E-2 β No Net Gain : "Cap total energy consumption at FY11levels."(1) Improving data center energy efficiency (decreasing
ππ πΈ , for example) will reduce the required power drawnfrom power sources potentially contributing to an orga-nizationβs carbon emissions. Actions as trivial as dim-ming data center lights when not in use contributes tothis goal.(2) Over time, matriculating a data centerβs power genera-tion sources from non-renewable sources to renewablesources can reduce an organizationβs overall carbonemission footprint. β’ Water [25]: β W-1 - Reduce Water Consumption by 30% : "Reduce potablewater consumption inside buildings and for irrigation by30% by FY25 on a gallon per square foot basis, with FY11as the baseline year."(1) Employing best practices to improve rack cooling effi-ciency and optimizing operational servers in use canreduce the amount of chilling required for the data cen-ter, thus decreasing water consumption.
REFERENCES
ASHRAE Trans
ASHRAE Transactions
Page 8
Framework for Auditing Data Center Energy Usage and Mitigating Environmental Footprint
Table 4: Items included only in the "lite" data center energy efficiency audit.
Item Title Audit Type
Physical Separation of Hot and Cold Air of Rack Aisles
Lite
Structured Cabling for a Rack
Lite
Air Filter MERV Rating Compliance
Lite
Data Center Ambient Temperature
Lite
Power Usage Effectiveness
Lite
Data Center Infrastructure Efficiency
Lite
Energy Reuse Effectiveness
Lite
HVAC System Effectiveness
Lite
Airflow Efficiency
Lite
Cooling System Efficiency
Lite
Identify Unused Operational Servers
Lite
Equipment Efficiency
Lite
Identify Power Sources
Lite
LED Bulbs
Lite
Lighting Control and Dimming
Lite
Occupancy Sensors to Control Lights
Lite
Table 5: Items included only in the "full" data center energy efficiency audit.
NOTE: To complete a "full" audit, the items from"lite" an "full" must both be completed.
Item Title Audit Type
Return Temperature Index
Full
Alternating Hot and Cold Rack Aisles
Full
Rack Cooling Index
Full
Monitor Server CPU Utilization
Full
Table 6: All audit items, with their audit type classification.
Item Title Audit Type
Return Temperature Index
Full
Alternating Hot and Cold Rack Aisles
Full
Physical Separation of Hot and Cold Air of Rack Aisles
Lite
Structured Cabling for a Rack
Lite
Air Filter MERV Rating Compliance
Lite
Data Center Ambient Temperature
Lite
Rack Cooling Index
Full
Power Usage Effectiveness
Lite
Data Center Infrastructure Efficiency
Lite
Energy Reuse Effectiveness
Lite
HVAC System Effectiveness
Lite
Airflow Efficiency
Lite
Cooling System Efficiency
Lite
Identify Unused Operational Servers
Lite
Monitor Server CPU Utilization
Full
Equipment Efficiency
Lite
Identify Power Sources
Lite
LED Bulbs
Lite
Lighting Control and Dimming
Lite