Achieving Energy Proportionality in Data Centers

In a recent article titled Power, Pollution and the Internet, The New York Times investigated a well-known and persistent problem: “Most data centers, by design, consume vast amounts of energy in an incongruously wasteful manner, interviews and documents show.  Online companies typically run their facilities at maximum capacity around the clock, whatever the demand.  As a result, data centers can waste 90 percent or more of the electricity they pull off the grid, The Times found.”Such waste came under scrutiny nearly a decade ago, and most organizations have taken at least some steps to improve energy efficiency in their data centers.  But very few have achieved what should be the ultimate goal: energy proportionality.  That is, the energy being consumed is directly proportional to the work being performed by the servers in the data center.  There are many steps organizations can (and should) be taking to achieve energy proportionality, and the four that deliver the best results are highlighted here.

Step 1: Consolidate and Virtualize the Servers

Dedicated servers have a typical utilization rate of only around 10 percent, representing what is perhaps the most egregious waste of energy in data centers today.  Most organizations have, therefore, consolidated and virtualized most of their servers already.  The motivation may or may not have involved energy savings, as virtualization is also an excellent way to increase utilization by using some of that 90 percent “idle” capacity in dedicated servers.

Nevertheless, taking the step to consolidate and virtualize servers does improve energy proportionality by increasing utilization to between 20 and 50 percent in most data centers.  And with the cost to power a server over its useful life now exceeding the cost to purchase it, the energy savings contributes significantly to the overall cost savings afforded by virtualization.

Step 2: Use Energy-efficient Servers

Energy efficiency is sometimes not considered an aspect of energy proportionality, but it should be.  To understand why, consider the analogy of commuting to work—alone—in either a pickup truck or compact car.  The far superior gas mileage of the compact car results in a more “energy proportional” commute, and this same thinking should apply to servers.

To promote the use of energy-efficient systems in data centers, the U.S. Environmental Protection Agency created an ENERGY STAR rating for servers and other IT equipment.  This is a step in the right direction, but the rating has only limited value.  The reason is: an ENERGY STAR rating is given to the 25 percent most efficient servers from a specific vendor, so depending on the servers tested, the manufacturer can influence which models get rated. Furthermore, the rating does not factor in the age of the equipment nor does it show the year of measurement.  With an average of two-times improvement in server performance every two years, this is a serious shortcoming.

A far better measure of energy efficiency for servers is transactions per second per Watt (TPS/Watt), and this metric is available using the standard power measurement procedure (UL2640) from Underwriters Laboratories.  UL2640 utilizes the PAR⁴ Efficiency Rating system to determine TPS/Watt and other useful metrics, such as idle power consumption under no load.  These metrics enable IT managers to compare the transactional efficiency of legacy servers with newer ones, and newer models of servers with one another.  And a best practice today is to assess energy efficiency during every hardware refresh cycle and whenever adding capacity.  In addition, PAR⁴ publishes idle and peak power consumption, which is essential to optimize the utilization of available power in each rack.

Step 3: Optimize Power Distribution

Another source of waste in data centers involves power distribution, and a symptom of this problem is the extent of stranded power.  Most IT departments configure racks based on the nameplate power ratings of servers or a percentage based derating, and even if a rack has available slots, it is considered “full” when the total rated maximum power consumption of the servers matches the power distributed to the rack.  But because nameplate ratings are notoriously conservative, this approach results in available power (and space) being underutilized.

Optimization minimizes the energy being wasted by the inefficiencies inherent to any power distribution system, and can also reclaim stranded power and space to facilitate additional capacity in the data center.  Because the UL 2640 standard also measures actual server power consumption under peak application load, it can be used to minimize or eliminate stranded power by more fully populating racks.  Power Assure estimates that most data centers should be able to increase overall transactional capacity by 40-50 percent by taking this step.

Step 4: Match Capacity to Load Dynamically

Even with the use of the most energy-efficient, highly-virtualized servers, energy continues to be wasted during periods of low application demand.  This is why the ultimate step IT managers can take toward achieving energy proportionality is to abandon the wasteful “always on” mode of operating servers and to instead adopt a fully efficient “on demand” approach.  The tool needed to make this change is a Data Center Infrastructure Management (DCIM) system capable of performing real-time management of application workloads, such as Power Assure’s EM/4 or ABB’s Decathlon.

The “on demand” approach to server utilization involves continuously and dynamically matching available server capacity in a virtualized cluster to the current application load.  The goal is to power up servers only when they are needed to perform real work, and the effort can reduce total server energy consumption by over 50 percent in most data centers.

Automated runbooks are normally used to handle the steps involved in resizing clusters and/or de-/re-activating servers.  Different versions of runbooks can be created for use on a predetermined schedule and for dynamic response to changing loads.  Utilization rates for the active servers can be 80 percent or more with such dynamic load management, making this the most energy proportionate way to support variable application loads.

Matching server capacity to load is achieved within the framework of Software Defined Data Center (SDDC), where all IT infrastructure is virtualized and applications are delivered as a service, whether in house or from a colocation or hosted environment.  In the future, enhanced versions of DCIM systems, like the ones Power Assure is working on, will need to support “Software Defined Power” by creating a layer of abstraction with the power and cooling infrastructure to shift application capacity within and across data centers. This approach guarantees the highest level of reliability, as there is always spare capacity, and shifting and shedding occurs automatically as hardware is dynamically turned on and off depending on the application demand.

Clemens Pfeiffer is the CTO of Power Assure and is a 25-year veteran of the software industry, where he has held leadership roles in process modeling and automation, software architecture and database design, and data center management and optimization technologies.