Headlines tout mega data center campuses (e.g. Meta’s 2GW Louisiana campus) as if they energize at full tilt on day one. In reality, these sites ramp over years, and the binding constraint is power delivery.

Obtaining GPUs is hard. Getting firm, timely power is harder. Grid incentives don’t match developer timelines - a point I made in a paper back in 2023:

[grid operators] are under a reliability obligation to provide energy to existing customers, which means they have a conservative approach to network changes. In contrast, data center operators want to bring on new capacity quickly to address customer demand, and housing developers want to address the social and political imperative to build more.

The headlines obsess over how much electricity AI will use. The real bottleneck is simpler: can the grid deliver power to the right place at the right time?

Power transmission you can’t use

The UK built a lot of wind in northern Scotland while most demand sits in southern England. Moving that power south requires long-distance transmission, which is where congestion hits.

When there’s plenty of electricity but not enough transmission capacity, system operators turn down generators to avoid overloading lines - this is known as curtailment - and it’s expensive. The operator pays wind farms to reduce output and pays other units (often gas) to ramp via the Balancing Mechanism. These constraint costs flow to consumers.

An analysis of the two main Scotland / England transmission corridors (B4 / B6) suggests that £1.65BN of curtailment costs Jan 2024 to Apr 2025 could have been 25% lower if just an additional 500 MW more transfer capacity had been available. The reduced capacity was caused by planned upgrades so average usable capacity across the key boundaries sat around ~60% in 2024, with long stretches nearer ~40%. Congestion costs exploded as a result.

The timeline to increase capacity is long. New links were planned back in 2015, but are only expected to be completed in 2029.

Similar challenges exist in the US. Dominion Energy paused new data-center connections in 2022 due to transmission constraints, then later resumed with limits until new lines are in place (until a 500 kV reinforcement lands in 2026). In 2023, Virginia’s DEQ even floated a diesel-generator variance so data centers could keep running during grid stress while upgrades finished. These workarounds are common.

Clever workarounds

xAI became famous for how quickly it built its first Colossus data center, becoming the largest cluster in the world (~300 MW with 200k GPUs) in just 122 days. It also became infamous for using temporary gas turbines to generate power (some argued were illegally installed without permits) because of the lack of power capacity in Memphis, Tennessee.

Working around the long grid timelines with ad-hoc, on-site power generation seems to be part of the plan for Colossus 2. Even though Tennessee has refused to allow more gas generators, xAI has acquired a power plant over the border in Mississippi, deployed temporary power generators there, and connected them with medium voltage power lines.

The goal is to build a 1GW data center and the only way to do it quickly enough is to build your own power generation.

Can we build?

Building new capacity in a sustainable way is the biggest challenge faced by grid operators. The median wait for US interconnection is now 5 years (with 2.6TW in interconnection queues) and in the UK new applications have been paused. When the market is demanding new capacity, it’s inevitable that the workarounds are going to cause problems.

Which leaves the question: what actually fixes this?

For buyers, “cheap green kWh” is the wrong metric: contracts must include deliverability. Some will call for flexible compute so data centers can participate in demand response, but that’s mostly unworkable.

Governments should stop awarding mega-campus tax breaks before firm deliverability is secured. They should make any subsidy contingent on infrastructure improvements, like we’ve seen with Meta’s data center.

Near term, expect more self-provisioned power (often gas, because it’s dispatchable and financeable) while transmission and substations catch up. Long term, nuclear is a perfect fit with data center load profiles, as shown by Microsoft.

Whether incumbents (ISOs/TSOs and regulated utilities) remain in the critical path will come down to permitting reform. If not, large buyers will keep going around them.

Following high level AI environmental numbers from OpenAI and Mistral, Google just published a detailed technical report about the environmental impact of Gemini AI.

Let’s walk through the paper.

System boundary

This report provides real numbers - not estimates - energy per prompt is metered in production and emissions and water are computed from fleet‑average factors (market‑based emissions, on‑site WUE) with amortized embodied hardware. Unlike other publications which focus on theoretical modeling or measurements on limited scale systems, Google has included the full serving stack (under their operational control):

• Measured power supply across TPUs and GPUs.

• Actual utilization in the production fleet.

• Host CPU and memory power.

• Provisioned idle capacity included for reliability/low‑latency.

• Campus‑level PUE applied (data‑center overhead).

Excluded from their methodology are:

• Networking: external networks are out of scope; internal DC networking is assessed as negligible (but no quantitative bound published).

• End user devices, also because this is outside of Google’s system boundary.

• Training and data storage. This is a major exclusion if your goal is to consider the full lifecycle impact (like Mistral did). However, as AI shifts to more “thinking” models which involve test-time compute, the relevance of inference is increasing.

There are very few organizations operating AI at production scale. Google is one of them, which means we can now see the environmental impact of a real AI system.

Metrics

Using Google’s “Comprehensive Approach” which uses metrics across their data center fleet, they calculate the following metrics as of May 2025:

• Energy: 0.24 Wh per prompt

• GHG emissions: 0.03 gCO2e per prompt

• Water consumption: 0.26 mL per prompt

A breakdown of energy consumption by component shows that 58% of the energy goes into the TPUs and GPUs, 24% on CPU and memory, with 18% for operational redundancy and the data center itself.

What is a prompt?

A precise definition of “prompt” is missing, but the technical paper makes several interesting comments:

They “measure total user prompt count across Gemini Apps products for each LLM model” and then “divide each energy component by total user prompt count over the same measured time period to get energy/prompt”.

They also: “measure over a day or longer. This is sufficiently longer than the prompt duration, so partial prompt counts do not measurably impact the results.”

From this:

We find that the distribution of energy/prompt metrics can be skewed, with the skewed outliers varying significantly over time. Part of this skew is driven by small subsets of prompts served by models with low utilization or with high token counts, which consume a disproportionate amount of energy. In such skewed distributions, the arithmetic mean is highly sensitive to these extreme values, making it an unrepresentative measure of typical user’s impact.

This is a pragmatic approach because it considers various types of user text prompts across multiple models, rather than focusing on a single model and specific token lengths. The number of tokens expands as a conversation progresses, so prolonged usage might skew the results especially if you use a mean figure.

Medians reduce sensitivity, but prompt length and low‑utilization conditions still drive the tail - publishing percentiles and per‑token figures would improve comparability.

What doesn’t this tell us?

I’m impressed by the level of detail in Google’s technical report, but it doesn’t tell us everything. There are still questions I’d like to ask:

• Emissions calculations are market-based rather than location-based. This makes sense because the calculations are aggregated across Google’s global data center fleet so they are justified in considering their global emissions. Differences in efficiency and carbon intensity through local grid mixes will all be considered in the aggregate number. However, being able to compare location-based emissions would help understand local impact and inform discussions about clean energy purchasing policies. Otherwise it’s too easy to criticize the results as “hiding” emissions in carbon accounting.

• Location is also relevant for total water consumption. It’s common for water metrics to only include on-site water (typically used for data center cooling), but a larger component of total water consumption comes from energy generation. Google only includes on-site water consumption.

• “Gemini Apps” is ambiguous and not comprehensive. Google has many AI products including Gemini chat, the Google Cloud Gemini APIs, AI Mode & AI Overview in Google Search. The methodology says it covers “all LLM models serving the Gemini app”, which excludes other AI products.

• It specifically only covers text prompts. Multi-modal is increasingly common when analyzing files, generating images, and producing video.

• Training is excluded. This has so far been where the majority of the energy consumption comes from because of the compute-intensive long running jobs involved.

• Google only reports on their owned and operated data center fleet, but they use many other facilities. For example, Google Cloud has a London region but that doesn’t appear in their fleet list because it's operated by a third-party. How efficient are these facilities and are they included in this reporting methodology?

What does this tell us?

Overestimating data center energy has been a hobby of many academics and journalists for over a decade.  I’ve written about how many widely cited estimates overstate data‑center energy, water, and other impacts due to outdated assumptions and extrapolation.

Instead, what has actually been happening is:

• Multiple models are used for different use cases and phased out as newer generations come in. The exact models aren’t explained, but this is a safe assumption given the number of newly announced models over the analysis period.

• New models are significantly more efficient than older models (as reported by Google in 2022). Google’s paper says this amounts to a 33x reduction in per prompt energy consumption driven by software efficiencies (23x from model improvements and 1.4x from improved hardware utilization).

• Data center operations are becoming highly efficient, particularly with specialist hardware like TPUs. Google’s paper shows 1.4x improvement in emissions intensity from data center electricity consumption through choosing workload location and clean energy consumption.

As the media hype I predicted continues to exaggerate the environmental impact of AI, we’re now seeing companies respond with real data. This technical report avoids making puritan value judgments by avoiding misleading terms like “thirsty”, “drink”, “power-hungry”, etc. all of which distract from the engineering science.

Google’s technical report is the first time we have a breakdown of an in-production, global fleet-level system. It provides a comprehensive serving boundary and offers concrete deltas over the previous year. The paper is transparent about skew and justifiably uses medians, but not publishing distributional stats e.g. P90/99 or per‑token metrics limits comparability and could hide the worst‑case load.

There is a huge buildout of AI infrastructure under way and it’s likely that total energy consumption related to AI is going to grow. However, I expect efficiency gains, batching, and load‑placement should temper growth versus extrapolations.

The misconception is that AI is destroying the planet, but that is partially the fault of the AI companies because they’ve not been transparent in their energy, water, and emissions data. This is starting to change.

Mistral Large 2, the flagship LLM released in 2024 with a 128k context window and 123 billion parameters, now has a lifecycle analysis (LCA) detailing its environmental impact.

The assessment covers greenhouse gas (GHG) emissions, water consumption, and materials use. Training and inference together account for most of the GHG and water impacts, while hardware manufacturing dominates materials impact.

This aligns with end-user devices vs. servers. For consumer devices like phones and laptops, most of the footprint comes from manufacturing, meaning the best way to reduce impact is to extend device lifetimes (not pointless actions like deleting emails). By contrast, servers have smaller share of their overall footprint from manufacturing, since those emissions are amortized over 5 - 6 years of use. Their primary impact comes from ongoing energy use.

Mistral reported aggregated numbers for training and usage over the last 18 months:

• GHG emissions: 20.4 ktCO2e

• Water consumption: 281,000 m3

• Resource depletion: 660 kg Sb eq (used as a proxy for resource depletion)

They did not publish the full analysis or separate training from inference. However, they did calculate the marginal impacts of inference based on using their Le Chat AI assistant for a 400-token response:

• GHG emissions: 1.14 gCO2e

• Water consumption: 45 mL

• Resource depletion: 0.16 mg Sb eq

These numbers are useful to understand the aggregate impact so we can compare them to other activities. For example, in 2020 the European average video streaming carbon footprint was calculated to be 55 gCO2e.

Depending on usage intensity, an hour of continuous Le Chat queries could approach a similar order of magnitude as video streaming. However, the comparison isn’t exact: Mistral’s figures include upstream emissions but exclude the user’s device, whereas this was included in the video streaming footprint. The numbers are directionally useful, but not strictly comparable - we’d need the full LCA report to know for sure.

It’s encouraging to see this transparency, especially with a third-party-audited LCA. Still, I find the omission of underlying energy consumption data frustrating. ChatGPT’s 0.34 Wh/query had the opposite problem: an energy figure without environmental context.

Both energy use and environmental impact need to be reported together. Energy data enables efficiency tracking, while impact data depends on the electricity mix of the data center location. Mistral is building their own data center in France, which will be interesting to learn more about because of France’s high nuclear mix.

This situation reminds me of the early days of data center energy reporting where numbers were calculated by third parties using proxies, and often wildly overestimated. At least this time we’re getting some numbers directly, just not enough to properly analyze them.

In a June 10 post on his personal blog, OpenAI CEO Sam Altman wrote:

People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours, about what an oven would use in a little over one second, or a high-efficiency lightbulb would use in a couple of minutes. It also uses about 0.000085 gallons of water; roughly one fifteenth of a teaspoon.

This is several orders of magnitude less than most estimates we’ve seen in the media (mainstream and academic). Now we have a real number, perhaps the overestimates of AI energy and water consumption will stop?

What’s missing?

Although it’s good to have a number to compare to estimates, there are plenty of things missing from this quote. To use this number meaningfully, we need context:

• What is an average query? One exchange? A full conversation thread? What happens if you trigger a query of ChatGPT’s search index vs its search bot that goes out to websites in real time?

• How does multi-modal affect this? What is the impact of analyzing a PDF or generating an image?

• Is energy from supporting systems (storage, networking, cooling) included?

• Which model is being measured? GPT-3.5? GPT-4o? Some are far more compute-intensive than others.

• Is this just for inference? Does it consider the resources required for training?

• Which year is this for? Is it an estimate over all usage for that time period? How has it changed over the years?

• Water usage varies by location. Is this a global average, or specific to a data center region?

• Is this “water consumption” direct (evaporative cooling) or indirect (e.g., hydroelectric power generation)?

• What about carbon? With a specific location we can better understand the electricity mix and create carbon footprint estimates.

What does it mean for future AI energy usage?

Energy intensity shows per-query impact. But it says nothing about total energy use, which depends on scale. Over the last decade we’ve seen significant improvements in IT energy efficiency which has meant that total data center energy consumption has grown only slightly even with major expansion in usage.

I expect to see continuing improvements in the energy efficiency of AI, but the total energy consumption is likely to increase because of the huge usage AI is seeing. This is shown in the 2024 US Data Center Energy Report estimates of 325 to 580 TWh (6.7% to 12% of total electricity consumption) in the US by 2028, mostly driven by AI.

The puritan movement has moved on from the environment impact of video streaming because AI is the new trendy thing to attack. The major players like OpenAI haven’t helped themselves by remaining so secretive about their energy and water footprint, but this is a small step in the right direction. I hope we’ll see more official metrics (not just a blog post from the CEO) and details on some of the open questions above!

The politicization of climate change has clouded sustainable computing research, often leading to exaggerated claims about technology’s environmental impact. We can see this from the mostly worthless body of literature (over)estimating data center energy consumption1, most recently with an AI-focus.

This distracts from real challenges sourcing enough clean energy, optimizing data center water consumption with the appropriate location context, and dealing with the materials challenges in equipment manufacturing, and recycling.

Amid this noise, I was pleased to read the online publication of Herglotz et al. (2025) on measuring power and energy consumption in video communication. It offers a refreshing, rigorous approach with some interesting conclusions around the appropriateness of using software modeling vs hardware power analyzer devices in different scenarios.

This post discusses how studies like Herglotz et al. (2025) can refocus the field.

Challenges with measuring video energy consumption

Video streaming, a massive driver of network traffic, involves complex systems with multiple hardware and software components. This complexity makes accurate energy measurement challenging, as errors compound across the end-to-end system. This has made it an interesting subject not just for research, but for scrutiny of the environmental impact. The involvement of the natural enemies of the mainstream media, Big Tech, has also made it the perfect target for attack.

The sophistication of these systems is something I spent a lot of time describing in my 2024 paper describing how to model the power consumption of networks, but even that used a simplified example. For real systems like video streaming, there are multiple components making up not just the network hardware but the software components within a data center region. 

This recent, detailed description of Meta’s infrastructure further strengthens the point I repeatedly make about how energy intensity (kWh/GB) only being a useful metric when looking backwards at known total power consumption and data volumes - it cannot be used to make predictions or estimate the current power consumption of a system.

All this is to say that energy estimates of complex end to end systems are highly likely to contain serious errors which compound, something I saw in my review of data center energy estimates between 2007 and 2021.

Measure for absolute accuracy; model for relative comparisons

Herglotz et al. analyze video decoding energy use, comparing software-based RAPL estimates with direct measurements from a Tektronix power analyzer. RAPL, a model built into Intel processors which I have written about previously, estimates energy based on activity counters but lacks transparency and precision. 

Real measurements, like those from the power analyzer, offer absolute accuracy—something rare in this field. This distinction matters: models like RAPL are fine for optimization comparisons, but only measurements reveal true energy use.

The paper notes that software modeling, like RAPL, is useful when making comparisons for optimization, but contains inherent limitations if used for producing absolute calculations. This is the distinction between measurement and estimation.

Not at all surprising when you think about it, but few publications use real measurements! It’s rare to see this level of detail, including full analysis of the statistical methods that can be used for discussing uncertainty. Again, something I rarely see when reading publications that purport to provide real numbers behind specific activities.

Conclusions

Herglotz et al. stand out for their scientific rigor - offering a concise literature review, a well-defined focus on video decoding, and a methodology that avoids hype. Their use of real measurements and statistical analysis provides a model for tackling complex systems, countering the overestimations that plague the field. 

This approach should guide future research, ensuring sustainable computing studies prioritize accuracy over sensationalism. It’s no surprise that some of the authors were also involved in the 2021 Carbon Trust report on the carbon impact of video streaming, which I consider to be the definitive analysis.

We long ago reached the point where software systems such as video streaming evolved past their ability to be modelled without detailed inside information. The academic literature has still not fully caught up - there are always opportunities for better transparency - but Herglotz et al. shows how to analyze specific components of those systems.

Discussion about the environmental impact of computing tends to focus on energy and carbon. These are easy to understand because the goal is to get to (net) zero carbon, improve energy efficiency, and transition energy generation to clean sources like nuclear and renewables.

But energy & carbon aren’t the only environmental impacts of computing.

Water consumption is more complex because it’s not necessarily about getting to zero. The location context matters. Efficiency is always good, but there are many locations with abundant supply so this doesn’t need to be top priority. However, there are also regions of water stress (sometimes within the same country) where it really is important.

Siddik, Shehabi, & Marston (2021) discusses this in the context of the US:

we calculate spatially-detailed carbon and water footprints of data centers operating within the United States, which is home to around one-quarter of all data center servers globally. Our bottom-up approach reveals one-fifth of data center servers direct water footprint comes from moderately to highly water stressed watersheds, while nearly half of servers are fully or partially powered by power plants located within water stressed regions.

Then there are the other impacts - pollution, mining, recycling, chemicals…these are the subject of much less scrutiny.

Big tech! AI! Pollution! Billions of dollars!

I was interested to come across an FT article “Pollution from Big Tech’s data centre boom costs US public health $5.4bn” which reports on a new claiming:

Air pollution derived from the huge amounts of energy needed to run data centres has been linked to treating cancers, asthma and other related issues, according to research from UC Riverside and Caltech.

The academics estimated that the cost of treating illnesses connected to this pollution was valued at $1.5bn in 2023, up 20 per cent from a year earlier. They found that the overall cost was $5.4bn since 2019.

This is certainly worth looking into, but I found the headline quite alarmist, so I decided to look into the paper behind the reporting.

Unfortunately, the paper is not actually linked from the article. After some digging, I found it as a Dec 2024 preprint titled: “The Unpaid Toll: Quantifying the Public Health Impact of AI”.

…AI's lifecycle, from chip manufacturing to data center operation, significantly degrades air quality through emissions of criteria air pollutants such as fine particulate matter, substantially impacting public health. This paper introduces a methodology to model pollutant emissions across AI's lifecycle, quantifying the public health impacts. Our findings reveal that training an AI model of the Llama3.1 scale can produce air pollutants equivalent to more than 10,000 round trips by car between Los Angeles and New York City. The total public health burden of U.S. data centers in 2030 is valued at up to more than $20 billion per year, double that of U.S. coal-based steelmaking and comparable to that of on-road emissions of California.

The authors’ names rang a bell - the same who published another preprint1 overestimating AI’s water footprint back in 2023 - even so, data center pollution is an important topic that needs more research, so I looked through the paper.

Pollution from diesel generators

The paper covers several different environmental impacts across scope 1 (emissions from on-site backup generators), scope 2 (emissions from power generation on the grid), scope 3 (manufacturing and other third party sources).

I focused on Scope 1 because this is rarely discussed, yet is an important part of data center operations. As I discussed previously in relation to demand response, the purpose of a data center is to provide a reliable environment for competing equipment. This means guaranteeing power availability.

In the US (the focus of this paper)2, the grid is pretty reliable, but there are outages (which may be increasing). Data centers mitigate this by having rapid response batteries which provide interim power before long-term generators spin up (usually within a few minutes). These generators are usually fueled with diesel so have high emissions.

The paper appendix states the assumption is “the actual emissions are 10% of the permitted level” based on an analysis of “a dataset of the air quality permits: permits issued before January 1, 2023 and permits issued between January 1, 2023 and December 1, 2024”. The actual calculations aren’t provided, which would help clarify exactly what basis the conclusions are derived from.

Backup generators are tested regularly, but only for short periods (1-2 hours). If this were performed monthly, that’s only 12-24 hours per year. Further, most diesel generators in the US are limited to a maximum of 100 hours per year (4 days) due to the pollution levels.

If we take the preprint assumptions to mean 10% annual capacity (this assumption has been clarified in the comments on this post), equivalent to roughly 36 days of continuous use, then this starkly contrasts with industry norms and represents a 30-fold overestimation. Even in an extreme scenario, such as a 15-day grid outage (360 hours), generator usage would amount to just 4% of the year—still less than half of the preprint’s estimate. This inflated assumption artificially boosts the calculated Scope 1 emissions, despite their minor role in the overall emissions profile.

To clarify this point, consider this 2023 emissions data (note this is global, not US specific):

• Google

  • Scope 1 = 79,400 tCO2e

  • Scope 2 = 3,423,400 tCO2e

  • Scope 3 = 10,812,000 tCO2e

• Meta

  • Scope 1 = 48,925 mtCO2e

  • Scope 2 = 1,658 mtCO2e

  • Scope 3 = 7,445,621 mtCO2e

• Microsoft

  • Scope 1 = 144,960 mtCO2e

  • Scope 2 = 8,077,403 mtCO2e

  • Scope 3 = 16,624,000 mtCO2e

These figures reveal that Scope 1 emissions, which encompass generator contributions, are a tiny fraction of Scope 2 and Scope 3 emissions.

The authors could have leveraged such publicly available data (actual emissions vs estimated), been more specific about what their calculations are based on, and performed a sensitivity analysis to align their estimates with reality.

Other problems with the paper?

The 2024 US Data Center Energy Report (one of only two credible sources of data center energy estimates I discussed here) projects a range of 325 to 580 TWh (6.7% to 12% of total electricity consumption) in the US by 2028.

In the preprint they assume 519 TWh from assessments in a McKinsey white paper. This is drawn from a “medium” growth rate without justification. The linked white paper does not justify this scenario either, merely describing it as a “medium scenario” and citing “Global Energy Perspective 2023, McKinsey, October 18, 2023; McKinsey analysis”.

Unfortunately, this 2023 Perspective is not available publicly (a 26 page executive summary does not explain the various scenarios) and there is no further information about the McKinsey analysis. So why did they pick this number? This is important for understanding the basis of their analysis of Scope 2 and 3 emissions.

We see something similar with the analysis of GPU power consumption - they make the common mistake of using the maximum thermal design power (TDP) when estimating total energy consumption of AI training:

We consider Llama-3.1 as an example generative AI model. According to the model card, the training process of Llama-3.1 (including 8B, 70B, and 405B) utilizes a cumulative of 39.3 million GPU hours of computation on H100-80GB hardware, and each GPU has a thermal design power of 700 watts. Considering Meta’s 2023 PUE of 1.08 and excluding the non-GPU overhead for servers, we estimate the total training energy consumption as approximately 30 GWh.

As I discussed previously, this is too simplistic an approach. Workload configuration, batch size, and number of nodes has a major impact on power demand and manufacturer TDP is never reached.

Without digging into the paper any further, these assumptions suggest a pattern of overestimation which will compound errors in the final conclusions.

Conclusions

Computing’s environmental footprint is complex, involving energy consumption, water usage, pollution, resource extraction, and beyond. Preprints like "The Unpaid Toll" play a role in spotlighting overlooked issues, such as data center pollution.

However, flawed assumptions - such as overestimating generator usage by a factor of 30 or relying on maximum thermal design power (TDP) for energy estimates - can skew results and mislead readers. Here, the preprint likely overstates the public health impact by a wide margin, undermining its credibility.

For meaningful progress, the field demands rigorous, standardized approaches to evaluate computing’s full environmental impact, tailored to regional and global contexts. This means grounding analyses in real-world data - such as corporate emissions reports - and factoring in variables like local grid reliability or water scarcity.

Both researchers and journalists must approach preprints with caution, verifying claims before they shape public discourse or policy, especially on high-stakes issues like health and sustainability.

Data center demand response is one of those topics that academics love. If you want to guarantee funding, journal paper acceptance, and press coverage, then researching data center demand response is a good bet.

Yet, despite its theoretical appeal, data centers are unlikely to adopt it in any meaningful way. This is a classic example of academic research being completely disconnected from commercial realities.

The topic has come up again with a new report:

Norris, T. H., T. Profeta, D. Patino-Echeverri, and A. Cowie-Haskell. 2025. Rethinking Load Growth: Assessing the Potential for Integration of Large Flexible Loads in US Power Systems. NI R 25-01. Durham, NC: Nicholas Institute for Energy, Environment & Sustainability, Duke University.

I wrote about this topic back in 2023, but let’s take another look.

Data center buildout is power constrained

It’s clear that there will be a need for much more data center capacity in coming years. The problem is not building the data centers themselves, but provisioning the power capacity (transmission and interconnection) to support the load. The US grid is not in a good state to handle such expansion, with backlogs of up to 7 years in some regions. I’ve written about similar problems in the UK.

So the results of this report are interesting:

• 76 GW of new load—equivalent to 10% of the nation’s current aggregate peak demand—could be integrated with an average annual load curtailment rate of 0.25% (i.e., if new loads can be curtailed for 0.25% of their maximum uptime)

• 98 GW of new load could be integrated at an average annual load curtailment rate of 0.5%, and 126 GW at a rate of 1.0%

This means that the infrastructure buildout wouldn’t be needed if those new data centers curtailed their power demand for certain periods.

The constraints on power procurement and grid improvements are mostly regulatory, so this is an opportunity for the new administration to have a real impact. However, there is so much regulation to deal with at various levels of government that other solutions are worth considering.

Uptime assumptions

Data centers are engineered for uptime, offering a controlled environment for reliable compute services. Redundant power supplies, backup batteries, and rapid-start generators are standard.

Uptime is assumed, but well architected applications assume failure might happen. This is why cloud services are designed around zones within regions. Each zone tends to be a physical data center, with zones in a small geographic region. Network latency between zones is minimized, which means those data centers need to be close to each other.

Applications can often handle zonal outages if architected for it. Regional failures, however, are tougher - especially for stateful applications. Regional failover is usually reserved for disaster recovery, as AWS explains:

A Multi-AZ architecture is also part of a DR strategy designed to make workloads better isolated and protected from issues such as power outages, lightning strikes, tornadoes, earthquakes, and more. DR strategies may also make use of multiple AWS Regions. For example, in an active/passive configuration, service for the workload fails over from its active Region to its DR Region if the active Region can no longer serve requests.

Changing this assumption to one where a data center might go offline at short notice is a major shift in how applications must be designed. Sure, one data center (or zone) might be able to go offline if you have built zonal failover into your architecture, but are zones spread across distinct grid segments? Unlikely.

Predictable flexibility

Short notice flexibility of this sort is incompatible with how the majority of applications are built today. However, what about managing workloads with more notice? The California Duck curve is a known phenomenon, so could we architect facilities around known periods of load?

Google is already doing this. They have three case studies across Europe, Asia and the US where they use predictable events to reduce peak power consumption:

• Scheduled power reductions during peak periods of Winter 2022-23 between 5pm-9pm in the Netherlands, Belgium, Ireland, Finland, and Denmark.

• Daily peak power reduction in Taiwan during the summers of 2022 and 2023.

• Reduced data center power consumption in Oregon, Nebraska, and the Southeast during extreme weather events.

These successes share a theme: predictability (or sufficient notice) and Google’s total control over its software and facilities. As they explain:

When we receive notice from a grid operator of a forecasted local grid event, for example an extreme weather event that will cause a supply constraint, we can alert our global computing planning system to when and where it will take place. This alert activates an algorithm that generates hour-by-hour instructions for specified data centers to limit non-urgent compute tasks for the duration of the grid event, and allows them to be rescheduled after the grid event has passed. When feasible, some of these tasks get rerouted to a data center on a different power grid.

To what extent can others apply the same level of sophistication to their data center usage, particularly when using shared cloud services? The report itself notes this:

These facilities house multiple tenants, each with varying operational requirements. Coordinating demand response participation in such environments introduces layers of administrative and logistical complexity, as operators must mediate cost- and reward-sharing agreements among tenants.

Spatial flexibility: moving workloads

The report focuses on how AI-specialized data centers, with deferrable tasks like neural network training, could enable load flexibility:

The central hypothesis is that the evolving computational load profiles of AI-specialized data centers facilitate operational capabilities that are more amendable to load flexibility. Unlike the many real-time processing demands typical of conventional data center workloads, such as cloud services and enterprise applications, the training of neural networks that power large language models and other machine learning algorithms is deferrable.

Unlike real-time cloud or enterprise workloads, AI training can theoretically pause. But this overlooks key costs:

• Duplication: Maintaining redundant infrastructure across regions is expensive.

• Migration: Moving large datasets across networks is slow and costly.

Does shifting workloads to off-peak grids save enough to justify these expenses? If workloads pause instead, can they resume seamlessly?

Snapshotting & checkpointing is possible, but convergence is a key part of AI training runs. That relies on regular recalculations across large-scale distributed GPU or TPU clusters, adaptive optimization, and learning rate (step size) tweaks, which cannot always be restarted without data loss.

This might be feasible for small jobs, but the risk involved with interrupting large jobs could outweigh any benefits.

What’s missing from these studies

Even without data center growth, there are clearly major issues with how long it takes to upgrade grid capacity, deliver new equipment, and improve reliability. Regulations in the UK and US have been designed conservatively to minimize disruption and maintain reliability - precisely what you want when grid demand is stable. But that is changing. Regulations now need to adapt to allow for a faster pace.

Examining demand response is a good idea because it can offer another solution (why not do both?). However, every analysis I’ve read misses a discussion of the practicalities:

• Workload Flexibility: How many tasks are truly movable in time and space?

• Incentives: How do operators weigh power savings against uptime? Does this differ for single-tenant (e.g., Google) versus multi-tenant (e.g., cloud providers) setups?

• Spot Markets: Could cloud providers use spot pricing to nudge customers toward flexibility?

• Development Costs: How should developers handle replication and migration costs? What notice period mitigates these?

• Workload Fit: Are AI training runs as pausable as assumed? How do checkpointing and scheduling affect convergence, especially for large-scale jobs?

Without tackling these, data center demand response remains an academic exercise - intriguing but impractical.

The biggest challenge with assessing the energy impact of AI workloads is the lack of data. Model developers rarely publish useful data about the machines, configuration, training time, or any other figures that could be used to calculate how much energy was involved.

Despite this, we still get weird and wonderful estimates such as “A single ChatGPT conversation uses about fifty centilitres of water, equivalent to one plastic bottle.” (Forbes) and total ChatGPT energy consumption being enough to charge every EV in the United States four times (BestBrokers).

Without commenting specifically on these particular estimates, the first question to ask whenever you see any of these claims is: what is the methodology behind the calculation?

As we found in the data center energy review I co-authored, most of the ridiculous estimates are a result of extrapolation. If power consumption was x for y users in the past, let’s just figure out how many more users there will be and assume the energy consumption is the same for each user1.

Estimating power consumption (x) is where you should pay attention.

Thermal Design Power (TDP): Keyword “maximum”

In the absence of actual energy measurements, estimations often look to the manufacturer specifications. These are useful to understand the maximum power draw of a particular component or system.

The key word there is maximum.

For example, an Nvidia H100 SXM GPU TDP is (up to) 700W. The H100 DGX node is 10,200 W. These can be seen from the datasheet:

A common error is to take TDP as the actual power draw, but that is rarely achieved. To produce accurate estimates for real workloads, you have to measure the actual power draw of the system under realistic conditions. This is time consuming and expensive, so most people don’t bother.

For servers, the SPECpower database is often used because of the test runs, but the same type of data isn’t available for GPUs.

Evaluating the Node Level Power Draw

At the Open Compute Global Summit in October 2024, Alex Newkirk gave a great presentation of real world measurements he and his team conducted for real AI workload power draw. This used a DGX H100 node to run training for Resnet 2, LLAMA-13b, and several other open source training sets from the MLPerf™ Training v4.0 benchmark.

It’s worth watching the short presentation, but the highlights are:

• Workload configuration had a significant impact on energy usage. Using higher batch sizes caused higher instantaneous power demand, but used less energy overall.

• The number of nodes had an opposite effect - more nodes resulted in shorter training time, but more energy use.

• The manufacturer TDP was never reached, even with more model parameters. A good rule of thumb is 75% of TDP, but even that shows a wide range.

So many parameters

The talk also mentioned a 2023 preprint from Wang et al. which highlighted the difference between node generations: Nvidia RTX 8000 vs A-100.

This backs up a 2022 Google paper which showed how model choice and hardware generation make a major difference in energy consumption (see my post about it).

Timestamp estimates

Any estimate must include a timestamp because training costs are falling rapidly. From Sam Altman:

The cost to use a given level of AI falls about 10x every 12 months, and lower prices lead to much more use. You can see this in the token cost from GPT-4 in early 2023 to GPT-4o in mid-2024, where the price per token dropped about 150x in that time period. Moore’s law changed the world at 2x every 18 months; this is unbelievably stronger.

This is especially a problem for academic research because of the incredibly slow pace. It’s common for papers to take 1-2 years to be published, by which time they are entirely out of date. The only way to assess them is precise dates are provided.

If model developers released more data about the energy profile of their training runs then we’d have a clearer picture. In the absence of this, be careful when you come across AI energy estimates - understanding data center energy is difficult enough. AI energy consumption is even more challenging.

The first article I wrote about data center energy was back in 2020: How much energy do data centers use? It remains one of my most popular because it analyzed all the sources I could find (very little information was public then). Much of my research came from the 2016 United States Data Center Energy Usage Report by Shehabi, et al, which was the definitive source at the time.

The Energy Act of 2020 instructed the US Department of Energy to produce an updated report, which was published at the end of 2024. It once again provides a detailed look at data centers in the US.

I have updated my 2020 blog post with all new information from that report, so this is a summary of the key changes. There’s a lot more to write about, but these are the highlights.

Data center energy consumption

In the United States, data center energy consumption in 2023 is estimated to be 176 terawatt-hours (TWh), or 4.4% of the country’s total electricity consumption. Between 2014 and 2016, consumption remained relatively stable, hovering around 60 TWh. However, it began to increase steadily from approximately 2017 onwards.

Globally, data center energy consumption in 2022 is estimated to be 240 - 340 TWh, accounting for around 1 to 1.3% of total global electricity demand. It’s important to note that this figure excludes networking activities, which were estimated to consume approximately 260 to 360 TWh in 2022. Additionally, cryptocurrency-related energy consumption is estimated to have been around 100 to 150 TWh in 2022.

Although you can find many reports about data center energy, there are only two research groups who produce reliable estimates.

In a review I co-authored, we analyzed 258 data center energy estimates from 46 original publications between 2007 and 2021 to assess their reliability by examining the 676 sources used. This revealed that the only credible global models are from the two research groups represented by Masanet, and Hintemann & Hinterholzer (Borderstep):

• Global, 2010 = 193 TWh1

• Global, 2018 = 205 TWh2

• Global, 2021 = 350-500 TWh3

• Global 2022 = 240-340 TWh4

• USA, 2018 = 76 TWh5

• USA, 2023 = 176 TWh6

The variance in estimates is a major challenge for anyone trying to get to the bottom of how much energy data centers use. This can be seen by plotting the various estimates:

More servers, better at idle, but drawing more power

Power proportionality is key to understanding the efficiency of servers. This scales in proportion to utilization. With perfect power proportionality, a server at 10% utilization will draw 10% of its maximum power. Idle power consumption has been improving over time (idle power is now measured at 20% of the server’s rated power).

Between 2007-2023, the average single socket server drew around 118W and the average dual socket server drew 365 W. However, updated data from Green Grid and reported values from the SPEC database for 2023-2024 show the average dual socket server drawing 600-750 W.

Servers have become better at using less power when idle, but are now drawing significantly more power during usage.

The total number of servers is also increasing and there is a new type - AI related. 6.5 million servers were shipped in 2022, estimated to rise to 7.7 in 2028. This brings the total installed base from 14 million in 2014 to 21 million in 2020. Of those, 1.6 million are related to AI.

Network energy consumption is not proportional to usage

Global network energy consumption was 260 to 360 TWh in 2022, but this is related to the number of ports on a network device, not the usage of the network.

Incorrectly linking data volume to network energy consumption has been a common error in academic literature and mainstream media. Networking is an important part of overall data center energy consumption, but total network energy consumption is not related to the growth in usage like servers.

More hyperscale cloud

Data center efficiency is improving. The average PUE has dropped from 2.5 in 2007 to 1.58 in 2023, however this has been flat for the last few years.

Hyperscale cloud data centers from the likes of Amazon, Google, and Microsoft, are mainly responsible for the overall efficiency improvements. Internal data centers were most common in 2010, with hyperscale and colocation providers making up less than 10% of all deployments. As of 2023, 74% of all servers are now in colocation or hyperscale facilities.

This offsets some of the increased server energy consumption because the facility they are deployed into is more efficient. Google is best in class7 with 1.08 PUE for some of its US data centers and the worst performing still doing very well (1.19 PUE for Singapore).

AI changes things

Although the past two decades have witnessed significant efficiency improvements, AI is going to present challenges.

Panic over data center energy is not new. A peer-reviewed article from 2015 predicted that data centers would consume 1,200 TWh of energy by 2020. As I covered in my data center energy review, this projection proved completely wrong.

I expect the same kind of alarm with AI, especially considering how early we are and how fast things are moving. Last year I wrote about the variables that make it difficult to predict: new models with fewer parameters, more energy efficient models, different data center hardware, and different client hardware.

We can see this is happening with the significantly lower cost of training from the Deepseek model vs what has previously been possible with OpenAI et al:

During the pre-training stage, training DeepSeek-V3 on each trillion tokens requires only 180K H800 GPU hours, i.e., 3.7 days on our cluster with 2048 H800 GPUs. Consequently, our pre- training stage is completed in less than two months and costs 2664K GPU hours. Combined with 119K GPU hours for the context length extension and 5K GPU hours for post-training, DeepSeek-V3 costs only 2.788M GPU hours for its full training. Assuming the rental price of the H800 GPU is $2 per GPU hour, our total training costs amount to only $5.576M.

That is not the fully loaded cost, but considering how much it cost to train the current leading models, this is indicative of the direction things are taking.

Despite advancements in AI energy efficiency, the overall energy consumption is expected to rise due to the substantial increase in usage. A significant portion of the energy consumption surge is attributed to AI-related servers: from 2 TWh in 2017 to 40 TWh in 2023.

This is a big driver behind the projected scenarios in the 2024 US Data Center Energy Report ranging from 325 to 580 TWh (6.7% to 12% of total electricity consumption) in the US by 2028.

With network energy consumption remaining flat or improving, and storage and data center infrastructure energy consumption also showing similar trends, the primary focus for improvements lies in the servers and GPUs that drive the majority of the increase in energy consumption. This is where the opportunity for efficiency lies.

The Journal of Industrial Ecology just published a new article I co-authored with Dag Lundén and Jens Malmodin:

Title: Network energy use not directly proportional to data volume: The power model approach for more reliable network energy consumption calculations.

Abstract: It is commonly assumed that data volume and network energy consumption are directly proportional, a notion perpetuated by numerous studies and media coverage. This paper challenges this assumption, offering a comprehensive examination of network operations to explain why the relationship between energy consumption and data volume is nonlinear. The power model approach is explored as an alternative methodology for calculating network energy consumption providing a more reliable representation of network energy use. The power model demonstrates that simple energy intensity calculations, expressed as kilowatt hours per gigabyte of data, are insufficient for accurately estimating real-world network energy consumption.

You can read it open access.

We started work on this back in early 2022 so I'm pleased to see it finally published in a peer reviewed, open access journal.

The launch of ChatGPT on November 30, 20221 marks the beginning of the new AI era. We’ve had many AI cycles in the past going all the way back to 1966. However, the last year has been particularly exciting. Today, everyone can have a compelling experience for free with ChatGPT 3.5 or Google Bard, with added amazement if you pay OpenAI to get ChatGPT 4 and DALL-E.

I expect this to continue in 2024. The AI train is just getting going. ChatGPT 5 is on the way, Facebook has been pushing Llama, Google is trying to compete with Gemini, Apple has quietly released Ferret, and startups like Mistral are also on board.

AI is the number one thing everyone2 is talking about. The doomerism is almost as prevalent as the hype, and we’ve also started to see that doomerism spread to predictions of the vast quantity of energy (and water) AI is undoubtedly going to consume.

Remember how a peer reviewed article from 2015 claimed that by 2020 data centers were going to consume 1,200 TWh of energy? The actual figure was 200-220 TWh (prediction, outcome). And there were many more like it. Just like data center energy before it, AI energy consumption doomerism is the perfect horror story for media e.g. BBC, Scientific American, New York Times.

AI definitely consumes energy. Google has reported how machine learning represents around 15% of its annual energy consumption and we know that GPUs are very energy intensive. The question is: how much?

What to watch out for

The big red flag is extrapolation from current public data. Something like:

1. A Google search consumes x energy.

2. Google has said that an AI query will cost x10 more than a normal search. query.

3. Therefore AI energy will be (Current Search Volume) x 10.

Or you might see:

1. OpenAI consumes x energy today.

2. OpenAI has 100 million users.

3. Allocate x energy across 100 million users.

4. If OpenAI grows to a billion users, that will be (Per user energy allocation) x 1 billion.

These are arguments from extrapolation and they are always wrong. You can’t trust any prediction about a complex system more than a few months out. Technology changes too rapidly.

That Google quote is also deliberately misleading – the full quote is:

In an interview, Alphabet’s Chairman John Hennessy told Reuters that having an exchange with AI known as a large language model likely cost 10 times more than a standard keyword search, though fine-tuning will help reduce the expense quickly.

Reuters.

The incentives to reduce the cost are very strong, particularly when services are offered for free and/or have business models indirectly linked to search volume e.g. ads or fixed subscriptions.

What might change?

I’m excited by the coming technological changes. Lots of new products and APIs to play with! What might we see that will complicate accurate calculations of the energy consumption of AI systems?

• New models with fewer parameters, but higher quality. For example, the Mixtral of Experts model “outperforms Llama 2 70B on most benchmarks with 6x faster inference”.

• More energy efficient models. Google reported the choice of model can impact the amount of computing power required by a factor of 5-10. Different tasks (even different search query types) will be given to different models.

• Different data center hardware. NVIDIA has a monopoly on GPUs, which is the largest incentive that can operate on a market to encourage more competition. Google Gemini was trained entirely on TPUs which “compared to the unoptimized P100s from 2017, the ML-optimized TPU v2 in 2019 and TPU v4 in 2021 reduced energy consumption by 5.7x and 13.7x, respectively.”

• Different client hardware. Apple has neural cores built into all their current computers and mobile devices. Transformers are already running on macOS 14 to give you predictions as you type. This happens locally. The M-series chips are probably the most power efficient chips in the world and “the M3 GPU is able to deliver the same performance as M1 using nearly half the power, and up to 65 percent more performance at its peak” (Apple).

• What to measure? Measuring “AI” is not the same as measuring the energy consumption of a network switch or a server because it’s all software. GPUs (and TPUs, etc) are a more easily measurable component, but AI also uses parts of other systems in the data center. How to account for training and/or inference on client devices will also be difficult.

This makes me optimistic. What I expect to see is AI initially consuming more energy as new technology emerges3. Just as with data centers, efficiency will improve quickly and then we’ll see AI energy consumption decoupling from demand. Eventually, it will plateau even as AI usage grows massively.

However, that will take some time. So in the meantime, keep an eye out for claims about the planetary damage AI Is causing. These will almost certainly be bogus.

Paper

Malmodin, Jens, Nina Lövehagen, Pernilla Bergmark, and Dag Lundén. “ICT Sector Electricity Consumption and Greenhouse Gas Emissions – 2020 Outcome.” SSRN Electronic Journal, 2023. https://doi.org/10.2139/ssrn.4424264.

Notes

• This paper presents the most up to date analysis of the electricity consumption and associated greenhouse gas emissions of the ICT sector.

• It examines user devices, networks, and data centers, but components such as TVs and gaming devices are separated because they tend to fall outside of a tight definition of ICT.

• ICT is estimated to account for ~4% of global electricity consumption in 2020.

  • Data center energy consumption is estimated at 223 TWh.

  • Network energy consumption is estimated at 272 TWh.

• The full lifecycle of all equipment, including the use stage, translates to around 1.4% of global GHG emissions in 2020. The majority of this (57%) is from user devices with 37% of the GHG footprint from the embodied emissions.

• Since 2015, total ICT emissions have increased by around 5%. Intensity has decreased (efficiency has increased) so per subscriber/user emissions are falling, but usage has increased dramatically. This is a classic example of the Jevons paradox.

Conclusions

The interesting thing about the methodology behind this paper is how much data has been collected from company reporting. This covers energy consumption as well as lifecycle assessments from specific products. There is still a significant modeling component from shipment data, but more and more organizations are publishing data so the estimates are more accurate.

It’s clear that the ICT industry has been focused on both efficiency (hardware energy efficiency and software infrastructure) and buying clean energy. This has resulted in a reduction in use-stage energy per subscription and carbon footprint per subscription even as usage has dramatically increased from 2007 to 2020. It provides another example of how wrong extrapolation-based projections are.

However, the total footprint is still increasing. More devices are being sold and most consumers are powering them using non-clean sources of energy. It’s difficult/impossible for consumers to make much difference here because they must rely on their local electricity grid to transition to clean energy.

I consider that individual impact on the environment rounds to zero because although some people will change their behavior, most won’t. If a relatively small number of manufacturers focus on their supply chain emissions and we continue with the clean energy transition for data centers and networks, a large portion of the ICT GHG emissions footprint could be addressed.

Last week, a journalist requested that I comment on a new preprint article discussing the water footprint of AI. Water is a crucial subject, as the discussion about the environmental impact of IT has mainly centered around energy and carbon emissions, a limitation I addressed in a 2021 article.

I agree with the preprint's conclusions that data center water consumption is often an overlooked aspect, with data center operators traditionally treating it as a trade secret. We need to better understand the full environmental footprint so we can model whether strategies like demand response or load shifting are worth it (maybe not). Although major companies like Microsoft and Google are becoming more transparent about their water usage, comprehensive data on the water footprint of large data centers is still lacking

However, the study's reported figures likely overestimate actual consumption. They describe an average on-site Water Usage Effectiveness (WUE) of 3.8-5.2 L/kWh based on a WSJ article from 2015 referencing a presentation from 2009 (the lower number) and an online post in Chinese (the higher number). Their model utilizes generic commercial cooling tower specifications and weather data. While the actual figures are not provided (a footnote, strangely not in the methodology section, explains a range of “0.5L/kWh and 5L/kWh depending on weather conditions”), the paper's August 2022 on-site WUE graphs indicate ranges of ~3.8-6.2 L/kWh, depending on location.

This is an interesting approach because data center operators do not publish granular data, so they have tried to infer it. However, the use of generic equipment likely misses the efficiencies that exist in hyperscale data centers.

Research from 2016 suggests the average US data center WUE is 1.8 L/kWh, while Facebook (Meta), an industry leader in data center water footprint, reports 0.26 L/kWh (pg39). Microsoft’s WUE ranges from 0.1 L/kWh (EMEA) to 1.65 L/kWh (Asia Pacific), with the Americas sitting in the middle (0.55 L/kWh). Google's efficiency is likely similar, making the graphed values excessive.

It is also crucial to contextualize data center water usage in relation to other industries. Billions of liters may seem like a lot, but understanding location-specific water stress is vital. While the paper mentions this, it does not thoroughly explore the topic alongside its results. In comparison to other industries, data centers consume relatively little water. All water counts, but how much it counts has to be considered relative to the local resources. Unlike carbon, zero water is not necessarily the goal.

Being a preprint, the article may undergo significant changes before publication, if it is published at all1. However, this is another example of selecting the high end of a range of possible input values to suit a particular narrative. We’ve seen similar scare tactics used when discussing “digital sobriety” in relation to extreme projections for future IT energy usage.

Perhaps a revised version could compare the estimates from the commercial cooling specs alongside the industry average and reported values from Facebook. That would be an interesting illustration. The methodology also needs clarification - different numbers are cited at different points, the ranges are wide, and the graphs in the appendix aren't helpful in understanding the actual values used in the calculations.

Researchers must exercise caution when selecting model inputs, avoiding the temptation to adopt the highest possible value. They should show the range. Overestimates only undermine the credibility of vital environmental research topics.

Paper

Pasek, A., Vaughan, H., & Starosielski, N. (2023). The world wide web of carbon: Toward a relational footprinting of information and communications technology’s climate impacts. Big Data & Society, 10(1). https://doi.org/10.1177/20539517231158994

Notes

• There are several problems with assessing the environmental impact of IT which mean that it may never be possible to create an accurate account.

• The paper accurately describes the differences in the key publications:

There are ongoing difficulties in acquiring, analyzing, and sharing high-quality data about the sector and remarkably enduring dissensus about research methods and implications. According to different voices in this debate, the carbon and energy costs of the Internet are growing (Andrae, 2020; Belkhir and Elmeligi, 2018; Ferreboeuf, 2019; Makonin et al., 2022), shrinking (Accenture Strategy, 2015; Aslan et al., 2018; Lange et al., 2020), holding flat (Masanet et al., 2020; Shehabi et al., 2018), or even moving in all directions at once, depending on how you count (Bieser and Hilty, 2018; Coroama et al., 2014; Court and Sorrell, 2020; Lorincz et al., 2019; Malmodin et al., 2014; Sorrell et al., 2020). Even within peer-reviewed literature estimates regarding the carbon intensity of network transmission vary by up to five orders of magnitude (Aslan et al., 2018). These differences matter; they make consensus on empirical questions impossible and polarize wider public debates.

• It also accurately diagnoses the high-level factors which cause these differences: data access, assessment methodologies, use of averages, functional units, system boundaries, and focusing on energy efficiency (most of which have been highlighted before). However, it doesn’t consider the technical differences between the methodologies underlying these conclusions (or the differences in the key publications listed in the quote above).

• For instance, the key difference between the “growing” and “holding flat” perspectives is the link between energy consumption and data volume. The Andrae (et al) modeling assumes that energy consumption and data volume are directly proportional. A rapid increase in data volume (as has been happening, and will continue) would result in an equally rapid increase in energy consumption. This isn’t true. There is no direct relationship between data volume and energy consumption. This paper makes it seem like there are real debates in the literature, when actually it is more about whether the models present an accurate picture of reality.

• Access to consistent industry data is a problem, but the paper focuses only on the researcher affiliations rather than discussing the details of data sourcing. This was one of the major issues identified in the data center energy estimates paper I co-authored last year. Transparency is improving. Only last year, Lundén et al released an updated paper based on survey data from 15 different telecom operators covering 21 European countries to provide an updated analysis of energy consumption and carbon footprint for 2015-2018. This follows from their earlier analysis of 2010-2015. This also uses more granular, country-specific carbon intensity factors (which is a legitimate criticism of certain methodologies that only use global averages). Another paper looks at real world observations for cellular networks in Belgium. These are recent papers with real data, a big improvement towards more transparency.

• System boundaries are also a major factor in differing assessments, but that is inherent in lifecycle assessments. Insufficient care is taken with comparisons because the reader did not understand the system boundary and/or the publication does a poor job at describing the boundary it is using. This was another issue I found analyzing center energy estimates - comparison between papers with incompatible (or inconsistent) system boundaries is common.

• Pasek et al correctly analyzes the differences between top-down and bottom-up assessment methodologies, but misses when they are appropriate to use. They say “top-down studies have the appeal of being better adapted to estimate future trajectories and outcomes.” In reality, top-down methodologies should only be used for retrospective analysis i.e. when you have the data and want to calculate intensities or allocate carbon footprint. Bottom-up estimates are a better basis to project a few years into the future, but all must make assumptions e.g. future shipment figures, or changes in technology, which become less accurate the further into the future they project.

• With the assumption that these factors may never be resolved the paper proposes that relational footprinting may be the solution as “an empirical and strategic orientation toward demarcating particular relationships between elements—geographic, spatial, technical, and social—within a broad infrastructural network”. In particular, they say:

…if the carbon intensity of data derives predominantly from its location, rather than its distance, then the ‘where’ of ICT becomes a modulating factor to its size. From this perspective, networks might be re-configured to leverage more intense development in renewably-powered locations and degrowth in fossil-fuel-based locations (say, to connect more data centers in Brazil and less in Qatar). Emissions can thus be lowered, with certainty, and without first winning global support for a dramatic endorsement or curtailment of industry business models or the value of particular forms of digital content.

• Despite the claim that relocating (or restricting the location of) networks and data centers doesn’t require changes in business models, it doesn’t really explore the details of what this would mean in practice. There are always multiple factors when considering infrastructure locations. Land cost, availability of (clean) energy, grid infrastructure requirements (new buildout or connection to existing substations), location of fibre networks (new, or existing), proximity to users, availability of water, availability of staff, ease of access, data residency requirements, local regulations and permitting all play a role. The paper mentions a few of these (e.g. latency and data residency/privacy), but only in passing. The primary driver of this proposal seems to be social/political.

• Finally, it doesn’t provide any methodology, examples, calculations, or suggestions for how to implement “relational footprinting”.

Conclusions

The main limitation of this proposal is that it is qualitative rather than quantitative. This paper provides no methodology for assessing the environmental impact in any measurable or comparable way. The focus on sub-sea cables is unusual because their impact is described primarily as a social case study of - their actual environmental impact is insignificant. The social impacts are important, but it’s unclear how they are relevant to “climate impacts”.

I find this paper quite strange. It purports to provide a solution to the challenges in assessing the environmental impact of IT, accurately diagnoses the high-level problems, but then completely misses the technical details. Perhaps this is because I’m used to reading scientific papers rather than social “science”1, so maybe that isn't the goal. Indeed, it suggests that numbers are suspect:

Throughout this study, we are reminded of Ted Porter’s famous observation that the credibility of numbers is a profoundly social problem, created and sustained by groups that often lack the public legitimacy and internal unity necessary to democratically resolve conflicts within and between institutions…Scholars overlook these complexities when they uncritically draw on the charismatic numbers of a single study.

The conclusion of the paper is that accurate assessments may be impossible so a different approach is needed, but that approach is not described in any way that is reproducible.

My conclusion is that past assessments have methodological flaws which can be addressed and better data made available. We need to improve our understanding of the models and communicate better so that they can be used outside of the technical literature. This is already happening and is an important part of how the field progresses, but there is more to do.

A survey on the need for updating the Greenhouse Gas (GHG) Protocol standards recently closed. Having been in existence since 2004 (with scope 2 and scope 3 guidance published in 2015 and 2013), it is a good time to consider changes.

Over the last few years, there has been a surge in interest in reporting accurate GHG figures. Many companies now report annually, either voluntarily or because they are large enough to be required by laws. The GHG protocol is the standard for these reports, even if they often are published in annoying formats (glossy PDFs).

In IT, transparency has also been improving. A year ago, AWS released its carbon calculator to bring it up to a similar level as Google and Microsoft. There have been some recent comments about AWS defunding their sustainability initiatives, but you can now at least get emissions numbers based on your cloud carbon footprint.

Unfortunately, the calculations are not all equal. Google is the most transparent with its methodology. AWS is the least - they give you a number, but the documentation is limited. This makes it impossible to compare between providers, but even within a single provider there insufficient detail to properly link energy generation with usage.

Can this be addressed with updates to the GHG protocol? Google published their submission to the GHG Protocol survey, and it’s a point they highlighted as a priority for change:

Under current GHGP methodologies, the actions that are credited as emissions reductions within companies’ inventories do not align sufficiently with the real-world GHG impact of those actions. Company strategies and actions that differ widely in effectiveness, both in their impact on a company’s own GHG footprint and on broader energy and economic systems, are in many cases credited equally…The updated GHGP should remedy this by more closely aligning credit for GHG reductions with real and measurable impacts of company actions.

Reporting also still requires lengthy analysis, often involving expensive consulting projects. It is very difficult to get data, particularly if you are a small business. The near real-time reporting we’re used to in finance is not yet possible in carbon accounting, although the cloud providers do a good job of updating their calculations frequently. Google is the best at this (every month), AWS the worst (3 months lag). This is going to be important for proper scope 3 reporting so that direct (scope 1 and scope 2) emissions figures can be reported through the supply chain.

Improving the granularity of electricity emissions data will help here, not least for serving as an input for temporal and spatial load shifting. This is important to be able to link clean energy generation to demand, says Google:

Under today’s GHGP guidance, geographic boundaries for scope 2 do not correspond to the physical markets where a company consumes electricity. A company can purchase Energy Attribute Certificates (EACs) far from where it consumes electricity,13 because the electricity represented by EACs is not required to be deliverable14 to the grid(s) where the company operates. Thus, EACs that are physically disconnected from underlying electricity consumption function effectively as offsets: they are reductions claimed elsewhere to compensate for a company’s electricity based emissions. This approach has become widespread in practice under today’s scope 2 guidance, despite the scope 2 standard ostensibly prohibiting the use of offsets to reduce a company’s emissions.

This is a limitation with how instruments like RECs are accounted for. There’s a disconnect between the grid mix at the time of use and using products like RECs to mitigate emissions. It’s why unbundled RECs are so cheap and are ineffective at achieving anything:

The Residual Mix of electricity from the grid is defined as the leftover once the consumed GOs are taken out from the total production mix. The basic idea is that companies (and consumers) that do not purchase the GOs or RECs, get this Residual Mix from the grid.

However, the calculation of the Residual Mix in a country is still highly inaccurate. The reason for this inaccuracy is that there is a time delay between the trading of electricity and the trading of the corresponding RECs and GOs. The RECs and GOs are created in the month after the month of the production, or later, and the trading has a maximum delay of 1 year for GOs, and 21 month for RECs. It is mathematically not correct to subtract data from those two different trading systems, because Solar and Wind power have heavy fluctuating production characteristics, and the unbundled hydropower GOs are issued irregularly. Concluding: the Residual Mix should not be used in LCA.1

Such distortions show up in things like Norway’s emissions factors which are 10 kg CO2/MWh on a grid-level, but 402 kg CO2/MWh on a residual basis because most of the clean energy credits produced in Norway are actually sold and claimed outside of the region2. If these are then used to produce claims of 100% renewable energy on an annual basis, the emissions reductions can be overstated by as much as 50%3. If you see anyone making such claims, the only safe assumption is that they are bogus.

One of my first peer-reviewed articles was on this topic back in 2020 where I analyzed the suitability of the GHG Protocol for calculating cloud carbon emissions. The cloud providers subsequently made this a lot easier through their cloud carbon calculator products. That was the first step - getting a number.

Now the GHG Protocol needs to evolve to better represent the complexity of how energy markets work, how companies respond to incentives to get to net zero, and how that flows through supply chains to ensure all organizations can produce accurate emissions inventories.

Machine learning is not new, but all the hype around generative AI and ChatGPT is indicative of its rapidly growing importance. As usage grows, minimizing the environmental impact is important to consider.

Use of AI is broken into two main stages - 1) training i.e. producing the model; 2) inference (or prediction) i.e. getting something out of the model. Training is what takes the most time and resources. This is particularly true if the training dataset is very large, but also because the process involves iterating on the model, training, and retraining to improve its accuracy.

A publication last year (also freely available) which included previously unavailable numbers on the energy & carbon impact of Google’s AI efforts provided some interesting insights:

• For the last 3 years, machine learning has represented around 15% of Google’s overall energy consumption, but 70-80% of Google’s total floating point operations per second (FLOPs). Google’s total energy consumption has increased, but the proportion attributed to machine learning has remained the same.

• The choice of model can impact the amount of computing power required by a factor of 5-10. Improvements to algorithms and reducing model density result in significant improvements in energy efficiency and a reduction in training time even as the number of parameters increases.

• Using specialist hardware (like Google’s TPUs) and modern GPUs specifically optimized for machine learning rather than graphics (like the NVIDIA V100 and A100) can improve performance per watt by 2-5 times compared to general purpose processors. Specifically, “compared to the unoptimized P100s from 2017, the ML-optimized TPU v2 in 2019 and TPU v4 in 2021 reduced energy consumption by 5.7x and 13.7x, respectively.”

This can be simplified to: energy consumption is related to the compute operations needed for training (i.e. training time), which is becoming more efficient through improved model implementations. At the same time, hardware is also becoming more efficient. This is offsetting the increase in usage.

What about the carbon emissions of AI?

As with all software neither AI nor machine learning emit carbon by themselves, but generating the energy used to power the infrastructure does. There is a correlation between the training time and energy consumption, but that doesn’t necessarily mean there is a close relationship between training time and carbon emissions. When and where the training happens is crucial to understanding the carbon impact.

High performance computing clusters typically associated with scientific computing tend to be run at very high utilization. They’re often co located near universities and jobs are booked to try and ensure there is always something scheduled to run. This is good when considering the high fixed cost of purchasing the equipment, but doesn’t help optimize use-stage carbon emissions.

The grid electricity mix continually changes so that one minute there might be an abundance of clean energy, but the next the grid might switch to being powered by fossil fuels. The ability to delay processing by a few hours can have a significant impact on the carbon footprint. Especially so if the processing can also be moved location (although this is very difficult and rarely done).

This means you can’t use the average carbon intensity to accurately calculate the carbon footprint of a workload that is running for just a few hours or days. The grid mix fluctuations are smoothed out and mean that any estimate would be either under- or over-estimating the total carbon. This is a limitation in a study that was just published in preprint considering the factors influencing the emissions of machine learning.

Better transparency is a running theme of my posts because without hourly (at least) tracking of emissions it’s difficult to produce an accurate estimate. This is why projects like EnergyTag exist and why companies like Google are really pushing for 100% clean energy 24/7.

The energy consumption of AI can be calculated, but translating that to carbon emissions is much more challenging - there currently isn’t enough data.

Demand response is where a user of energy (usually electricity) varies its demand for a period of time in response to a request from the grid operator. This typically occurs when demand might outweigh supply. If it is not possible to increase supply in response to higher prices, the next option is to ask some users to reduce their demand. This is possible where that demand is not time sensitive and can easily switch off for a time.

A common example of this is with air conditioning that could receive remote instructions to temporarily set itself to a higher temperature. Another example might be a fridge which could adjust its boundary of allowed ranges and not switch on so frequently - a few extra degrees for a few hours probably makes no difference.

In both of these scenarios, the consumer might be compensated through a variable rate tariff. Price increases are a good way to reduce demand. Alternatively, a bonus payment might be paid if demand is reduced during a specified time. This can even work in reverse during periods of low demand where consumers can be paid to take advantage of the abundant supply. Octopus Energy in the UK has several tariffs that incorporate this.

So why can’t this apply to data centers? Why do so few participate in demand response programs?

Reliability.

The primary purpose of a data center is to provide a secure, reliable, and well connected place for IT workloads to operate efficiently. Power is a crucial aspect of this and data centers invest a lot of money into ensuring that the IT equipment always has power. This involves backup batteries, generators, and emergency fuel delivery contracts to ensure continuity.

If power to a data center could be taken offline at short notice it would either need to shut down entirely or (more likely) would have to switch to an alternative source of power. Shutting down entirely is impractical even if a single entity controlled all the equipment and systems within the data centers. Building applications to shift load is very expensive and difficult to do. And switching to backup is a risky process in itself, even when tested regularly. Long duration batteries are still too expensive and too immature in their development to make them a reliable option, although large scale batteries are being deployed.

In theory the cloud makes load migration a lot easier. The concept of zones and regions is well understood and APIs make it easy to create new resources. Indeed, before Google Cloud was a proper product, Compute Engine zones used to turn off for up to 2 weeks for maintenance! 

Going a bit further up the stack to a larger geographical area, serverless products might make this more feasible. If you deploy a Lambda function to “US East”, but that means it’s distributed over several regions on the US East Coast from Boston to Florida, that would cover a wide enough area to migrate a workload in response to localized grid pricing. This could even be handled transparently by the cloud provider themselves, routing based on carbon signals. Google Cloud Storage already has this built into its “Multi-Region” deployment option, although that is for redundancy rather than reducing carbon.

However, the cloud does not make load migration cheap. Nothing about the cloud is cheap - you’re buying managed services at all levels of the stack. Migration of anything other than ephemeral loads immediately hits the problem of data gravity and it becomes prohibitively expensive (and time consuming) to transfer data out (or replicate it to multiple regions). Moving between zones is insufficient because they are deliberately within a few tens of miles of each other, which means they will be on the same electricity grid, with the same pricing. Latency and replication delay becomes a problem as well.

The additional revenue of participating in demand response is therefore meaningless in comparison to the risks and costs of migrating workloads. That’s why data centers don’t (and won’t) participate in demand response. 

There is an opportunity for cloud workloads, though. Edge services will make this more likely - data can live centrally with compute migrating between edge locations, caching appropriately. But it might be more expensive, especially with networking fees. It remains to be seen how customers prioritize the sustainability benefits of moving workloads spatially vs the costs of doing so.

I often see people making value judgements when discussing sustainable computing. This puts a higher value on one type of computer use versus another. The implication is that we should do less of some things because their value cannot justify the environmental impact.

These judgments are commonly expressed in terms of waste: duplicate photos are a waste, social media is a waste, “hoarding” data is a waste. The analogy is between bits and atoms where the atoms (in the form of physical objects) take up space that could be used for something else (supposedly better), or are discarded and cause disposal or recycling problems.

This analogy fails because it misrepresents the fundamental innovation of digital services: zero marginal cost and abundance. Bits and atoms are not the same. At the scale of the internet, storing additional data does not deprive someone else of their ability to store data. Streaming video at 4k does not prevent someone else from also streaming video at 4k. This is completely different from the physical world where there are real limits and space is truly “used up”.

Another analogy is with harmful or addictive substances, like alcohol. This is the idea that we’re “drunk” on computing resources like cloud storage or data, and we must adopt the idea of “digital sobriety”. One supposed solution is an “internet speed limit” to stop wasteful behavior. This is often used in the context of data center or network energy, particularly when data centers are competing for physical resources like water.

These arbitrary rules are applied by people who feel like they have the right to judge the behavior of others. How the rules should be define are ambigious, or are a rehashed argument from authority. They distract from the real challenges of building sustainable infrastructure by focusing on inconsequential actions like deleting unwanted photos or reducing video streaming quality settings. These are meaningless interventions that confer a false sense of “making a difference” that could be better directed elsewhere. Or worse, exacerbate the problem.

Effort is better directed at altering systems so that everyone can benefit from improvements at scale. This means working towards greater deployment of clean energy systems that power our IT. It means increasing transparency to help analyze the energy consumption of cloud computing, so we can understand the energy (and carbon, water, materials..) footprint of services built on top.

The goal of sustainable computing is to be able to use more and more IT services whilst reducing the environmental impact. This means providing data, tools, dashboards, and best practices for those developing and operating the systems we use every day. It’s already happening in networking with data transfer volumes vastly increasing at the same time as energy consumption decreases.

Puritan value judgments don’t help and any analysis based on them should be rejected.

Using Renewable Energy Certificates (RECs) or Renewable Energy Guarantees of Origin (REGOs) are common, but they can’t credibly be used to back claims of 100% renewables. Power Purchase Agreements (PPAs) or green supply agreements are much more complex, but do at least deliver on the claims.

However, if the focus is on “renewables” like solar and wind, then they are still inherently variable. Even high quality products like direct PPAs can’t make up for times when there is insufficient renewable energy available on the grid.

This is why there is a push for what’s known as 24/7 carbon free energy. The goal is that every unit of consumption is generated by carbon free sources of electricity. The goal is no longer pure “renewables”, but “carbon free” or “clean”, which can include wind and solar but also allows for nuclear, hydro, long-duration storage and bio-gas.

It’s important that a broad portfolio of energy sources is considered, particularly nuclear, because the barriers are too high to achieve decarbonization goals with renewables alone.

Complex energy systems

Google has been leading the drive towards the 24/7 carbon free energy goal. Amongst the three big cloud providers, Google has been a consistent leader in both pushing for 100% renewables and now 24/7 carbon free.

Microsoft has also been doing good work, but perhaps surprisingly, Google has been the most transparent. Google is aiming to match its global data center energy demand by carbon free energy 24/7 by 2030 and has published a policy roadmap towards that.

Unfortunately, it’s not just a case of building out more sources of clean energy and connecting them to the grid. Keeping the energy system in balance is tricky: grid operators must ensure that supply continually matches demand. The system has to deal with changes in weather, demand patterns from normal daily life, maintenance of equipment, and the fact that electricity is often generated nowhere near where it needs to be used.

For example, wind generates a lot of electricity in Scotland, but often needs to be curtailed because there is insufficient transmission capacity to move it down to where demand is in the South.

System level impacts of 24/7 carbon free energy

Google has funded two studies to look at the impact of 24/7 carbon free energy on how the grid operates. The first of these was published in 2021 for the United States and the second was recently published for Europe.

Both of these studies use open source modeling software (GenX for the US and PyPSA for Europe) to create scenarios with representative demand and a mix of generation technologies. The goal is to see how the system responds to demand and supply, and in particular to compare cost and carbon emissions vs the existing model of using 100% annual matching.

The details of the results are worth reading, but my main two takeaways1 are:

1. Reaching 90-95% carbon free energy can be achieved with only a small cost premium compared to 100% annual matching. However, pushing closer to 100% significantly increases the cost to more than double.

2. There is a big assumption about the available technologies, including long-duration battery/energy storage which has yet to be proven economically viable. The price of gas is assumed to be 35 EUR / MWh, which may turn out to be wildly optimistic considering gas futures were >100 EUR / MWh over the summer 2022.

I find it concerning that the success of this approach relies on hand-waving assumptions about the viability of unproven storage technologies when nuclear is mentioned only in a footnote for one of the scenarios. It should be a major part of every scenario. Not including it is consistent with the insane policies of some European countries to decommission their nuclear plants - Germany - but seems like a missed opportunity to test the potential of nuclear energy for achieving the 24/7 goal.

This is unique to the European study whereas the US modeling includes nuclear as part of the power generation modeling. It’s a disappointing example of the negative impact of political decisions following decades of pressure from self-defeating environmental activists.

Conclusions

It is notoriously difficult to make projections more than a few years out, but is necessary given the long lead times for deploying energy infrastructure. It’s good to see Google funding work to consider grid-wide changes rather than privately procuring clean energy for itself or relying entirely on questionable financial instruments to back its marketing. However, the shadow of misguided anti-nuclear activism is still limiting the technologies under consideration in Europe.

That said, it is encouraging that both studies (run independently, using different software) show how it is possible to reach the goals of 24/7 carbon free energy. As with clean energy generally, the greater the demand the more opportunity for the market to develop, allowing technologies to be proven, and prices to eventually fall. That this is being spearheaded by a private company is worthy of note, but it also highlights that the goals are not just limited by the limits of technology. Long term storage still needs to be proven, but nuclear needs to be seriously considered if Europe wants to develop a secure and sustainable energy grid.

Khan, K.N., Hirki, M., Niemi, T., Nurminen, J.K., Ou, Z., 2018. RAPL in Action: Experiences in Using RAPL for Power Measurements. ACM Transactions on Modeling and Performance Evaluation of Computing Systems 3, 9:1-9:26. https://doi.org/10.1145/3177754

General Notes

• This paper presents an analysis of accessing and validating process power consumption metrics available for Intel CPUs via the Running Average Power Limit (RAPL) interface. This is a commonly used method for examining power consumption, such as when comparing programming language efficiency.

• RAPL is not the only method for measuring power consumption. The Intelligent Platform Management Interface (IPMI) is also an option, but it was considered to be insufficiently accurate.

• An advantage of using RAPL is that it is built into the chip hardware. It can measure the entire chipset including embedded DRAM and GPUs, although this paper focuses on the CPU. This also means there is no measurement overhead because the metrics are enabled from boot, managed by the SoC, and are running regardless of whether you want them or not.

• RAPL was introduced in the Intel Sandy Bridge architecture, and is available in all subsequent processors. It measures a particular power domain, which was expanded in the Skylake architecture from Powerplane 0 and Powerplane 1 to include a new domain called Psys which covers the entire SoC.

• Access to the metrics are via Model Specific Registers (MSRs). These are 32-bit registers with different units depending on the chip (Sandy Bridge uses energy units of 15.3μJ, whereas Haswell and Skylake uses units of 61μJ.) and are updated every ~1ms. This presents several challenges:

  • The 32-bit register will roll over. No timestamps are provided, so you must track this yourself. The paper provides a method for calculating the overflow time based on the energy units and the power consumption. The higher the power consumption, the faster the counters are incremented. They use the example of a Haswell processor consuming 84 W triggering an overflow after 52 mins, however that is quite low power. It is quite easy to benchmark processors to 5 times that (400-600 W), which would significantly reduce the time between rollovers.

  • You will need to determine the chip type to know which registers to query their units.

  • Directly accessing the registers is only available to the root user on Linux, although there are alternative methods of polling them, such as via the /sys/class/powercap filesystem, via perf events, or via the PAPI library. Using the PAPI interface can add up to 30% of overhead to the measurement, although it depends on which RAPL attributes you request.

  • The 1ms update period is not consistent and has some jitter. This is a problem if you need measurements with more granularity e.g. where you want to profile a single function or code which executes faster than 1ms.

• The system temperature has an impact on power consumption. The paper reports that for Haswell, the package power grows by approximately 10–12% between 37C and 74C. For Skylake, this is 8-10% for between 23C and 32C. As the temperature increases, the cooling system, e.g. the fan, has to do more work. Getting accurate measurements requires warming up the processor from a cold start to avoid a sudden spike in power consumption as the cooling kicks in.

• Registers are updated non-atomically, which means you need to know in which order they are updated and allow an appropriate time in between each one to ensure that you are getting the right result.

• Individual core measurement is not supported, which limits the usefulness of power consumption against real world multi-threaded code or where hyperthreading is used.

RAPL and cloud instances

This paper was published in 2018, which is a long time ago in the cloud computing world! They have a section on using RAPL on Amazon EC2 where they were able to query the RAPL interface without any problems. However, the temperature metric remained at a constant 25C (probably falsified by the hypervisor), which meant they couldn’t calibrate their tests against temperature.

Furthermore, the EC2 hypervisor intercepted the readings and the clock speed was lower than their other tests, which created a polling delay. Virtualized CPUs are not guaranteed to map directly to physical CPUs, which is a particular problem in shared cloud environments where the resources are shared amongst other users. They could not perform the same validation as with the rest of the paper where the register values were compared against power meter measurements.

Since then, other researchers have found that the newer EC2 KVM hypervisor instances no longer provide access to the RAPL metrics. This is not a bad thing because the limitations above mean the results are probably not that useful. Using bare metal cloud instances seems like the only way to get access to the RAPL data now.

Although we do now have carbon calculators for Amazon, Google, and Microsoft cloud environments, these are abstracted away from the underlying energy data. This makes it difficult to optimize cloud applications for energy efficiency.