Empirically-Calibrated H100 Node Power Models for Reducing Uncertainty in AI Training Energy Estimation

Overview

As AI's energy demand grows, understanding the characteristics of that demand is critical for grid infrastructure planning and environmental assessment. The work combines empirical measurements during AI training on 8-GPU H100 systems with open-source benchmarking data to develop statistical models relating computational intensity to node-level power consumption, and measures the gap between manufacturer-rated thermal design power (TDP) and actual power demand. Even computationally intensive training operated at only 76% of the node's 10.2 kW TDP rating.

The architecture-specific asymptotic model predicts energy with 11.4% mean absolute percentage error (MAPE) versus 27-37% for TDP-based approaches. Measured node idle with a rear-door heat exchanger at 24 °C was 1.8 kW, and a combined GPU+CPU stress test drew a stable 8.4 kW, used as the empirical maximum. The highest average node power observed during training was 7.79 kW; one profile showed phases of initialisation (~2.5 kW) and compute-intensive training (~5.8 kW). Transformer workloads showed characteristic power fluctuations not seen in CNNs.

A worked example: for a 2,500-node (20,000 H100) cluster training for 90 days at PUE 1.1, the model's 7.3 kW per node gives about 49.0 GWh, versus 68 GWh if the 10.2 kW TDP were used (a 19 GWh difference), showing how a TDP-based estimate inflates energy and emissions.

Limits: a single node architecture (8-GPU H100) and specific benchmark workloads; models must be recalibrated for other hardware. The grid-stability impact of transformer power fluctuations was not quantified here.

Key points

1. Even computationally intensive training workloads operated at only 76% of the 10.2 kW node TDP rating.
2. The architecture-specific model predicts energy with 11.4% MAPE versus 27-37% for TDP-based approaches.
3. Measured node idle with a rear-door heat exchanger at 24 °C was 1.8 kW, and a combined GPU+CPU stress test drew a stable 8.4 kW, used as the empirical maximum.
4. The highest average node power observed during training was 7.79 kW; one profile showed phases of initialisation (~2.5 kW) and compute-intensive training (~5.8 kW).
5. For a 2,500-node (20,000 H100) cluster training for 90 days at PUE 1.1, the model's 7.3 kW per node gives about 49.0 GWh, versus 68 GWh if the 10.2 kW TDP were used (a 19 GWh difference).

Using it in design

• Calculate AI training cluster energy and power demand from measured node power (here 7.3-7.79 kW in training, 76% of TDP) rather than TDP; a TDP-based estimate inflated a 20,000-GPU cluster by 19 GWh.
• Model node power with idle (1.8 kW), initialisation (~2.5 kW) and training (~5.8-7.79 kW) phases and the power fluctuations of transformer workloads; consider the fluctuations in facility electrical design.
• Reduce energy-estimate uncertainty with an architecture-specific, FLOP-calibrated model (MAPE 11.4%) and recalibrate it for other hardware.

Who should read it

Designers and analysts estimating the power demand, facility capacity and emissions of AI training clusters.

Document contents

1. Introduction
2. Methods (measurements on 8-GPU H100 nodes, open benchmark data, statistical power models)
3. Results (gap between TDP and actual power, architecture-specific asymptotic model, transformer and CNN power signatures, 20,000-GPU cluster example)
4. Discussion
5. Conclusion

Read the document

Publication details

Authors

Alex C Newkirk; Jared Fernandez; Jonathan Koomey; Imran Latif; Emma Strubell; Arman Shehabi; Constantine Samaras

Publisher

Environmental Research: Energy (IOP Publishing)

Year

2025

Document number

arXiv:2506.14551

Volume

2

Issue

4

Pages

045016

Document language

English

Source type

Scientific

Access

Open licence — downloadable

DOI

10.1088/2753-3751/ae2486

Licence

CC BY 4.0 — share with attribution

Licence evidence

arxiv.org

Citation