Measurement of Generative AI Workload Power Profiles for Whole-Facility Data Center Infrastructure Planning

Overview

The rapid growth of generative AI has introduced unprecedented computational demands and is increasing data-centre energy footprint, yet existing power data is largely proprietary and reported at varying resolutions, which makes estimating whole-facility energy use and planning infrastructure hard. The work presents a methodology that links high-resolution workload power measurements to whole-facility energy demand. On NLR's HPC data centre with NVIDIA H100 GPUs, power of AI training, fine-tuning and inference jobs was measured at 0.1-second resolution, with workloads characterised using MLCommons benchmarks for training and fine-tuning and vLLM benchmarks for inference.

Each node (4 × H100 SXM at 700 W TDP, 2 CPUs at 360 W) can draw up to 3,520 W; measured idle was 72.5 W per GPU and 64.1 W per CPU socket, and a GPU stress test gave a mean peak of 696 W. In 16-node Stable Diffusion training, power fluctuated between approximately 12 kW and 48 kW during evaluation phases. Online inference server power saturated near 2.8 kW at modest request rates. Training power scales roughly linearly with node count while inference power saturates.

The profiles feed a bottom-up discrete-event whole-facility model (DIPLOEE) including job arrival, scheduling and execution. Facility simulations show diurnal, weekly and seasonal patterns at low to medium utilisation, and even with all servers utilised, facility power reached only 73% and 80% of the data-centre rated design. NERC reported a 2024 Virginia event where a 1.5 GW data-centre load drop occurred within 82 seconds after a 230 kV line fault.

Limits: auxiliary loads (cooling, distribution losses) were not modelled; the authors note they might be less than 10% of total load for large state-of-the-art AI data centres. Measurements are for one GPU generation (H100) and specific benchmark workloads; it is a preprint.

Key points

1. NERC reported a 2024 Virginia event where a 1.5 GW data-centre load drop occurred within 82 seconds after a 230 kV line fault.
2. Each node (4 × H100 SXM at 700 W TDP, 2 CPUs at 360 W) can draw up to 3,520 W; measured idle was 72.5 W per GPU and 64.1 W per CPU socket, and a GPU stress test gave a mean peak of 696 W.
3. In 16-node Stable Diffusion training, power fluctuated between approximately 12 kW and 48 kW during evaluation phases.
4. Online inference server power saturated near 2.8 kW at modest request rates.
5. With all servers utilised, facility power reached only 73% and 80% of the data-centre rated design.
6. Auxiliary loads (cooling, distribution losses) were not modelled; the authors note they might be less than 10% of total load for large state-of-the-art AI data centres.

Using it in design

• Size AI data-centre facility power with measured profiles rather than the rated design: even with all servers utilised, facility power reached only 73-80% of rated design.
• Model node power between idle (72.5 W per GPU) and peak (3,520 W per node) with the fast 12-48 kW swings in training; consider sub-second transitions in UPS, battery and grid-connection studies.
• Treat inference power saturation (about 2.8 kW per server) and the linear scaling of training with node count as separate behaviours in capacity planning.

Who should read it

Designers and grid planners planning AI data-centre facility load profiles and infrastructure capacity.

Document contents

1. 1. Introduction (1.1 Whole-Facility Data Center Load Modeling, 1.2 Characterization of AI Workloads, 1.3 Contributions)
2. 2. Methodology (2.1 Overview, 2.2 Measuring Power Profiles of Individual AI Jobs: CPU and GPU power measurement, training and fine-tuning benchmarks, inference benchmarks; 2.3 Whole-Facility Simulation - DIPLOEE)
3. 3. Results (3.1 Experimental Setting, 3.2 Llama-2 70B fine-tuning, 3.3 Stable Diffusion training, 3.4 Llama-3 70B offline inference, 3.5 Online/Serve Inference Workload Power Profiles)
4. 4. Whole-Facility Simulation Results (4.1 Colocation Data Center, 4.2 Inference Data Center)
5. 5. Conclusion

Read the document

Publication details

Authors

Roberto Vercellino; Jared Willard; Gustavo Campos; Weslley da Silva Pereira; Olivia Hull; Matthew Selensky; Juliane Mueller

Publisher

arXiv preprint (National Laboratory of the Rockies)

Year

2026

Document number

arXiv:2604.07345

Document language

English

Source type

Scientific

Access

Open licence — downloadable

DOI

10.48550/arXiv.2604.07345

Licence

CC BY 4.0 — share with attribution

Licence evidence

arxiv.org

Citation