Power Stabilization for AI Training Datacenters

Overview

This arXiv preprint (arXiv:2508.14318, 2025) comes from a large group of authors at Microsoft, OpenAI and NVIDIA. Using production telemetry from DGX-H100 racks, it characterises the synchronous power swings of large training jobs and explains why their sub-synchronous frequency content threatens turbine-generator shafts and transmission networks. The subject is how the power profile of training jobs looks from the grid side.

The paper also sets out how utilities express requirements: in the time domain (ramp rates, dynamic power range) and in the frequency domain (critical band, maximum spectral magnitude). The FFT energy of training power is concentrated between 0.2 and 3 Hz, close to known resonant modes of turbine-generator shafts and long transmission lines. A typical frequency-domain spec example caps spectral magnitude at 20% of total harmonic energy in a critical range of 0.1-20 Hz.

Three mitigations are compared: a software secondary-workload filler (Firefly), GB200 GPU power smoothing with a minimum power floor and ramp limits, and rack-level energy storage. Firefly kept primary-workload overhead below 5% but needed considerable CPU and host-device bandwidth to process GPU power data at 1 ms granularity. Simulating GPU smoothing with the floor at 90% of TDP on a real training waveform gave a total energy overhead of 10.5%.

The GB200 limits are a maximum minimum power floor of 90% of TDP and a minimum EDP of 1.1x TDP, leaving at least 20% of TDP as dynamic range. GPU smoothing alone therefore cannot meet, for example, a 10 MW range on a 100 MW job. Rack-level energy storage is identified as the best placement because AC-DC converters are already at rack level and a rack failure domain barely affects the site waveform. The authors recommend combining GPU-level smoothing with rack-level storage, plus a fast telemetry backstop and larger BESS for future sites.

Key points

1. Training power FFT energy is concentrated between 0.2-3 Hz, close to known resonant modes of turbine-generator shafts and long transmission lines.
2. A typical frequency-domain utility spec: critical range 0.1-20 Hz with spectral magnitude capped at 20% of total harmonic energy.
3. GPUs contribute more than 50% of provisioned server power (GB200 breakdown), so GPU phase changes appear as large swings at rack, row and datacenter feed level.
4. GPU power smoothing with the floor at 90% of TDP gave a total energy overhead of 10.5% in simulation.
5. GB200 limits: maximum power floor of 90% of TDP and minimum EDP of 1.1x TDP, leaving at least 20% of TDP as dynamic range.
6. The Firefly software approach kept primary-workload overhead below 5% but needed considerable CPU and bandwidth for 1 ms data.
7. Rack-level energy storage is identified as the best placement.
8. The authors recommend GPU-level smoothing with rack-level storage, a fast telemetry backstop and larger BESS for future sites.

Using it in design

• For AI-training halls, carry a 'synchronous load' flag and capture utility time-domain (MW/s ramp up/down, dynamic range) and frequency-domain (critical band, spectral cap) limits as interconnection inputs.
• Offer a mitigation selector: GPU power floor/ramp limits (with an energy overhead of up to about 10% at a 90% floor), rack-level BESS or supercapacitors sized for compute/communicate swings, and site BESS.
• Check that the chosen combination meets the spec.
• Warn that UPS and PDU exposure to swings grows when storage is placed higher in the hierarchy, and recommend fast power telemetry with FFT monitoring as an operational backstop.

Who should read it

Electrical engineers and energy consultants planning the power architecture and grid interconnection of AI training halls.

Related standards

• IEC 61000-3-3

Publication details

Authors

Esha Choukse; Brijesh Warrier; Scot Heath; Luz Belmont; April Zhao; Hassan Ali Khan; Brian Harry; Matthew Kappel; Russell J. Hewett; Kushal Datta; Yu Pei; Caroline Lichtenberger; John Siegler; David Lukofsky; Zaid Kahn; Gurpreet Sahota; Andy Sullivan; Charles Frederick; Hien Thai; Rebecca Naughton; Daniel Jurnove; Justin Harp; Reid Carper; Nithish Mahalingam; Srini Varkala; Alok Gautam Kumbhare; Satyajit Desai; Venkatesh Ramamurthy; Praneeth Gottumukkala; Girish Bhatia; Kelsey Wildstone; Laurentiu Olariu; Ileana Incorvaia; Alex Wetmore; Prabhat Ram; Melur Raghuraman; Mohammed Ayna; Mike Kendrick; Ricardo Bianchini; Aaron Hurst; Reza Zamani; Xin Li; Michael Petrov; Gene Oden; Rory Carmichael; Tom Li; Apoorv Gupta; Pratikkumar Patel; Nilesh Dattani; Lawrence Marwong; Rob Nertney; Hirofumi Kobayashi; Jeff Liott; Miro Enev; Divya Ramakrishnan; Ian Buck; Jonah Alben

Publisher

arXiv preprint (Microsoft / OpenAI / NVIDIA)

Year

2025

Document number

arXiv:2508.14318

Document language

English

Source type

Scientific

Access

Free to read

DOI

10.48550/arXiv.2508.14318

Licence

CC BY-NC-SA 4.0 — share with attribution for noncommercial purposes under the same licence

Citation