Electrical considerations with large AI compute (Daniel Bizo)

Overview

The article explains why large transformer-model training clusters create synchronized, millisecond-scale step loads and how this stresses power distribution. Uptime estimates that load swings can exceed 100 percent at system level and that repeated overshoots above the rack power rating can occur millions of times a year. The author argues that capacity planning must move from derated nameplate to repeated peak power.

In the worst case, the low-to-high swing during training can exceed 100 percent at system level, with load nearly doubling within milliseconds every few seconds. Even a few dozen racks can produce step changes from several hundred kW to a few MW, risking voltage sags and swells, harmonics and sub-synchronous oscillations.

Conventional planning derates nameplate by 25-50 percent for diversity, whereas AI training clusters can exceed the sustained maximum rating. In the 2022-2024 era, 8-GPU nodes swing up to about 4 kW per node (16 kW per 4-node rack) against a rack rating around 41 kW. GB200 NVL72 racks can jump from about 60-70 kW to over 150 kW against a 132 kW specification, which means repeated overloads of up to about 20 percent.

Liquid cooling lowers steady fan power and adds thermal headroom, which widens the relative power excursions. Breakers, tap-off units, busways and conductors sized close to the rated AI load may see repeated overloads they are not tested for, causing cumulative thermal stress. IEC 62040-3 and IEEE 1100 are named as related documents. The article is an expert view that offers estimates, not firm measurement data.

Key points

1. Worst case, the low-to-high swing during training can exceed 100 percent at system level, with load nearly doubling within milliseconds every few seconds.
2. Even a few dozen racks can produce step changes from several hundred kW to a few MW, risking voltage sags and swells, harmonics and sub-synchronous oscillations.
3. Conventional planning derates nameplate by 25-50 percent for diversity; AI training clusters can exceed the sustained maximum rating.
4. 2022-2024 era 8-GPU nodes swing up to about 4 kW per node (16 kW per 4-node rack) against a rack rating around 41 kW.
5. GB200 NVL72 racks can jump from about 60-70 kW to over 150 kW against a 132 kW specification, repeated overloads up to about 20 percent.
6. Liquid cooling lowers steady fan power and adds thermal headroom, which widens relative power excursions.
7. Breakers, tap-off units, busways and conductors sized close to rated AI load may see repeated overloads they are not tested for.

Using it in design

• Catalogue AI racks with both 'ratedPowerKw' and 'peakTransientKw' (e.g. 132 vs 150+ for NVL72).
• Size branch circuits, busway tap-offs and UPS for repeated peak, not derated nameplate.
• Do not apply diversity factors to AI training halls.
• Warn when headroom falls below 20 percent.

Who should read it

Engineers and capacity planners sizing electrical distribution and UPS for AI halls.

Related standards

• IEC 62040-3
• IEEE 1100

Publication details

Authors

Daniel Bizo

Publisher

Uptime Institute Journal

Year

2025

Document language

English

Source type

Industry body

Access

Free to read

Licence

Copyright retained; citation only — the publisher’s terms apply to the original document

Citation