The Unseen AI Disruptions for Power Grids: LLM-Induced Transients

Overview

Recent breakthroughs of LLMs have exhibited superior capability across major industries and stimulated multi-hundred-billion-dollar investment in AI-centric data centres in the next 3-5 years. Concerns over AI energy usage are rising too; for instance, training GPT-4 consumed over 50 GWh, approximately 0.02% of California's annual electricity consumption and a 50-fold increase over GPT-3. But focusing only on these figures might obscure a more critical challenge: the disruptive dynamic power consumption behaviour of AI on power grids. The paper proposes metrics for AI-centric data centres (TDP, PUE, peak/average and peak/idle ratios, ramp and decline rates), develops high-level mathematical models of training, fine-tuning and inference power, and presents case studies from the MIT Supercloud dataset and single-GPU experiments.

The authors' experiment indicates LLM training can move instantly from cold start to peak load of over tens of megawatts, whereas existing distribution-grid minute-level ramping capability is typically limited to a few megawatts. Component power considerations: GPU up to 400-700 W, CPU 200-400 W, HBM 30-80 W per GPU, cooling 10-20% of total system power, power supplies typically 90-96% efficient. Traditional computing is 10-15 kW/rack versus 30+ kW/rack for AI workloads. Long-running training jobs show a peak/average ratio close to 1, compared with 1.5-2.0 for conventional data centres; the peak/idle ratio indicates how large the drop is after a training interruption.

A BERT job in the MIT Supercloud dataset peaked at approximately 48.70 kW with an average of 17.80 kW and standard deviation of 12.39 kW over 4 days. Single RTX 4090 GPT-2 training showed power drops of about 320 W and ramps of 350 W, with average power 414 W. The authors conclude that sub-second peak-idle power fluctuation, unpredictable interruptions during training and user-behaviour-dominated inference power together pose great challenges to power grids.

Limits: high-level mathematical models and a limited number of case studies (one MIT dataset and single-GPU experiments), with simple models rather than real facility measurements. It is a preprint and touches on grid-scale and hybrid storage opportunities.

Key points

1. The authors' experiment indicates LLM training can move instantly from cold start to peak load of over tens of megawatts, whereas existing distribution-grid minute-level ramping capability is typically limited to a few megawatts.
2. Component power: GPU up to 400-700 W, CPU 200-400 W, HBM 30-80 W per GPU, cooling 10-20% of total system power, power supplies typically 90-96% efficient.
3. Traditional computing is 10-15 kW/rack versus 30+ kW/rack for AI workloads.
4. Long-running training jobs show a peak/average ratio close to 1, compared with 1.5-2.0 for conventional data centres; the peak/idle ratio indicates how large the drop is after a training interruption.
5. A BERT job in the MIT Supercloud dataset peaked at approximately 48.70 kW with an average of 17.80 kW and standard deviation of 12.39 kW over 4 days.
6. Single RTX 4090 GPT-2 training showed power drops of about 320 W and ramps of 350 W, with average power 414 W.

Using it in design

• Define AI load not only by energy but by peak/average (about 1 in training), peak/idle ratio and ramp/decline rates; make sub-second dynamics inputs for UPS, generator and grid-connection studies.
• In connection studies check that distribution-grid minute-level ramp capability (a few MW) may be far below the instantaneous step of a training load (tens of MW).
• Model rack density as separate classes, 10-15 kW (traditional) and 30+ kW (AI), and assess the drop after a training interruption with the peak/idle ratio.

Who should read it

Designers, grid planners and researchers assessing the dynamic power behaviour and grid impact of AI workloads.

Document contents

1. I. Introduction (A. AI Growth and Associated Impacts on Power Consumption, B. Related Works)
2. II. Characteristics of AI Loads (A. Structure of an AI Compute Node, B. Classification of AI Electricity Load, C. The Correlation of LLM and its Hardware, D. Unique features of AI workloads)
3. III. Power Consumption Modeling of AI (A. Metrics for AI-centric Data Center, B. Total Data Center Power Consumption, C. AI Workload Power Consumption, D. Dynamic Power Consumption)
4. IV. AI Power Consumption Analysis: Case Studies (data center behaviour, baselines: LLMs, LLM training, fine-tuning of LLM, LLM inference, LLM inference with different batch sizes)
5. V. Future Research Directions (A. AI-User Perspective, B. Data-Center Perspective, C. Power Grid Perspective)
6. VI. Conclusions

Read the document

Publication details

Authors

Yuzhuo Li; Mariam Mughees; Yize Chen; Yunwei Ryan Li

Publisher

arXiv preprint (University of Alberta)

Year

2024

Document number

arXiv:2409.11416

Document language

English

Source type

Scientific

Access

Open licence — downloadable

DOI

10.48550/arXiv.2409.11416

Licence

CC BY 4.0 — share with attribution

Licence evidence

arxiv.org

Citation