From Servers to Sites: Compositional Power Trace Generation of LLM Inference for Infrastructure Planning

Overview

Datacenter operators and electrical utilities rely on power traces at different spatiotemporal scales: operators use fine-grained traces for provisioning, facility management and scheduling, while utilities use site-level load profiles for capacity and interconnection planning. Existing datacentre power models do not capture LLM inference workloads, in which GPUs shift rapidly among compute-intensive prefill, lower-power decode and idle states. The paper shows LLM inference power can be represented compositionally through workload-driven transitions among operating states and configuration-specific power distributions within those states.

Prefill drives GPU power to 80-90% of TDP while memory-bound decode runs at 40-60% of TDP; under continuous batching both coexist in the same batch. Classifiers trained on measured vLLM server traces turn request-arrival streams into state trajectories and power traces, which are then aggregated to rack, row and facility level. The generated traces achieve median absolute energy error below 5% for most configurations, while a flat TDP assumption overestimates energy by more than 3.5×.

In the facility case study, TDP sizing gives 1.19 MW peak versus 0.75 MW from generated traces, so nameplate provisioning overstates required interconnection capacity by roughly 60%. Generated facility traces show a peak-to-average ratio of 1.19, a load factor of 0.84 and a maximum 15-minute ramp of 0.11 MW. With a 600 kW row limit, TDP provisioning allows 23 racks, whereas the trace-based P95 criterion allows 57 racks (observed peak about 580 kW), more than double the rack density. Oversubscription headroom depends on temporal decorrelation of rack peaks; more strongly synchronised arrivals would reduce the safe oversubscription limit.

Limits: the models are trained on measured vLLM server traces and need configuration-specific distributions; the case study rests on a single production Azure inference trace. It is a preprint.

Key points

1. Prefill drives GPU power to 80-90% of TDP while memory-bound decode runs at 40-60% of TDP; under continuous batching both coexist in the same batch.
2. The generated traces achieve median absolute energy error below 5% for most configurations, while a flat TDP assumption overestimates energy by more than 3.5×.
3. For the facility case study, TDP sizing gives 1.19 MW peak versus 0.75 MW from generated traces, so nameplate provisioning overstates required interconnection capacity by roughly 60%.
4. Generated facility traces show a peak-to-average ratio of 1.19, a load factor of 0.84 and a maximum 15-minute ramp of 0.11 MW.
5. With a 600 kW row limit, TDP provisioning allows 23 racks, whereas the trace-based P95 criterion allows 57 racks (observed peak about 580 kW), more than double the rack density.
6. Oversubscription headroom depends on temporal decorrelation of rack peaks; more strongly synchronised arrivals would reduce the safe limit.

Using it in design

• For an AI inference facility, set interconnection capacity from traces generated from the workload, not nameplate TDP or mean power; TDP sizing gave 1.19 MW against 0.75 MW trace-based (about 60% too much).
• Assess row limits (for example 600 kW) for rack density and oversubscription with a trace-based P95 criterion instead of TDP (57 racks instead of 23), and state the temporal-decorrelation assumption of rack peaks (synchronised arrivals lower the limit) as an explicit risk.
• Take into account in the power model that prefill (80-90% of TDP) and decode (40-60%) states coexist in the same batch.

Who should read it

Designers and grid planners planning the electrical load, interconnection capacity and rack oversubscription of AI inference facilities.

Document contents

1. 1 Introduction
2. 2 Motivation and Observations (2.1 Power States in Continuous-Batch Inference)
3. 3 Methods (3.1 Planner-Facing Interface, state-transition classifiers, within-state power distributions, rack, row and facility aggregation)
4. 4 Evaluation and Case Studies (energy error, temporal autocorrelation, facility case study, row oversubscription)
5. 7 Conclusion

Read the document

Publication details

Authors

Grant Wilkins; Fiodar Kazhamiaka; Ram Rajagopal

Publisher

arXiv preprint (Stanford University / Microsoft Azure Research)

Year

2026

Document number

arXiv:2603.18383

Document language

English

Source type

Scientific

Access

Open licence — downloadable

DOI

10.48550/arXiv.2603.18383

Licence

CC BY 4.0 — share with attribution

Licence evidence

arxiv.org

Citation