Coordinated Cooling and Compute Management for AI Datacenters

Overview

AI datacentres are being deployed on a large scale to support the training and deployment of power-intensive large language models (LLMs). Cooling is cited as contributing approximately 45-50% of total energy consumption in AI data centres, and recent studies are cited projecting that 80-90% of total LLM workload and energy in production comes from inference rather than training. Previous work has examined the energy efficiency of LLM inference but explicitly ignored thermal behaviour and cooling systems.

The proposed controller provisions GPU count and tensor parallelism by linear programming, picks per-class GPU frequency (DVFS) by MILP, and sets supply air temperature and airflow by model predictive control (MPC). Cooling control is formulated as an MPC problem (SciPy SLSQP) and the TP and server-allocation problems as MILPs (PuLP), driven by real GPU profiling and a one-week Azure inference trace. Simulation constraints include an upper limit of 70 °C on return-air temperature and 50 °C on GPU inlet temperature, with PID gains Kp 4.5, Ki 0.18 and Kd 0.1.

The controller achieves 24.2% computing-energy savings and 31.2% cooling-energy savings and lowers average GPU temperature by 17.0%, with negligible impact on inference latency. Under the coordinated control, maximum per-GPU temperature stays below 50 °C while service level objectives (SLOs) are maintained.

Limits: the cooling plant modelled is air-based, and the authors list liquid and hybrid cooling as future work. The evaluation is a simulation on a one-week Azure inference trace, not a live system.

Key points

1. Cooling is cited as contributing approximately 45-50% of total energy consumption in AI data centres.
2. Recent studies are cited projecting that 80-90% of total LLM workload and energy in production comes from inference rather than training.
3. The proposed controller achieves 24.2% computing-energy savings and 31.2% cooling-energy savings and lowers average GPU temperature by 17.0%, with negligible impact on inference latency.
4. Simulation constraints include an upper limit of 70 °C on return-air temperature and 50 °C on GPU inlet temperature, with PID gains Kp 4.5, Ki 0.18 and Kd 0.1.
5. Under the coordinated control, maximum per-GPU temperature stays below 50 °C while SLOs are maintained.
6. Cooling control is formulated as an MPC problem (SciPy SLSQP) and the TP and server-allocation problems as MILPs (PuLP), driven by real GPU profiling and a one-week Azure inference trace.

Using it in design

• Design AI-hall cooling control together with workload management: when GPU count and tensor parallelism, DVFS and supply air temperature and airflow were optimised jointly, cooling energy fell by 31.2%.
• Define GPU inlet temperature (50 °C) and return-air (70 °C) upper limits as control constraints and verify them together with SLOs.
• Model the cooling plant as air-based; re-verify results for liquid or hybrid cooling.

Who should read it

Engineers and researchers developing coordinated compute and cooling control for AI inference clusters.

Document contents

1. I. Introduction (A. Main Contribution, B. Article Outline)
2. II. Background (A. Datacenters Cooling Systems, B. Tensor Parallelism and Thermal Constraints, C. DVFS and LLM Workload Characteristics)
3. III. Modelling (A. General Framework for Rack-Based Air Cooling, B. Temperature Field Model and Power Model Development, C. Workload Forecasting Model, D. Impact of TP and DVFS on LLM Inference Workloads)
4. IV. Controller Design (A. Cluster-Level Control, B. Window-Level TP Selection, C. Cooling MPC Formulation, D. Per-Job DVFS with Class-Aware Constraints)
5. V. Evaluation (hierarchical control-based provisioning, experimental evaluation, spatial distribution of server exhaust temperature, performance comparison, PID-based datacenter cooling, GPU frequency sensitivity)

Read the document

Publication details

Authors

Nardos Belay Abera; Yize Chen

Publisher

IEEE Transactions on Cloud Computing (Institute of Electrical and Electronics Engineers (IEEE))

Year

2026

Document number

arXiv:2601.08113

Pages

1-21

Document language

English

Source type

Scientific

Access

Open licence — downloadable

DOI

10.1109/tcc.2026.3746242

Licence

CC BY 4.0 — share with attribution

Licence evidence

arxiv.org

Citation