Data Center Rack Density Has Doubled. And It's Still Not Enough
Data Center Knowledge · Bill Kleyman
An article using AFCOM State of the Data Center 2024 results to show that average rack density doubled from 6.1 to 12 kW in eight years yet remains far below what AI needs.
On this page
Overview
Using AFCOM State of the Data Center 2024 results, the author shows that average rack density doubled over eight years, from 6.1 to 12 kW, yet remains far below what AI systems need. The average was 6.1 kW in AFCOM's first report and 12 kW in the 2024 report. Sixty percent of respondents are working to increase density, with methods being airflow improvements at 58%, containment at 42% and liquid cooling at 40%.
Operators are mostly raising density through airflow improvements and containment, which are reaching their physical limits. Fifty-three percent believe generative AI will definitely increase colocation capacity requirements. A single NVIDIA DGX H100 system draws up to 10.2 kW, so a 12 kW rack fits only one. Operators are being asked to move from 6-12 kW to 40-60 kW or more per rack, beyond practical air cooling.
The author argues for rear-door heat exchangers (RDHx) as a retrofit path and direct-to-chip (DTC) cooling for the highest densities. RDHx suits medium and high density in existing air-cooled halls, and DTC and RDHx can be combined to move the full heat load to liquid. The piece is a freely accessible industry commentary; only the record's summary and key facts were used here and the full text was not read.
Key points
- AFCOM average rack density: 6.1 kW in the first report, 12 kW in the 2024 report.
- 60% of respondents are working to increase density; methods: airflow improvements 58%, containment 42%, liquid cooling 40%.
- 53% believe generative AI will definitely increase colocation capacity requirements.
- A single NVIDIA DGX H100 system draws up to 10.2 kW, so a 12 kW rack fits only one.
- Operators are being asked to move from 6-12 kW to 40-60 kW or more per rack, beyond practical air cooling.
- RDHx suits medium and high density in existing air-cooled halls; DTC and RDHx can be combined to move the full heat load to liquid.
Using it in design
- When a user places GPU servers in a rack, compute kW from server count and warn if the rack budget is exceeded (e.g. one DGX H100 = 10.2 kW).
- Offer RDHx as a retrofit cooling choice for existing air-cooled rooms.
- Recommend DTC above about 40 kW.
- In a density-increase plan, state the physical limits of airflow improvements and containment.
Monthly newsletter · free
Keep your reading list current
Once a month: new library sources, changes to Turkish regulations and upcoming data centre events.
SoSo CAD
Check cooling capacity alongside your design
Review zone heat loads, available capacity and the capacity balance after the largest unit is lost in SoSo CAD.
Request a demo
Who should read it
Operators and cooling designers planning to raise rack density in existing halls and prepare for AI loads.
Publication details
- Authors
- Bill Kleyman
- Publisher
- Data Center Knowledge
- Year
- 2024
- Document language
- English
- Source type
- Expert / community
- Access
- Free to read
- Licence
- Copyright retained; citation only — the publisher’s terms apply to the original document
Citation
Bill Kleyman (2024). Data Center Rack Density Has Doubled. And It's Still Not Enough. Data Center Knowledge. https://www.datacenterknowledge.com/cooling/data-center-rack-density-has-doubled-and-it-s-still-not-enough
Events on this subject
- SC26 – International Conference for High Performance Computing, Networking, Storage and Analysis
- NVIDIA GTC
- ISC High Performance
All content and rights belong to the respective authors and publishers. SoSo CAD provides only the citation, its own summary and a link to the source, and is not responsible for the sources' content.
Related sources
All sources on this topic
Open Compute Project Foundation 2026
Whitepaper: Thermal Boundaries in AI Infrastructure - Matching cooling technologies to compute through the power inflection, Version 1.00 (October 2026)
Choose cooling architecture against target rack power and each method's usable envelope: a 45 kW air ceiling (60-80 kW with rear doors), 120-250 kW class cold-plate racks, and the paper's anchor values for announced 600 kW-class racks.
- Practical air-cooled rack ceiling is taken as 45 kW, extendable to about 60-80 kW with rear-door heat exchangers.
Open Compute Project Foundation 2025
Modular Technology Cooling System for "Cloud Scale": Design & Delivery of Liquid to Rack Distribution Systems, Version 1.0 (25 August 2025)
When sizing TCS capacity against electrical capacity, state the cold-plate capture fraction (70-80%) and target delta-T (10 or 20 C) explicitly, and use the roughly doubled delivery at 20 C (4,845 versus 2,422 kW) as design headroom.
- Most IT equipment is designed for a 10 C coolant delta-T, with some designs considering 20 C.
Open Compute Project Foundation 2023
Cold Plate Cooling Loop Requirements, Rev 2
Require each vendor to fill in the parameters of importance tables for cold plates, hoses, quick disconnects, loops, manifolds and CDUs so offers can be compared.
- Glycol at 25% or above prevents bacterial growth; dielectric fluids are insulating but cost more and have higher GWP.
Open Compute Project Foundation 2023
Evaluating the Limits of Door Heat Exchanger Solutions Optimized for Open Rack V3
Build a rear-door HX sizing helper that takes rack kW, airflow (cfm), facility water temperature and liquid-capture fraction and returns approach, leaving air temperature and fan power, with 27 C facility water and 30 C air as default reference points.
- Assumptions: 30 C ambient (cold-aisle upper limit), sea level, 12-15 C air delta T, W27 facility water.
Open Compute Project Foundation 2023
Immersion Requirements Rev 2.0, First Amended (Revision 2.10, August 18, 2023)
Compare vendor offers on the same template: request W32 footprint density (kW/m2), full-solution static floor load, ceiling clearance and non-IT power per kW of IT in writing.
- Containment must hold at least 100% of the fluid volume of any single system without breach.
Open Compute Project Foundation 2022
Reservoir and Pumping Unit (RPU) Specification, Ver 1.0
Model AALC (RPU plus door or rack liquid-to-air heat exchanger) as a cooling option for air-cooled halls, counting the RPU's 4RU space, 75 kg mass and the 2%/5% parasitic power caps against rack U-space, weight and power budgets.
- AALC total power is at most 5% of rated cooling capacity, of which the RPU takes at most 40%.