Emergence and Expansion of Liquid Cooling in Mainstream Data Centers

Overview

The paper brings two classification changes. For high-density air-cooled ITE it defines class H1: allowable 15-25 degC and recommended 18-22 degC. Facility-water classes are renamed by their upper supply limit: W17 (was W1), W27 (W2), W32 (W3), W40 (new), W45 (W4) and W+ (W5), all with a 2 degC lower limit. The limits of air cooling are put in numbers.

Server fans can consume up to about 20% of server power, which in a 50 kW rack is at least 5 kW. A 40-50 kW rack can need about 5,000 cfm while the best-in-class perforated tile passes about 1,900 cfm, so beyond a certain density a single raised-floor tile is physically insufficient. The energy section states liquid cooling can bring PUE below 1.1, and a case study using warm water (40-45 degC) with absorption chilling reached a cooling pPUE under 0.07.

The key planning warning is that facilities running W32 today may have to move to W27 as chip power rises, hence the recommendation to design new data centers to accommodate liquid cooling later. It is a classification and planning white paper; it does not give detailed hydraulic design or CDU sizing. For context it also cites A1 allowable maximum 32 degC and A4 maximum 45 degC, and should be read with the 5th edition of ASHRAE Thermal Guidelines and the Liquid Cooling Guidelines.

Key points

1. Class H1: allowable 15-25 degC, recommended 18-22 degC.
2. Liquid classes: W17 (was W1), W27 (W2), W32 (W3), W40 (new), W45 (W4), W+ (W5); all with 2 degC lower limit.
3. Server fans can use up to about 20% of server power; at least 5 kW in a 50 kW rack.
4. A 40-50 kW rack can need about 5,000 cfm; best-in-class perforated tile about 1,900 cfm.
5. Liquid cooling can bring PUE below 1.1; 40-45 degC water with absorption chilling gave cooling pPUE under 0.07 in a case study.
6. Facilities on W32 today may need to move to W27 as chip power rises.
7. Context: A1 allowable max 32 degC, A4 max 45 degC.

Using it in design

• Add class H1 and the W17/W27/W32/W40/W45/W+ facility-water classes to the thermal catalog as enumerations.
• Above about 1,900 cfm per rack (about 12-15 kW at 20 degF dT) one tile is not enough; require containment, in-row cooling or liquid.
• Add a 'liquid-ready' option that reserves pipe routes, CDU space and room for W32 or colder water.
• Check facility-water capacity against a move down to W27, not just the first chip generation.

Who should read it

Thermal and mechanical designers preparing an air-cooled facility for liquid cooling or planning a new one.

Related standards

• ASHRAE Thermal Guidelines for Data Processing Environments 5th ed.
• ASHRAE Liquid Cooling Guidelines for Datacom Equipment Centers
• ASHRAE 90.4

Publication details

Authors

ASHRAE TC 9.9

Publisher

ASHRAE TC 9.9

Year

2021

Document number

TC 9.9 White Paper

Document language

English

Source type

Standard

Access

Free to read

Licence

Copyright retained; citation only — the publisher’s terms apply to the original document

Citation