OCP ACS Liquid Cooling Cold Plate Requirements Document, Revision 1.0 (October 9, 2019)

Overview

The document, written by authors from Intel, Iceotope and CPC, sets out requirements that future OCP liquid cooling specifications must meet; a checklist derived from it must be filled in and peer reviewed before a contribution goes to the Incubation Committee. Its scope is the TCS loop, from the CDU to the rack, through the manifold and IT equipment and back, with heat passed to a facility water system (FWS) that is out of scope, as are the IT equipment itself and fully air-cooled designs. It says there is no general guideline for when liquid cooling becomes necessary for CPUs and GPUs, and that the decision belongs in a TCO analysis. It assumes wetted materials are compatible with the coolant and that coolants are never mixed.

The terminology section follows ASHRAE. Cold plates are heat exchangers on the components, with micro-channel designs common for high power, and can be single- or two-phase. Hybrid cooling combines cold plates on high-power parts with air elsewhere and still needs room air conditioning, while full liquid cooling sends all heat to liquid. CDUs may be in-rack, row-level or facility-level and isolate the TCS from the FWS. Rack manifolds distribute flow, and quick disconnect couplings should be drip-less, as hand mate or blind mate types. Coolants covered are water with additives, glycol-based, dielectric and refrigerant fluids; propylene glycol is preferred over ethylene glycol, and glycol at or above 25% prevents bacterial growth while lowering heat transfer.

For each component the document lists a table of parameters of importance to be specified. Cold plate parameters include flow rate, inlet and outlet temperature, active area and filtering requirement. CDU parameters include maximum cooling capacity, liquid volume, approach temperature, pump capacity, PQ curve, power draw at 100%, 80% and 30% load, and connectors. Manifold velocity should stay within 1.5 to 2.1 m/s, depending on pipe diameter, to avoid erosion per ASHRAE. Coupling parameters include burst pressure, spillage and connection cycles; pump parameters include MTBF, IP rating and suction height. The document stresses that operating and burst pressures differ, and gives typical TCS operating pressure as 140-450 kPa (20-65 psi) for in-rack and row CDUs.

The requirements chapter covers certification (UL and FCC in the US, CE in Europe), pressure safety and risk management. IEC 62368-1 requires leak tests at 3 times normal operating pressure and 2 times under abnormal and single-fault conditions, while ASME B31.3 requires 1.5 times design pressure, and loops must meet the highest. It defines four IT equipment classes (hybrid basic, intermediate, advanced, full liquid), notes that CPU/GPU cold plates commonly capture 70-75% of IT heat, and two liquid-ready rack classes with or without a door heat exchanger. Sensor tiers follow ASHRAE Tier 2 and Tier 3, Redfish telemetry is referenced, and leak detection (indirect or direct) and intervention (manual or automatic) are classified.

Key points

1. The document covers only the TCS loop (CDU to rack, manifold, IT equipment and back); the facility water system is out of scope.
2. Hybrid cooling puts cold plates on high-power parts and air on the rest and still needs room air conditioning; full liquid cooling sends all heat to liquid.
3. CPU/GPU cold plates commonly capture 70-75% of IT equipment heat in the hybrid basic class.
4. Typical TCS operating pressure is 140-450 kPa (20-65 psi) with in-rack and row-level CDUs; a facility CDU may give higher pressure.
5. IEC 62368-1 requires leak tests at 3x normal operating pressure and 2x under abnormal and single-fault conditions; ASME B31.3 requires 1.5x design pressure; test to the highest.
6. Manifold fluid velocity should not exceed about 1.5-2.1 m/s depending on pipe diameter, to avoid erosion (per ASHRAE).
7. Glycol at 25% or more prevents bacterial growth but reduces heat transfer; propylene glycol is preferred over ethylene glycol for toxicity.
8. Quick disconnects should be drip-less; operating and burst pressure are distinct parameters.
9. Minimum liquid cooling sensors align with ASHRAE DCIM Tier 2 and advanced sensors with Tier 3; Redfish is the telemetry interface.
10. Leak detection is classed as indirect (pressure, flow, temperature analysis) or direct (leak rope or spot detectors); intervention as manual or automatic.

Using it in design

• Use the parameter tables for cold plates, CDUs, manifolds, couplings and pumps as the data sheet checklist when comparing vendor solutions.
• Classify the IT equipment as hybrid basic, intermediate, advanced or full liquid and use the capture fraction to size the residual room air load.
• Pick the rack class (with or without door heat exchanger) according to whether the room must be heat neutral.
• Set the TCS test pressure to the highest of IEC 62368-1 and ASME B31.3 multiples and size the loop for 140-450 kPa operating pressure.
• Decide the leak detection (indirect, direct) and intervention (manual, automatic) level against the uptime requirement and sensor cost.
• Cap manifold velocity at the limits cited, and specify coolant, filtration and wetted-material compatibility together.

Who should read it

Liquid cooling designers, rack and CDU vendors, and facility engineers who need a consistent way to specify and compare TCS loop components.

Document contents

1. Introduction
2. Terminology
3. Liquid Cooling Technology Definitions
4. Liquid Cooling Cold Plates
5. Hybrid Cooling
6. Full Liquid Cooling
7. Single- and Two-Phase Cooling Liquids
8. Coolant Distribution Unit
9. Rack Manifold
10. Quick Disconnect Couplings
11. Cooling Components Selection and Parameters of Importance
12. Cooling Liquid Selection
13. Cold Plate Selection
14. Coolant Distribution Unit Selection
15. Rack Manifold Selection
16. Quick Disconnect Coupling Selection
17. Pump Selection
18. Liquid Cooling Requirements
19. Pressure Safety Requirements
20. Risk Management
21. Liquid Cooling Classifications
22. Sensor Requirements
23. DMTF Redfish
24. Leakage Detection and Intervention Requirements
25. Comparison Metrics

Read the document

Publication details

Authors

Jessica Gullbrand; Nigel Gore; Jason Matteson; Elizabeth Langer

Publisher

Open Compute Project Foundation

Year

2019

Document language

English

Source type

Industry body

Access

Open licence — downloadable

Licence

CC BY-SA 4.0 — share with attribution under the same licence

Licence evidence

www.opencompute.org

Citation