Guidelines for Connection of Liquid Cooled Information Technology Equipment (ITE) to Data Center Facility Systems, Revision 1, Version 1.0 (April 2022)

Overview

The paper starts from the fact that technology refresh happens many times in a data centre's life and that liquid-cooled IT, in use for decades mainly for HPC and usually installed during initial construction, has been refreshed only with a limited set of solutions, often requiring a facility shutdown. High-density applications such as AI, VR and machine learning are making adding and changing liquid-cooled IT a standard operating requirement. The document describes ITE connecting to the facility water system (FWS) either directly or through a CDU that separates the technology cooling system (TCS) loop with a heat exchanger.

The fundamental connection components are: connection and isolation (reducing mean time to repair and improving concurrent maintainability), vent, drain and flush connections, strainers to keep contamination off heat exchanger coils, a flow balancing valve to stabilise pressure differential across the CDU or ITE, metering points for differential pressure and flow, and alignment accommodation. Some connection point requirements are taken from the OCP ACS Immersion document: FWS interfaces marked blue inlet and red outlet with flow arrows; operation from 50 kPa absolute (partial vacuum) to 800 kPa gauge; support for water and glycol mixtures up to 50%; and connection to overhead or underfloor distribution.

Pipe connection size is the key parameter setting the future kW capacity of liquid-cooled ITE. Heat transfer follows Q = volumetric flow x delta T x specific heat capacity, and Table 1 gives maximum practical cooling by pipe size and fluid temperature rise; flow velocity affects energy, diameter affects construction and delta T affects both the cooling system and ITE performance. Rack-based solutions (door HX, cold plate, rack-based immersion) are typically 0.75 inch/DN20 to 1.25 inch/DN40 with threaded connections, and the paper recommends BSPP as the global thread standard. CDU and tank connections of 2 inch/DN50 and above use grooved couplings (ASTM F1476) or flanges.

Choosing among connection types is risk-based. Above 1.25 inch, quick disconnects become hard to operate at FWS pressures because of hydrostatic thrust, and no global non-proprietary quick disconnect standard exists, so isolation valves upstream of quick disconnects are strongly recommended. For pipes of 2 inch and above, grooved couplings are favoured over Class 150 flanges because they need no hot work, tolerate pipe movement and misalignment, and carry load in the groove rather than the bolts; mission critical grooved couplings are expected to have housing and gasket from one manufacturer, traceable marking and certified, auditable installation verification. Hoses are maintenance items, with 10 years the general industry guidance for mission critical hose.

The paper also covers condensation and risk: keeping FWS liquid above dew point reduces condensation risk, and the ASHRAE recommended dew point limit for air-cooled classes A1-A4 is 15 C. Connection points should be assessed with failure mode and effects analysis (FMEA) to reduce severity, probability and detectability; connection methods that cannot be verified should not be used in high-severity locations or should be protected by leak detection. Welded, fused and crimped joints give no reconnection capability, so threaded, grooved and flanged connections are the focus for ITE refresh. The paper is advisory: identifying exceptions to the pressure range is the facility owner's responsibility and final validation requires a licensed engineer.

Key points

1. FWS interfaces must be marked: inlet blue, outlet red, each with a flow direction arrow.
2. Connections should support FWS from 50 kPa absolute (partial vacuum) to 800 kPa gauge, water or glycol mixtures up to 50%, with filtration and water quality per ASHRAE TC 9.9.
3. Pipe connection size sets future kW capacity: Q = volumetric flow x delta T x specific heat; a table gives maximum practical cooling by pipe size and temperature rise.
4. Rack-based solutions are typically 0.75 inch/DN20, 1 inch/DN25 or 1.25 inch/DN40 with threaded connections; the paper recommends BSPP as the global thread standard.
5. Isolation valves upstream of quick disconnects are strongly recommended because hydrostatic thrust makes operation difficult at FWS pressures and there is no global non-proprietary standard.
6. For pipe sizes of 2 inch/DN50 and larger, grooved couplings (ASTM F1476) are favoured over Class 150 flanges for no hot work and tolerance to pipe movement.
7. Keeping FWS liquid above dew point avoids condensation; the ASHRAE recommended dew point limit for air-cooled classes A1-A4 is 15 C.
8. Each connection should include isolation, vent/drain/flush connections (so flushing bypasses strainers and HX), strainers and flow balancing valves.
9. Flange gaskets can relax 20-50% of thickness in the first 4-6 hours; torque should be verified at installation and again at least 4 hours later.
10. Hoses and alignment devices typically last under 20 years and are maintenance items; general guidance for mission critical hose is 10 years.

Using it in design

• Place standard tap-off points with isolation valves, vent/drain/flush connections and strainers at every rack or CDU connection in liquid cooling layouts, sized from Q = flow x delta T x cp so future kW per position is explicit.
• Add validation rules: FWS design pressure within 50 kPa abs to 800 kPa g, glycol at or below 50%, and a warning when supply temperature is at or below room dew point (15 C default).
• Colour-code supply and return in drawings and BIM exports (blue inlet, red outlet).
• For rack connections under 2 inch specify BSPP threaded connection points and place an isolation valve upstream of any quick disconnect; for CDU and tank connections of 2 inch and above choose grooved coupling or flange by movement, alignment and hot work needs.
• Run an FMEA on connection points; prefer verifiable methods (radiography or visual inspection plus pressure test) in high-severity locations and plan leak detection and drip trays elsewhere.
• Put components with warranties under 20 years (hoses, strainers, valves, flanges, movement devices) in the maintenance schedule and in low-risk locations.

Who should read it

Cooling designers, mechanical contractors, CDU and cooling solution manufacturers and operations teams connecting liquid-cooled IT to facility water systems.

Document contents

1. Executive Summary
2. Definitions
3. Introduction
4. Design Considerations - Adding Liquid Cooled ITE
5. ITE Connection Point Standardization
6. ITE Thread Connections
7. Quick Disconnects
8. Grooved Coupling Connections
9. Flange Connections
10. Alignment Considerations
11. Flexible Hose
12. Dew point
13. FMEA Analysis to Reduce Liquid Cooled ITE Connection Risk
14. Key Components / Functions of Connection Lines
15. Conclusion
16. Appendix A: Connection/Disconnection Considerations (threaded, grooved couplings, flange joints)
17. Appendix B: Fixed Connection Methods - Welds, Fusion, Crimped

Related standards

• ASHRAE TC 9.9 Liquid Cooling Guidelines
• ASTM F1476
• ASME B16.5 Class 150 flanges
• OCP ACS Immersion Requirements Rev 2

Read the document

Publication details

Authors

Mitchell, D. (Victaulic); Menoche, J. (Vertiv); Gross, J. (JMGross Engineers); Sorell, V. (Microsoft); Musilli, J. (CPS/Integra)

Publisher

Open Compute Project Foundation

Year

2022

Document number

Rev 1 v1.0

Document language

English

Source type

Industry body

Access

Open licence — downloadable

Licence

CC BY-SA 4.0 — share with attribution under the same licence

Licence evidence

www.opencompute.org

Citation