White paperFree to read2024

Keeping Emmy Cool - White Paper (STULZ Modular, University of Gottingen hybrid liquid/air cooled HPC data centre)

STULZ · Christian Boehme, Sebastian Krey et al.

A case study of the modular data centre built for the 'Emmy' supercomputer at the University of Gottingen. It gives design temperatures, the per-rack liquid/air heat split and the measured PUE results.

On this page

Overview

The facility has a 70 m2 air-cooled room and an 85 m2 direct liquid-cooled (DLC) room, about 1.4 MW in total. The DLC systems total 1,120 kW, with 11 racks installed at 96 kW and space for 14. Most server heat is taken directly from the chip into liquid, and the remainder is handled by precision air cooling and row coolers.

The per-rack split is explicit: of a 96 kW rack, DLC removes 74.9 kW (78 %) and air handles 21.1 kW (22 %). Chip return water reaches about 55 C, supply air runs at 27-35 C and water at 32-36 C. ASHRAE class H1 high-density equipment requires an inlet of 22 C or less.

Cooling relies on dry-cooler free cooling, with 35 % ethylene glycol in the primary loop as antifreeze. Free cooling saves about 40 % and the glycol economiser up to 55 %. Design PUE is 1.14 at 75 % peak load with 32 C ambient and at 100 % load up to 19 C ambient. The measured annual facility PUE is 1.13 against a target below 1.14, with a quoted 1.07 for the liquid-cooled part.

Key points

  1. Measured annual facility PUE 1.13 against a target below 1.14 (1.07 quoted for the liquid-cooled part).
  2. 70 m2 air-cooled room plus 85 m2 DLC room, about 1.4 MW total; 1,120 kW of DLC systems.
  3. 11 racks installed at 96 kW, space for 14.
  4. Per rack, DLC removes 74.9 kW (78 %) and air 21.1 kW (22 %).
  5. Design PUE 1.14 at 75 % peak load at 32 C ambient and 100 % load up to 19 C ambient.
  6. Chip return water up to about 55 C; supply air 27-35 C; water 32-36 C.
  7. 35 % ethylene glycol in the primary loop; free cooling saves about 40 %, the glycol economiser up to 55 %.
  8. ASHRAE H1 high-density equipment requires inlet at or below 22 C.

Using it in design

  • Model the per-rack heat split (liquid %/air %); it drives both CDU and CRAH sizing.
  • Use the Emmy figures (96 kW, 78/22) as a validation fixture.
  • If equipment is ASHRAE H1, cap the supply-air inlet at 22 C and warn on higher setpoints.
  • Give primary loops a glycol fraction field, since it affects cp and pump energy.

Monthly newsletter · free

Keep your reading list current

Once a month: new library sources, changes to Turkish regulations and upcoming data centre events.

SoSo CAD

Check cooling capacity alongside your design

Review zone heat loads, available capacity and the capacity balance after the largest unit is lost in SoSo CAD.

Request a demo
Zone heat loads and capacity balance in the SoSo CAD cooling workspace

Who should read it

Engineers designing hybrid liquid/air cooled HPC and AI halls.

Related standards

  • ASHRAE TC 9.9 (H1 class)
  • EN 50600

Publication details

Authors
Christian Boehme; Sebastian Krey; Dave Meadows; Norbert Wenk
Publisher
STULZ
Year
2024
Document language
English
Source type
Manufacturer
Access
Free to read
Licence
Copyright retained; citation only — the publisher’s terms apply to the original document

Citation

Christian Boehme; Sebastian Krey; Dave Meadows; Norbert Wenk (2024). Keeping Emmy Cool - White Paper (STULZ Modular, University of Gottingen hybrid liquid/air cooled HPC data centre). STULZ. https://www.stulz.com/fileadmin/user_upload/Documents/Downloads/Whitepaper/KEEPING_EMMY_COOL_WHITE_PAPER_STULZ_MODULAR.pdf

All content and rights belong to the respective authors and publishers. SoSo CAD provides only the citation, its own summary and a link to the source, and is not responsible for the sources' content.