Preparing for the Worst: Compartmentalization in Data Center Capacity Planning

Overview

In this 2018 piece in Data Center Knowledge, Chris Crosby argues that no facility is 100% reliable and that cloud-scale sites should therefore be compartmentalized so that a service-affecting incident stays contained. Compartmentalization means physically segmenting applications, hardware and network so that incidents cannot spread.

The author describes two layers: storage/compute clusters, and network nodes serving one or more data halls. On top of these comes inter-meshed connectivity between clusters, nodes and buildings. In the clusters example, a business unit is split into distinct 1 MW clusters to speed fault diagnosis. Each network node supports one or more halls and several clusters, and a cluster failure is contained within its node.

In a block-based design, each power block or hall group should have an independent MEP identity and its own fire zone. Each block needs multiple fibre manholes and multiple points of entry; mesh networking and compartmentalization are complementary. The article is qualitative, with a 1 MW block as its only numeric example. It shows neither the cost impact nor the reliability gain of compartmentalization quantitatively.

Key points

1. Compartmentalization is physically segmenting applications, hardware and network so incidents cannot spread.
2. Clusters example: a business unit is split into distinct 1 MW clusters to speed fault diagnosis.
3. Each network node supports one or more halls and several clusters; a cluster failure is contained within its node.
4. Each block or hall group should have an independent MEP identity and its own fire zone.
5. Each block needs multiple fibre manholes and multiple points of entry.
6. Mesh networking and compartmentalization are complementary.
7. The article is qualitative; the only numeric example is a 1 MW block.

Using it in design

• Model the campus as power blocks with block-level MEP independence and fire zones.
• Check that each block has at least two diverse fibre entry paths.
• Provide a fault-domain view showing which halls and clusters are lost for a given block or node failure.

Who should read it

Architects, capacity planners and reliability engineers designing cloud-scale campuses and block-based data centres.

Publication details

Authors

Chris Crosby

Publisher

Data Center Knowledge

Year

2018

Document language

English

Source type

Expert / community

Access

Free to read

Licence

Copyright retained; citation only — the publisher’s terms apply to the original document

Citation