Scaling Network Infrastructure to Support AI Workload Growth at Hyperscale

meta · Tech · 2024 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: hyperscale AI network infrastructure

Meta needed to accelerate backbone network capacity plans from 2028 into 2024-2025 to support AI workload growth, including geographically distributed AI clusters that exceeded single-site power and space limits.

Constraints:

• AI workloads drove a need for 10x backbone capacity on an accelerated timeline.
• The network had to serve data-center-to-data-center traffic for large AI workloads, not just user-facing POP traffic.
• Fiber construction, optical equipment, power, rack space, and operational disruption constrained scaling choices.

Architecture

Meta scaled its Express Backbone through pre-built metro fiber rings, platform scaling up and out, IP/optical integration with coherent receiver technology, and AI-specific Prometheus backbone designs. Short 3-10 km clusters use direct fiber with distance-appropriate plugs; longer links use DWDM systems that multiplex 64 connections over one fiber pair and horizontally scale 51.2T systems.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["AI data centers"] --> B["Metro fiber rings"]
  B --> C["Remote POPs"]
  B --> D["Long-haul fibers"]
  A --> E["Express Backbone"]
  C --> E
  E --> F["Backbone planes"]
  E --> G["Coherent receiver routers"]
  D --> H["DWDM optical system"]
  H --> G
  H --> E

Components

Data flow

AI data-center traffic flows over EBB, which uses in-house routing and traffic engineering. New data centers connect into prebuilt metro rings and POPs; backbone platforms scale through larger chassis, faster interfaces, and more planes; optical/IP integration reduces transponder power and rack footprint; Prometheus connects distributed AI cluster sites using direct fiber or DWDM depending on distance.

Why this architecture

Meta combined metro prebuilds, platform scaling, and optical/IP integration because AI workload growth forced a 10x capacity schedule into a much earlier window while power, space, and fiber construction remained hard constraints.

Models used: not applicable; network infrastructure for AI workloads

Scale

• Meta pulled network scalability plans from 2028 into 2024 to meet demand.
  “our scalability plans from 2028 had to be pulled into 2024 to meet the network demand.”transcript, stated
• A specific site reached 192x its 2017 platform capacity.
  “achieving a platform capacity that is 192 times the original capacity back in 2017.”transcript, stated
• CR technology reduces network power consumption by 80-90%.
  “In aggregate, we use 80 to 90% less power when we use CR technology.”transcript, stated
• In the 800G CR scenario, one petabit of capacity needs 14 racks instead of 58 racks in the non-CR scenario.
  “In this scenario to build the same one pabit of backbone capacity we just need six optical racks and 80b racks in total 14 racks.”transcript, stated
• DWDM systems carry 64 x 800G, or 51.2T, and can scale horizontally.
  “one of these systems 64 * 800 gig 51.2T and we just horizontally scale that in order to get the desired amount of capacity.”transcript, stated

Alternatives considered (roads not taken)

Wait for network connectivity after site selection

Rejected because: Network was historically painful to build to new data center locations, so metro components were prebuilt.

Rely only on vendor scaling up

Rejected because: Meta also needed internal scale-out controls such as more backbone planes.

Direct fiber pairs for longer-distance cluster connectivity

Rejected because: DWDM reduces fiber needs by 64x and avoids disruptive large-scale trenching over longer distances.

Tradeoffs

Metro readiness

Chose prebuilt metro fiber rings and POP connectivity over build each data center connection from scratch.

Rationale: Prebuilding lets Meta connect new data centers faster as AI capacity demand arrives.

Backbone platform scaling

Chose combine scaling up and scaling out over only larger chassis and faster interfaces.

Rationale: Vendor-dependent upgrades alone were not enough; adding planes gave Meta more direct capacity control.

Optical architecture

Chose coherent receiver integration in routers over standalone transponders with clean IP/optical demarcation.

Rationale: CR technology dramatically reduces power and rack footprint at the cost of blurrier layer separation.

Gotchas (what bit them)

AI demand compressed infrastructure timelines by years.

Root cause: Megawatt compute forecasts and AI workload growth exceeded previous scaling plans.

Resolution: Accelerate 10x backbone techniques into the 2024-2025 window.

Short-distance AI cluster fiber is physically disruptive.

Root cause: Every 400G/800G connection needs a fiber pair, leading to huge conduit and permitting work.

Resolution: Use distance-specific fiber designs and reserve DWDM for longer distances where fiber reduction matters more.

Optical protection switching reduces layer transparency.

Root cause: Protection handled in the optical layer can obscure IP/optical boundaries.

Resolution: Accept the complexity when it substantially reduces IP router interfaces.

Evaluation

Meta evaluated network designs through backbone capacity, rack positions, power footprint, fiber-pair requirements, reach ranges, and deployment disruption.

Metrics: backbone capacity · rack positions · power consumption · fiber pairs · reach distance · capacity per optical system

Outcomes

• Meta found 10x backbone scaling possible through scaling up, scaling out, metro prebuilds, and IP/optical integration.
• CR technology reduced active devices, rack footprint, and network power consumption.
• Prometheus AI backbone designs support geographically distributed AI clusters across 3-100 km ranges.

This is foundational AI infrastructure rather than an LLM application; it matters because hyperscale AI training and serving depend on backbone capacity.