Scaling AI Infrastructure: Managing Data Movement and Placement on Meta's Global Backbone Network

meta · Tech · 2022 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: global backbone networking for AI data movement

Meta's AI workloads transformed backbone-network demand: GPU requests grew quickly, traffic moved beyond the data-center region, data movement became exabyte-scale, and demand became volatile enough that traditional capacity assumptions were no longer sufficient.

Constraints:

• Meta's global backbone connects more than 25 data centers and 85 points of presence through millions of miles of fiber.
• AI demand from product groups grew more than 100% year over year and became harder to predict.
• AI data lifecycle stages require freshness, persistence, repeated movement, and exabyte-scale transfers.
• Data placement involves volatile demand/supply signals, dozens of product groups, dozens of physical sites, multiple buildings, hardware SKUs, and millions of variables and constraints.
• The backbone is shared by all Meta workloads, so AI traffic must be optimized without starving non-AI products.

Architecture

Meta responded by treating backbone capacity as a cross-stack optimization problem. The team improved data placement, reduced cross-region reads through caching and smarter placement, added observability into dataset flows, differentiated quality-of-service classes, and continued expanding supply through fiber, space, power, and cooling. The architecture tries to bend the demand curve while meeting AI's freshness, volume, placement, and latency requirements.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  A["Product-group demand signals"] --> B["Data placement planner"]
  C["Supply constraints"] --> B
  B --> D["Cross-region read optimization"]
  D --> E["Storage teams"]
  F["Data ingestion pipeline"] --> G["Data preparation and replication"]
  G --> H["Global backbone network"]
  H --> I["QoS network classes"]
  J["Observability of data flows"] --> B
  J --> I
  C --> K["Supply expansion"]
  G --> L["AI workloads"]

Components

Data flow

Fresh user and machine data is ingested globally, persisted in primary and secondary copies, prepared for AI workloads, and moved across Meta's backbone when training or processing needs require it. Product groups express AI demand, supply constraints shape hardware and site placement, and data-placement plans decide when, where, what, and how much demand to place. Observability then identifies wasteful cross-region reads or over-replicated data so caching, placement, and QoS policies can reduce backbone pressure.

Why this architecture

Meta could not solve AI backbone pressure by adding network capacity alone because the demand curve was shaped by product demand, data freshness, data placement, storage strategy, workload latency, and supply constraints. The architecture therefore combines demand reduction through smarter placement/caching, visibility through flow observability, differentiated service classes, and supply expansion.

Models used: AI workloads · generative AI · large model training

Scale

• Meta's backbone connects over 25 data centers and 85 points of presence.
  “connects over 25 data centers and 85 points of presence”transcript, stated
• Meta's backbone uses millions of miles of terrestrial and subsea fiber.
  “we do this through millions of miles of fiber in both terrestrial and subc routes”transcript, stated
• Backbone capacity grew more than 30% year over year for five years.
  “with a backbone capacity that has grown over 30% year of a year for the Last 5 Years”transcript, stated
• GPU demand from product groups grew more than 100% year over year.
  “demand for gpus from our product groups grew over 100% year-over-year”transcript, stated
• The data lifecycle moves data at exabyte scale.
  “the volume of data that has to be moved which is actually in the scale of exabytes”transcript, stated
• The placement problem spans millions of variables and constraints.
  “you end up with millions of variables and millions of constraints all of which have to be navigated”transcript, stated
• Cross-region reads were reduced to two-thirds of the data that has to cross when new data lands.
  “we're now limiting our cross region reads to just 2/3 of data that has to go across when new data lands”transcript, stated

Alternatives considered (roads not taken)

Assume AI traffic stays inside the data center or region

Rejected because: AI drove higher-than-anticipated backbone traffic and Meta missed parts of the AI lifecycle.

Solve by network supply alone

Rejected because: Meta also needed to bend demand through cross-stack data placement, caching, and observability.

Move all fresh data across the backbone whenever another region needs it

Rejected because: Unintentional cross-region reads become untenable at AI scale.

Uniform latency treatment for all traffic

Rejected because: Different workloads need different latency guarantees, so differentiated service classes are more efficient.

Patterns

observability-tracing (operations)

Meta instruments data flows to understand useful datasets, waiting data, over-replication, and backbone demand.

data-quality-loop (data)

Meta tunes data placement and replication based on whether datasets are actually useful to AI workloads.

latency-optimization (infrastructure)

Meta uses differentiated quality-of-service classes because not all workloads need the same latency guarantees.

cost-aware-routing (infrastructure)

Meta trades backbone reads against caching and placement using cost-of-ownership reasoning.

batch-offline-processing (infrastructure)

The case focuses on large-scale AI data preparation, persistence, replication, and movement across the backbone.

Tradeoffs

Demand management strategy

Chose cross-stack optimization and demand-curve bending over backbone-only capacity expansion.

Rationale: AI pressure depends on compute, storage, placement, workload latency, and supply constraints, not only network links.

Cross-region reads

Chose more caching and better data placement over letting fresh data reads traverse the backbone by default.

Rationale: The trade reduced backbone burden while preserving access to fresh AI data.

Traffic classes

Chose differentiated QoS guarantees over one uniform service class for all workloads.

Rationale: Workloads have different latency needs, so differentiated classes let the shared backbone serve AI and non-AI traffic more efficiently.

Gotchas (what bit them)

AI traffic exceeded data-center-region assumptions.

Root cause: Meta initially focused only on reading storage to feed GPUs and missed other AI lifecycle stages.

Resolution: The team broadened characterization of AI's backbone impact and optimized across the stack.

AI demand signals are volatile and hard to forecast.

Root cause: Product needs and model requirements evolve dynamically, while supply depends on construction, regulation, geopolitics, and market availability.

Resolution: Meta treats placement as an optimization problem with demand, supply, site, building, and hardware constraints.

AI workloads are less fungible across hardware.

Root cause: A100 and H100 servers have different network and hardware preferences, making migration harder.

Resolution: Placement planning accounts for hardware SKU and migration constraints.

Fresh data can create untenable cross-region read traffic.

Root cause: When fresh data lands in one region, other regions may fetch it over the backbone unless placement and caching are deliberate.

Resolution: Meta works with storage teams to add caching and better placement, reducing cross-region reads to two-thirds.

Evaluation

Meta evaluates the backbone strategy through demand growth, volatility, capacity growth, data-placement complexity, cross-region read volume, data-flow observability, QoS needs, and supply expansion.

Metrics: Over 25 data centers · 85 points of presence · Millions of miles of fiber · Backbone capacity over 30% year-over-year for five years · GPU demand over 100% year-over-year · Exabyte-scale data movement · Millions of variables and constraints · Cross-region reads reduced to two-thirds

Outcomes

• Meta identified AI as a fast-growing and volatile backbone traffic driver.
• The team reduced cross-region reads to two-thirds for fresh data scenarios through caching and placement.
• Meta added better data-flow observability to find useful datasets and unnecessary movement.
• Meta maintained differentiated QoS classes for workloads with different latency needs.
• The organization continues to expand supply through space, power, cooling, and fiber.

Meta reports a global backbone connecting over 25 data centers and 85 points of presence, millions of miles of fiber, 30%+ annual capacity growth for five years, 100%+ annual GPU demand growth, exabyte-scale data movement, millions of placement variables and constraints, and cross-region reads reduced to two-thirds.