Foundation Model for Personalized Recommendation at Scale

netflix · Media & Entertainment · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: large-scale personalized recommendation

Netflix needed to reduce the maintenance and innovation-transfer burden of many independently trained recommendation models while learning from long member histories under strict recommender-serving latency constraints.

Constraints:

• many specialized models served different recommendation needs despite sharing data sources
• user histories can span thousands of events, but online recommendation calls need millisecond-level latency
• interaction tokens must preserve meaningful information without creating impractically long sequences
• new titles need usable embeddings before engagement data exists
• downstream systems need stable embeddings even as the foundation model is retrained

Architecture

Netflix built a transformer-style recommendation foundation model using autoregressive next-token prediction over tokenized member interactions. The system merges raw actions into meaningful tokens, uses sparse attention and sliding-window sampling for long histories, modifies the objective with multi-token and auxiliary predictions, handles cold-start through incremental training plus metadata/ID embedding mixing, and serves downstream systems through direct prediction, batch embeddings, candidate generation, and fine-tuning.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  member-interaction-stream[Member interaction stream]
  interaction-tokenization[Interaction tokenization]
  autoregressive-foundation-model[Autoregressive foundation model]
  sparse-attention-context-layer[Sparse attention context layer]
  cold-start-title-embedding-layer[Cold-start title embedding layer]
  embedding-stability-layer[Embedding stability layer]
  downstream-recommendation-consumers[Downstream recommendation consumers]
  member-interaction-stream --> interaction-tokenization
  interaction-tokenization --> autoregressive-foundation-model
  autoregressive-foundation-model --> sparse-attention-context-layer
  sparse-attention-context-layer --> autoregressive-foundation-model
  autoregressive-foundation-model --> cold-start-title-embedding-layer
  cold-start-title-embedding-layer --> autoregressive-foundation-model
  cold-start-title-embedding-layer --> embedding-stability-layer
  embedding-stability-layer --> downstream-recommendation-consumers
  downstream-recommendation-consumers --> autoregressive-foundation-model
  downstream-recommendation-consumers --> embedding-stability-layer

Components

Data flow

Member interactions are tokenized into meaningful sequence events that combine context, action, and content metadata. Training samples overlapping windows and uses autoregressive plus auxiliary prediction objectives. The model creates member/entity representations, initializes new-title embeddings with metadata and prior parameters, stabilizes embedding spaces through a low-rank transformation, and exposes predictions or embeddings to downstream recommendation applications.

Why this architecture

The architecture borrows the LLM shift from many specialized models toward a shared foundation model, but adapts it for recommendation realities: unequal event importance, severe latency requirements, new-title cold start, and downstream embedding compatibility. Centralized preference learning lets innovations and learned representations be reused across multiple product canvases.

Models used: Netflix recommendation foundation model · autoregressive next-token prediction · sparse attention · KV caching

Scale

• Netflix had over 300 million users at the end of 2024.
  “With over 300 million users at the end of 2024, this translates into hundreds of billions of interactions”article, stated
• The data scale is hundreds of billions of interactions, comparable to LLM token volume.
  “hundreds of billions of interactions — an immense dataset comparable in scale to the token volume of large language models (LLMs).”article, stated
• Active users may have thousands of interaction events.
  “interaction histories from active users can span thousands of events, exceeding the capacity of transformer models with standard self attention layers.”article, stated
• Recommendation inference often has millisecond latency requirements.
  “these services typically require millisecond-level latency.”article, stated
• Sparse attention can extend context to several hundred events.
  “the model can extend its context window to several hundred events while maintaining computational efficiency.”article, stated
• Netflix explicitly contrasts a 5-minute trailer play with a 2-hour full movie watch when weighting events.
  “a 5-minute trailer play should not carry the same weight as a 2-hour full movie watch.”article, stated

Alternatives considered (roads not taken)

continue operating many specialized recommendation models

Rejected because: Independent models increased maintenance cost and made shared innovation transfer difficult.

use short recent-history windows for preference features

Rejected because: Short windows were driven by serving latency and training cost, but missed comprehensive member history.

use standard transformer self-attention over full active-user histories

Rejected because: Thousands of events exceed standard self-attention capacity under recommender latency constraints.

use plain single-token next-item prediction

Rejected because: Netflix wanted longer-term dependency learning and auxiliary targets to reduce noisy item-ID overfitting.

rely only on learned ID embeddings for titles

Rejected because: Cold-start titles need metadata-derived embeddings before interaction data accumulates.

Patterns

retrieval-ranking (personalization)

The foundation model supports prediction, candidate generation, title-to-title recommendations, and other personalized ranking applications.

fine-tuning-adaptation (model-adaptation)

Downstream teams can fine-tune the full foundation model or subgraphs with less data and compute for application-specific needs.

eval-driven-development (model-quality)

Netflix treats scaling as dependent on evaluation that can distinguish model quality and expose improvement areas as data and model size grow.

Tradeoffs

Move from specialized models to a foundation model.

Chose centralized data-centric preference learning over many small independent personalization models.

Rationale: The unified model reduces maintenance and makes learned preferences reusable across downstream applications.

Use sliding windows instead of full-sequence training contexts.

Chose overlapping sampled windows from full histories over requiring an impractically large transformer context window.

Rationale: Sliding windows expose the model to different history segments over epochs while keeping training practical.

Weight recommendation events differently from language tokens.

Chose multi-token and auxiliary prediction objectives over equal weighting of all target tokens.

Rationale: Recommendation interactions vary in importance, and auxiliary targets help capture intent while regularizing noisy item prediction.

Use metadata for cold-start entities.

Chose age-aware metadata and ID embedding mixture over ID-only embeddings for all titles.

Rationale: New titles should lean on metadata until engagement-specific ID signals become reliable.

Stabilize embeddings for downstream consumers.

Chose orthogonal low-rank transformation over raw embedding spaces from each retraining run.

Rationale: Stable dimensions reduce invalidated assumptions and bugs when the foundation model changes.

Gotchas (what bit them)

Specialized recommender models create costly maintenance and blocked innovation transfer.

Root cause: The models are trained independently even when they rely on common data sources.

Resolution: Centralize member preference learning in a reusable foundation model.

Recommendation tokenization can either erase signal or exceed processing limits.

Root cause: Raw user actions need compression, but overly lossy tokens drop preference information.

Resolution: Merge adjacent actions into meaningful interaction tokens while preserving details such as duration and engagement type.

Serving-latency constraints cap recommender context windows.

Root cause: Recommendation services require millisecond latency, unlike many LLM use cases where seconds can be tolerable.

Resolution: Use sparse attention, sliding-window sampling, and KV caching to trade off history depth and serving cost.

Cold-start titles lack engagement history.

Root cause: New catalog entities appear before training data includes enough member interactions.

Resolution: Warm-start embeddings and combine metadata-based and ID-based embeddings with an age-aware attention mixer.

Embedding consumers can break after each foundation-model retrain.

Root cause: Embedding dimensions are arbitrary and incompatible across different training runs.

Resolution: Stabilize the user/item embedding space with an orthogonal low-rank transformation.

Evaluation

Netflix evaluates the model through scaling-law experiments, downstream integrations, recommendation quality, embedding usefulness, and evaluation methods that differentiate performance as data, model size, and context scale increase.

Metrics: over 300 million users · hundreds of billions of interactions · millisecond-level serving latency · several-hundred-event sparse-attention context · thousands of events in active histories · consistent improvements with increased data and model size · promising results from downstream integrations

Outcomes

• Netflix created a unified foundation model for personalized recommendation.
• The model supports direct predictions, embeddings, candidate generation, and fine-tuning for downstream applications.
• Experiments confirmed scaling-law behavior as data and model size increased.
• The architecture addresses recommendation-specific cold start and presentation-bias constraints rather than directly copying LLM designs.
• Embedding stabilization makes retrained foundation-model outputs safer for downstream consumers.

The article reports over 300 million users, hundreds of billions of interactions, active histories spanning thousands of events, millisecond serving latency, and several-hundred-event sparse attention contexts.