Foundation Model for Unified Personalization at Scale

netflix · Media & Entertainment · 2024 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: unified personalization and recommendation systems

Netflix wanted one transformer-based foundation model to cover diverse recommendation use cases that had previously been handled by many independently built models, feature pipelines, labels, and canvases.

Constraints:

• Netflix recommendations span rows, entities, pages, kids experiences, mobile feeds, games, live streaming, and other content types
• independent models duplicated feature and label engineering and reduced innovation velocity
• recommendation tokens are rich interaction events, not simple language-token IDs
• downstream canvases need stable user and content representations that can be adapted or fine-tuned
• larger recommendation models face strict latency and cost constraints that may require distillation

Architecture

Netflix built a unified personalization foundation model based on autoregressive transformers. The model tokenizes rich interaction events, encodes event representations across when/where/what fields, combines ID embeddings with semantic content information for cold-start behavior, uses transformer hidden states as long-term user representations, supports multi-sequence/multi-target objectives, and exposes the foundation model through subgraph integration, embedding stores, fine-tuning, or distillation for downstream recommendation applications.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  interaction-event-stream[Interaction event stream]
  tokenization-layer[Tokenization layer]
  event-representation-layer[Event representation layer]
  embedding-feature-layer[Embedding feature layer]
  transformer-representation-layer[Transformer representation layer]
  objective-layer[Objective layer]
  downstream-consumption-layer[Downstream consumption layer]
  centralized-embedding-store[Centralized embedding store]
  interaction-event-stream --> tokenization-layer
  tokenization-layer --> event-representation-layer
  event-representation-layer --> embedding-feature-layer
  embedding-feature-layer --> transformer-representation-layer
  transformer-representation-layer --> objective-layer
  objective-layer --> transformer-representation-layer
  transformer-representation-layer --> downstream-consumption-layer
  downstream-consumption-layer --> transformer-representation-layer
  downstream-consumption-layer --> centralized-embedding-store
  centralized-embedding-store --> downstream-consumption-layer

Components

Data flow

Netflix starts with diverse user interaction histories across pages, rows, and entity types. Tokenization turns each interaction into a rich event token with time, location, device, canvas, target entity, interaction type, duration, and metadata. The foundation model learns user and content representations through autoregressive transformer layers and multi-target objectives. Downstream applications consume the model as a subgraph, pull member/content embeddings from a centralized store, or fine-tune/distill the model for strict-latency canvases.

Why this architecture

The foundation-model architecture exists to centralize user and content representation learning across Netflix's many recommendation canvases. The transcript argues that scaling semi-supervised transformer learning should improve personalization and that integrating a shared foundation model into downstream systems creates leverage, faster innovation, and easier launch of new applications.

Models used: Netflix personalization foundation model · autoregressive transformer · multi-token prediction · multi-layer representation · sparse attention · distillation

Scale

• Netflix's stated goal was one foundation model covering all recommendation use cases.
  “use one foundation model to cover all the recommendation use cases.”transcript, stated
• The model scaled over roughly two to two and a half years.
  “over the uh roughly two two to two and a half years we were scaling up”transcript, stated
• Netflix reports scaling from millions of profiles to around one billion model parameters.
  “from only on the order of 10 million profile or a few million profile to now on the order of 1 billion uh model parameters.”transcript, stated
• Recommendation latency and cost requirements constrained further scaling.
  “recommendation system usually have much stringent latency cost requirement. So scaling up scaling up more require us to also distill back.”transcript, stated
• The content space includes movies, TV shows, games, live streaming, and more planned entity types.
  “in addition to traditionally movie and TV shows now we have games we have live streaming and we are going to add more”transcript, stated

Alternatives considered (roads not taken)

continue spinning up independent specialized models for each canvas or use case

Rejected because: The approach duplicated feature and label engineering, offered little leverage, and slowed innovation.

use one generic tokenization for every stage and application

Rejected because: Tokenization choices affect model quality downstream and may need different abstractions for pretraining versus application fine-tuning.

learn only ID embeddings from scratch

Rejected because: Pure ID embeddings fail for unseen titles at inference time, so semantic content information is needed.

use only next-entity prediction as the objective

Rejected because: Recommendation events have many facets that can be targets, labels, weights, rewards, or masks for fine-tuning.

scale model size without distillation

Rejected because: Recommendation serving has strict latency and cost requirements, so larger models may need to be distilled back for online use.

Patterns

retrieval-ranking (personalization)

The foundation model centralizes recommendation representations used by many downstream canvases and applications.

fine-tuning-adaptation (model-adaptation)

Downstream teams can integrate the foundation model as a subgraph, use embeddings, or fine-tune/distill it for specific applications and latency requirements.

eval-driven-development (model-quality)

Netflix validates scaling behavior and metric improvements as it scales data, parameters, context, and LLM-inspired techniques such as multi-token prediction.

Tradeoffs

Centralize learning in one foundation model.

Chose shared user and content representation learning over independent per-canvas personalization models.

Rationale: Centralization increases leverage, reduces duplicated engineering, and lets downstream models become thinner layers over shared representations.

Adapt LLM techniques instead of copying them directly.

Chose recommendation-specific tokenization, objectives, and cold-start handling over plain language-token style autoregressive modeling.

Rationale: Recommendation interactions contain richer event fields and cold-start constraints that language models do not face.

Expose multiple consumption patterns for downstream models.

Chose subgraph integration, embedding push, fine-tuning, and distillation over a single fixed serving interface.

Rationale: Different canvases and applications have different latency, model-shape, and adaptation needs.

Use multi-target objectives rather than only item-next-token prediction.

Chose multiple sequences, heads, labels, rewards, masks, and metadata targets over only predicting next entity IDs.

Rationale: Multiple event facets can be used to adapt the model to subcategories of user behavior and downstream objectives.

Stop current scaling before latency/cost becomes untenable.

Chose current-scale model plus future distillation over unbounded online model scaling.

Rationale: Netflix says scaling laws continue, but serving requirements force compression or distillation work.

Gotchas (what bit them)

Specialized personalization models created duplicate feature and label engineering.

Root cause: Many models were developed independently for rows, pages, entities, and use cases even though they shared underlying interaction data.

Resolution: Centralize user and content representation learning in one foundation model and make downstream models thinner.

Recommendation event tokens are richer than language tokens.

Root cause: Each user event includes when, where, target entity, action type, duration, page/canvas, and other fields.

Resolution: Design tokenization and event representations carefully, with abstraction choices that can vary by pretraining or fine-tuning use case.

ID-only embeddings cannot handle unseen titles.

Root cause: Titles not present during training have no learned ID embedding signal at inference time.

Resolution: Combine ID embeddings with semantic content information for content-side cold start.

User representations must remain stable while profiles keep changing.

Root cause: The model uses transformer hidden states as user representations, but user histories continuously evolve.

Resolution: Consider sequence aggregation, layer aggregation, and downstream adaptation when exposing representations.

More scaling eventually collides with online serving cost.

Root cause: Recommendation systems have stricter latency and cost requirements than many LLM settings.

Resolution: Use distillation or other compression when integrating larger foundation models into strict-latency downstream applications.

Evaluation

Netflix evaluates the foundation model through scaling-law validation, metric improvements from LLM-inspired techniques, downstream-canvas leverage, representation stability, and whether new applications can fine-tune or distill the shared model more easily.

Metrics: scaling from millions of profiles to around 1 billion model parameters · roughly two to two and a half years of scaling work · notable metric improvement from multi-token prediction · improved leverage across downstream canvas-facing models · faster launch of new applications through fine-tuning

Outcomes

• Netflix consolidated user and content representation layers into a shared foundation model for personalization.
• Downstream application models became thinner layers built on top of the foundation model.
• Scaling-law behavior was validated for recommendation foundation models.
• Multi-token prediction produced notable metric improvement by reducing myopic next-action focus.
• Future directions include universal heterogeneous-entity representations, generative retrieval, and prompt-tuning-style adaptation.

The talk describes Netflix moving toward one foundation model for all recommendation use cases, scaling over roughly 2-2.5 years from millions of profiles to about 1 billion model parameters, and using the model across multiple canvases, content types, and downstream applications.