Integrating Foundation Models into Production Personalization Systems

netflix · Media & Entertainment · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production personalization model integration

Netflix had built a centralized personalization foundation model, but the hard production problem was how to integrate that large model into existing recommendation systems with different latency requirements, technology stacks, freshness needs, cost constraints, and appetite for model depth. The goal was not just to train a transformer model, but to make its representations useful across production personalization applications without overburdening every downstream team.

Constraints:

• the Netflix homepage was powered by several specialized models
• individual models required significant maintenance and innovation effort
• large transformer-model training and inference guidance did not fully answer production integration questions
• application teams had different latency requirements and technology stacks
• some applications could tolerate stale embeddings while others needed fresher representations
• subgraph integration increased feature-generation complexity, model size, and inference time
• fine-tuned application models increased the number of models and pipelines to maintain
• latency and service-level agreements had to be optimized per application

Architecture

Netflix integrated its personalization foundation model through three production paths. The lowest-friction path generates user and item embeddings from the foundation model, refreshes them through monthly pretraining, daily fine-tuning, batch inference, and an Embedding Store, then lets existing pipelines consume them as features or retrieval inputs. A deeper subgraph path embeds the foundation-model decoder stack inside downstream application model graphs, eliminating embedding staleness and exposing more layers at the cost of feature-generation complexity, model size, and inference time. A direct fine-tuning path lets application teams train product-specific variants with custom objectives, output heads, and frozen or unfrozen layers. Platform teams supported these options with stabilization, reusable code/jobs, near-real-time embedding work, distillation work, and standardized APIs.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  Netflix_Foundation_Model["Netflix Foundation Model"] --> Monthly_pretraining_job["Monthly pretraining job"]
  Monthly_pretraining_job --> Daily_fine_tuning_job["Daily fine-tuning job"]
  Daily_fine_tuning_job --> Batch_embedding_inference["Batch embedding inference"]
  Batch_embedding_inference --> Profile_embeddings["Profile embeddings"]
  Batch_embedding_inference --> Item_embeddings["Item embeddings"]
  Profile_embeddings --> Embedding_Store["Embedding Store"]
  Item_embeddings --> Embedding_Store
  Monthly_pretraining_job --> Embedding_stabilization["Embedding stabilization"]
  Daily_fine_tuning_job --> Embedding_stabilization
  Embedding_stabilization --> Embedding_Store
  Embedding_Store --> Candidate_generation["Candidate generation"]
  Embedding_Store --> Downstream_personalization_models["Downstream personalization models"]
  Netflix_Foundation_Model --> Foundation_Model_subgraph["Foundation Model subgraph"]
  Application_feature_generation_jobs["Application feature generation jobs"] --> Foundation_Model_subgraph
  Reusable_feature_generation_jobs["Reusable feature-generation jobs"] --> Application_feature_generation_jobs
  Foundation_Model_subgraph --> Downstream_personalization_models
  Netflix_Foundation_Model --> Fine_tuning_framework["Fine-tuning framework"]
  Fine_tuning_framework --> Fine_tuned_foundation_models["Fine-tuned foundation models"]
  Fine_tuned_foundation_models --> Downstream_personalization_models
  Near_real_time_embedding_framework["Near-real-time embedding framework"] --> Embedding_Store
  Distilled_foundation_model["Distilled foundation model"] --> Foundation_Model_subgraph

Components

Data flow

Netflix trains the foundation model from comprehensive user interaction histories and content data. For the embedding path, monthly pretraining and daily fine-tuning produce a current model; batch inference extracts profile embeddings from the last user-event hidden state and item embeddings from item-tower weights, maps the embeddings into a stable space, and publishes them to the Embedding Store for offline and online consumers. For subgraph integration, the application model generates the raw sequence features needed by the foundation-model decoder stack, runs the subgraph once per profile per request, and shares the representation across items. For fine-tuning, application teams adapt the foundation model to product-specific data and objectives, optionally freezing layers or adding a new output head.

Why this architecture

Netflix chose a portfolio of integration paths because no single path satisfied every product surface. Embeddings offered low-cost, low-friction reuse through existing pipelines, subgraph integration unlocked fresher and deeper foundation-model representations for high-impact cases, and fine-tuning let teams optimize the model for application-specific objectives. Platform abstractions such as the Embedding Store, reusable feature-generation jobs, fine-tuning framework, stabilization, distillation, and standardized APIs let teams choose an integration depth that matched their latency, cost, and benefit profile.

Models used: Netflix Foundation Model for personalization · large-scale transformer-based model · Foundation Model decoder stack · fine-tuned foundation-model variants · planned smaller distilled Foundation Model

Scale

• Netflix tested three production integration approaches: embeddings, subgraph, and fine-tuning.
  “At Netflix, we experimented with 3 distinct integration approaches — embeddings, subgraph and fine-tuning.”article, stated
• The three approaches are already used in production for different use cases.
  “we go over the 3 approaches we experimented with and which are now used in production for different use cases.”article, stated
• The embedding path retrains from scratch monthly and fine-tunes daily.
  “First, the Foundation Model is pre-trained from scratch every month. Then, the monthly pre-trained model is fine-tuned everyday based on the latest data.”article, stated
• Daily fine-tuning expands the entity ID space for newly launching titles.
  “The daily fine-tuning also expands the entity id space to include the newly launching titles.”article, stated
• The near-real-time direction updates embeddings based on user actions during the session.
  “This new framework also enables us to update the embeddings based on user actions during the session”article, stated

Alternatives considered (roads not taken)

Embeddings from the Embedding Store as the only integration path

Rejected because: Embeddings are low-cost and easy to adopt, but they can be stale and may not unlock the full value of the foundation model for real-time or high-adaptability use cases.

Foundation Model subgraph for every application

Rejected because: Subgraph integration gives deeper value, but Netflix reserves it for high-impact use cases because it increases feature generation complexity, model size, inference time, compute cost, and training-code complexity.

Direct fine-tuned foundation models for all product surfaces

Rejected because: Direct fine-tuning adapts well to application objectives, but it can create more models and pipelines to maintain and requires per-application latency/SLA optimization.

Current daily batch embeddings for all freshness needs

Rejected because: Daily refresh was valuable but left staleness for session-adaptive recommendations, prompting investment in near-real-time embedding generation.

Full-size foundation model in latency-sensitive subgraph paths

Rejected because: Netflix is working on a smaller distilled model because the subgraph approach carries inference-latency pressure.

Patterns

batch-offline-processing (operations)

Netflix refreshed embeddings through monthly pretraining, daily fine-tuning, batch inference, and publication to the Embedding Store.

fine-tuning-adaptation (modeling)

Application teams can fine-tune the foundation model on product-specific data, freeze layers, or attach new output heads for application-specific objectives.

latency-optimization (operations)

Netflix split the foundation-model subgraph so it runs once per profile per request, works on distillation, and optimizes fine-tuned models against application SLAs.

cost-aware-routing (architecture)

Netflix routes applications among embeddings, subgraph integration, and fine-tuning depending on latency, tech-stack constraints, metric wins, compute cost, and complexity.

data-quality-loop (data)

The foundation model refresh pipeline updates daily on latest data and expands entity IDs for newly launched titles, keeping representations current for downstream systems.

model-distillation (modeling)

Netflix is developing a smaller distilled foundation model to reduce latency when the foundation model is used as a subgraph.

Tradeoffs

Expose foundation-model learning through embeddings first.

Chose profile and item embeddings served by the Embedding Store over requiring every application to embed the full foundation model in its serving path.

Rationale: Embeddings offered a low-cost, high-leverage starting point that fit existing pipelines, but traded off freshness and full model expressiveness.

Use subgraph integration for high-impact applications.

Chose foundation-model decoder stack inside the downstream model graph over static embeddings from the Embedding Store.

Rationale: The subgraph path removed embedding staleness and exposed more foundation-model layers, but increased feature-generation cost, model size, inference time, and training complexity.

Support direct fine-tuning for product-specific surfaces.

Chose custom fine-tuned foundation-model versions over new model stacks and months of feature engineering.

Rationale: Fine-tuning let teams adapt objectives and reuse the foundation model as a baseline, but increased model/pipeline maintenance and SLA tuning burden.

Invest in near-real-time embedding generation.

Chose session-adaptive embedding updates over daily batch embedding refresh only.

Rationale: Near-real-time updates address staleness and adaptiveness, but Netflix notes that the framework cannot be used for very large models.

Standardize APIs while preserving multiple integration paths.

Chose refined and standardized APIs across embeddings, subgraph, and fine-tuning approaches over bespoke integrations per application team.

Rationale: The three-pattern portfolio remained flexible, while API standardization reduced adoption friction for application teams.

Gotchas (what bit them)

Embedding spaces changed across model refreshes.

Root cause: Monthly pretraining starts from random initialization and daily fine-tuning drifts from previous-day embeddings.

Resolution: Netflix used embedding stabilization so downstream models could consume pre-computed embeddings in the same space.

Precomputed embeddings could become stale before inference.

Root cause: There was a time gap between embedding computation and downstream recommendation inference.

Resolution: Netflix invested in a near-real-time embedding generation framework that can update embeddings from in-session user actions.

Subgraph integration added feature-generation and inference complexity.

Root cause: Application models had to generate all features required by the foundation-model subgraph, and merging the subgraph increased model size and runtime.

Resolution: The foundation-model team provided reusable code/jobs and optimized inference by running the subgraph once per profile per request.

Direct fine-tuning created operational sprawl.

Root cause: Application-specific fine-tuning can produce many models and pipelines across the organization.

Resolution: Netflix built a fine-tuning framework and treated latency/SLA optimization as application-specific work.

Large production models lacked practical integration guidance.

Root cause: Most available literature focused on training and inference rather than integration into existing production systems.

Resolution: Netflix treated integration itself as the engineering problem and documented embeddings, subgraph, and fine-tuning as production patterns.

Evaluation

The article is primarily an integration-design report rather than an evaluation paper. It evaluates approaches qualitatively through production use cases, application needs, latency constraints, cost/complexity tradeoffs, staleness, metric wins, and platform adoption. It reports that all three integration approaches are used in production, but does not disclose offline metric values, A/B-test sizes, or recommendation KPI lifts.

Metrics: latency requirements · tech stack constraints · metric wins · compute cost · inference time · embedding freshness · model and pipeline maintenance burden · service-level agreements

Outcomes

• Netflix now uses embeddings, subgraph integration, and fine-tuning in production for different personalization use cases.
• Embedding Store integration gives applications a low-barrier way to consume foundation-model profile and item representations.
• Subgraph integration gives high-impact applications fresher and deeper foundation-model representations.
• Fine-tuning gives application teams custom product-specific variants and a default baseline for new applications.
• Platform investment expanded to near-real-time embeddings, smaller distilled subgraph models, reusable feature-generation jobs, fine-tuning frameworks, and standardized APIs.

The article discloses three production integration approaches, monthly pretraining, daily fine-tuning, daily embedding refresh, and in-session near-real-time embedding goals. It does not disclose user counts, traffic volume, model size, or KPI lift.