Enhancing Knowledge Graphs with LLMs at Netflix

contextual_ai · Media & Entertainment · year n/a · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Entertainment knowledge graph enrichment

Netflix operates a large entertainment knowledge graph used across teams and platforms. Building and enriching it requires entity matching over millions of entities and billions of candidate pairs, plus newer LLM-based inference for entity types and semantic relationships that are not explicitly available in source data.

Constraints:

Entity matching must be accurate across tricky cases such as sequels, translated titles, and common movie names.
Candidate generation can still produce hundreds of millions or billions of pairs.
The pipeline must run daily and in parallel without bottlenecking on parsing, feature computation, or output writes.
Production models must not be overwritten by experiments or branch-specific models.
LLM-based extraction must fit into existing Metaflow operations with configurable models and resources.

Architecture

Netflix models entity matching as a parallel classification pipeline in Metaflow, using Apache Arrow-backed fast data loading, partitioned feature computation, independent parallel writes, resource tuning, and project/branch namespacing. For LLM enrichment, the team explores a RAG-style pipeline that queries graph inputs, applies entity-type and relationship extraction with rule-based or LLM models, and integrates new semantic relationships back into the graph.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  entertainment-knowledge-graph["Entertainment knowledge graph"] --> entity-matching-pipeline["Entity matching pipeline"]
  entity-matching-pipeline --> fast-data-layer["Fast data layer"]
  fast-data-layer --> parallel-partition-workers["Parallel partition workers"]
  parallel-partition-workers --> entertainment-knowledge-graph
  entity-matching-pipeline --> metaflow-operations-ui["Metaflow operations UI"]
  entertainment-knowledge-graph --> graph-inference-pipeline["Graph inference pipeline"]
  graph-inference-pipeline --> llm-relationship-extractor["LLM relationship extractor"]
  llm-relationship-extractor --> entertainment-knowledge-graph
  entertainment-knowledge-graph --> downstream-applications["Downstream applications"]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Entertainment knowledge graph`	Graph connecting movies, actors, directors, countries, source adaptations, genres, and other content semantics.	`Entity matching pipeline`, `Graph inference pipeline`, `Downstream applications`
`Entity matching pipeline`	Metaflow workflow that classifies candidate entity pairs at scale.	`Entertainment knowledge graph`, `Fast data layer`, `Parallel partition workers`, `Metaflow operations UI`
`Fast data layer`	Apache Arrow-based parsing/dataframe layer that speeds up large metadata loading.	`Entity matching pipeline`, `Parallel partition workers`
`Parallel partition workers`	Distributed workers that compute features, run matching, and write partitions independently.	`Entity matching pipeline`, `Fast data layer`, `Metaflow operations UI`, `Entertainment knowledge graph`
`Metaflow operations UI`	Operational surface for debugging skew, failed instances, resource use, and model branch separation.	`Entity matching pipeline`, `Parallel partition workers`, `Graph inference pipeline`, `Entertainment knowledge graph`
`Graph inference pipeline`	Work-in-progress LLM/RAG flow for extracting entity types and relationships from graph and unstructured inputs.	`Entertainment knowledge graph`, `LLM relationship extractor`, `Metaflow operations UI`
`LLM relationship extractor`	Model component that uses prompts, examples, and context to infer semantic graph relationships.	`Graph inference pipeline`, `Entertainment knowledge graph`
`Downstream applications`	Search, prediction, embeddings, and other Netflix systems that use graph semantics.	`Entertainment knowledge graph`

Data flow

Source data is turned into candidate pairs or graph inputs. Metaflow partitions the data, computes features or extraction outputs across nodes, writes partitions independently, and marks the table done when all outputs finish. For LLM enrichment, graph inputs are queried through a RAG process, passed into relationship/entity-type extraction, and integrated as new graph relationships.

“take the data um partition table just split the data into multiple nodes as you want and individually each node can handle the matching problem optimize it”transcript, stated

“you take the input graph you have some kind of ETL uh which channels some of the no uh input graph uh for where inference is required and we can all of these will be used for some kind of rag process”transcript, stated

Why this architecture

Entity matching at Netflix scale is an embarrassingly parallel classification problem if the data can be partitioned and written independently. Metaflow adds operational affordances for retries, resource tuning, and branch isolation, while the LLM enrichment path reuses the same workflow flexibility for experimenting with different models.

“the good part of the way modeling it as classification problem is uh you can just is a very simply paralyzable uh system right”transcript, stated

“this particular last piece is models metaflow job again because of the same reason that we need flexibility of what kind of model we can use tune resources change underneath”transcript, stated

Models used: Classification models for entity matching · Rule-based NLP models · LLM extraction models · Graph embeddings

Scale

The knowledge graph is widely used across different Netflix teams and platforms.
“we have a large Knowledge Graph specifically meant for entertainment data uh in Netflix which is widely used across different teams different platforms”transcript, stated
The entity matching problem involves millions of entities and hundreds of millions or billions of candidate pairs.
“when you deal with lot of data uh millions of entities is even if you have a good way of generating candidate payers you will end up with billions of payers”transcript, stated
Apache Arrow-backed parsing produced a 10x speedup in data parsing.
“we actually out of the gate found 10x speed up just parsing the data itself so that was like a a good win to start with”transcript, stated
The matching system can run for billions of pairs every day.
“one of the big change we made to kind of get the system onboarded so that we can run for billions of pairs like every day”transcript, stated
The speaker frames runtime as a resource-scaling choice, such as adding machines to finish in 30 minutes.
“you can just scale it like you want to do it in 30 minutes you just have to add more machines to it right”transcript, stated

Alternatives considered (roads not taken)

Run entity matching as a monolithic Spark-style job

Rejected because: The team struggled with debugging and log access in the previous PySpark system.

“this is something we struggled with in spice Park in the past but with flow now um we can do like a failed instance we can restart that locally”transcript, stated

Gather all partitions before writing output

Rejected because: A shuffle or gather phase would create a bottleneck in the parallel system.

“there is no U Shuffle or a gather phase you can just write each of those partition and just tell the data system that the table is done”transcript, stated

Use only explicit graph data

Rejected because: Some useful entity types and relationships are not explicitly available but can be inferred from context and world knowledge.

“this information may not be explicitly available but based on all the information we have and even lm's World Knowledge we could inform many of these”transcript, stated

Patterns

`batch-offline-processing` (data)

Netflix runs the entity matching workflow as a daily parallel batch pipeline over large candidate-pair tables.

“you start the um pipeline every day you have a small step which kind of makes sure the model is not deteriorated uh and then you split the input”transcript, stated

`durable-execution` (orchestration)

Metaflow provides restart, debugging, resource tuning, and branch-isolation mechanics for large workflow runs.

“we can do like a failed instance we can restart that locally just a failed instance put more debugging log and figure out maybe a particular data issue has caused that issue”transcript, stated

`rag-retrieval` (retrieval)

The LLM graph-enrichment path queries graph inputs through a RAG process before relationship extraction.

“all of these will be used for some kind of rag process to kind of query and then you have this module here which does the entity type and relationship extraction”transcript, stated

`structured-output-contract` (output)

LLM enrichment converts unstructured documents and ontology prompts into graph structures and relationships.

“converting struct unstructured document to structured document you could take an elaborate ontology prompt which is a graph structure”transcript, stated

Tradeoffs

Model entity matching as classification

Chose parallel classification over candidate pairs over less structured pairwise matching logic.

Rationale: The classification framing makes the work easy to partition and scale across nodes.

“the way modeling it as classification problem is uh you can just is a very simply paralyzable uh system right”transcript, stated

Spend machines to reduce runtime

Chose horizontal scaling over serial or bottlenecked execution.

Rationale: The pipeline is designed so runtime is mainly a cost/resource allocation decision.

“it's only question of cost of how many machines you want uh and how much resource you want to dedicate to it”transcript, stated

Allow model-specific resource tuning

Chose per-model resource cards and configurable Metaflow jobs over fixed resource allocation for every matching model.

Rationale: Complex neural models and simpler rule-based models need different resource budgets.

“somebody wants to try a complex neural net model for matching um sure but then the resource had to be tuned based on that and we don't want to waste resources”transcript, stated

Gotchas (what bit them)

Candidate generation still creates enormous matching workloads.

Root cause: Millions of entities can produce hundreds of millions or billions of candidate pairs even after blocking.

Resolution: Partition and parallelize the matching classification pipeline.

“you'll still end up with hundreds of millions or billions of paays in our case so that actually means to solve this problem we need a system”transcript, stated

Parsing large metadata blobs became a bottleneck.

Root cause: JSON blob parsing took significant time at billions-of-pairs scale.

Resolution: Use the fast data layer with Apache Arrow to parse into dataframes.

“dealing with large metadata with J the H Json blob takes lot of time and just parsing and that itself becomes a botton lack if you're dealing with billions of payers”transcript, stated

Data skew can stall a large parallel run.

Root cause: A source data issue or entity with huge filmography can make one task much slower than others.

Resolution: Use Metaflow UI to identify stuck tasks, skew, and data problems quickly.

“maybe it has a like in the case of movie example a movie with like thousand filmography or something like that and then that computation becomes very slow”transcript, stated

Experiment branches can accidentally interfere with production models.

Root cause: New engineers or experiments may deploy models from the same training pipeline.

Resolution: Use Metaflow project versioning, branching, and namespaces to isolate experiments from production.

“production model never kind of intersect into the uh test bran and uh we are sure that somebody can trade uh independently out of the uh you know the development branches”transcript, stated

Evaluation

The system uses daily model checks, operational debugging through Metaflow, resource monitoring, and branch isolation to keep matching and inference pipelines reliable.

Metrics: Model deterioration check · Data skew and stuck-task detection · Parsing speedup · Resource usage per model/entity · Production versus development branch isolation

“you have a small step which kind of makes sure the model is not deteriorated”transcript, stated

“you can use use the resource cards to see the resource usage and then kind of tune it based on each model for each entity”transcript, stated

Outcomes

Apache Arrow fast-data parsing produced a 10x speedup.
The pipeline can run billions of pairs daily through parallel Metaflow execution.
Metaflow UI and branch/versioning features improved debugging, resource tuning, and production isolation.
Netflix is exploring LLM-based graph relationship extraction inside the same workflow style.

The knowledge graph serves multiple teams and platforms, with entity matching over millions of entities and billions of candidate pairs.

“just doing this kind of enabled us to kind of run this extremely fast”transcript, stated

“this is one of the place where currently we're exploring and having some success um uh on on on uh incorporating into the New Gen pipeline”transcript, stated