Bridging Behavioral Silos in Multi-Vertical Recommendations with LLMs

doordash · E-commerce · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Multi-vertical marketplace recommendations

DoorDash needed to recommend across restaurants, grocery, retail, and convenience when customers had dense history in some verticals but sparse or cold-start behavior in others.

Constraints:

restaurants produce dense reorder data while retail/grocery catalogs are much larger and sparser
cross-vertical signals must map into a shared taxonomy
LLM-generated affinities must be stable enough for production ranking
inference cost has to remain practical
online gains should match offline improvements

Architecture

DoorDash built an LLM-powered hierarchical RAG pipeline that converts restaurant orders and search queries into structured category-affinity features over a four-level taxonomy. Those LLM-enhanced features are embedded and mean-pooled into fixed-size representations, then fed into production multi-task ranking models alongside existing recommendation signals.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  behavior-logs[Behavior logs] --> hierarchical-rag-pipeline[Hierarchical RAG pipeline]
  product-taxonomy[Product taxonomy] --> hierarchical-rag-pipeline
  hierarchical-rag-pipeline --> affinity-features[Affinity features]
  affinity-features --> feature-embedding-layer[Feature embedding layer]
  feature-embedding-layer --> multi-task-ranker[Multi-task ranker]
  multi-task-ranker --> online-evaluation[Online evaluation]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Behavior logs`	Orders, searches, clickstreams, and profile attributes from DoorDash users.	`Hierarchical RAG pipeline`
`Hierarchical RAG pipeline`	LLM pipeline that infers taxonomy-aligned category affinities.	`Behavior logs`, `Product taxonomy`, `Affinity features`
`Product taxonomy`	Four-level category hierarchy used to constrain LLM outputs.	`Hierarchical RAG pipeline`
`Affinity features`	Structured category-interest signals generated from behavioral history.	`Feature embedding layer`
`Feature embedding layer`	Maps variable-length taxonomy IDs into fixed-size representations.	`Affinity features`, `Multi-task ranker`
`Multi-task ranker`	Production ranking model consuming LLM-derived and baseline features.	`Feature embedding layer`, `Online evaluation`
`Online evaluation`	Production measurement of AUC-ROC and MRR gains against baseline.	`Multi-task ranker`

Data flow

Historical orders, searches, and profile attributes are formatted with DoorDash's product taxonomy. The H-RAG pipeline infers L1-L4 category affinities, emphasizes L2/L3 categories for ranking, drops low-confidence outputs, maps category IDs through embeddings with mean pooling, and sends fixed-size features into the multi-task ranking model.

3 supporting quotes

“translate unstructured user behavior, like restaurant orders and search queries, into a structured, four level product taxonomy”article, stated

“For our multi task learning (MTL) ranking, we focus mainly on L2 and L3, as L1 is too generic to provide meaningful signals and L4 is often too sparse”article, implied

“Variable length categorical fields (e.g., lists of taxonomy IDs in u LLM are handled by mapping each ID through a shared embedding table and applying mean pooling”article, stated

Why this architecture

LLMs act as semantic feature generators between verticals: they abstract dense restaurant behavior into taxonomy-aligned signals that ranking systems can use in sparser verticals. Prompt design, caching, small models, and confidence filtering keep the approach practical for production ranking.

3 supporting quotes

“translating noisy activity into high fidelity, generalizable user representations that our retrieval and ranking systems can understand and act on.”article, stated

“The approach improved relevance, especially for cold start scenarios, and shows consistent offline and online gains, while keeping inference costs practical via prompt design, caching, and small language models.”article, stated

“provide an explicit taxonomy, require per category confidence scores (and drop low confidence ones), use chronological histories, clear rules, and low temperature”article, implied

Models used: LLM · GPT-4o-mini · multi-task ranking model · embedding table

Scale

DoorDash ran the H-RAG pipeline on a 20 percent sample of the last three months of consumer data.
“On a 20% sample of the last three months of consumer data, we run a Hierarchical Retrieval Augmented Generation (H-RAG) pipeline”article, stated
Grocery, retail, and convenience catalogs contain tens to hundreds of thousands of SKUs.
“With tens to hundreds of thousands of SKUs, user behavior spreads thinly across an enormous catalog. The same customer may be well understood in restaurants yet effectively cold-start elsewhere.”article, stated
Offline results improved 4.4 percent AUC-ROC and 4.8 percent MRR over baseline.
“the proposed model achieved a 4.4% relative improvement in AUC-ROC as shown in Figure 2 and a 4.8% relative improvement in MRR”article, stated
Online production results improved +4.3 percent AUC-ROC and +3.2 percent MRR.
“The online results showed an improvement of +4.3% in AUC-ROC, and +3.2% MRR vs. the baseline (Figure 4), closely matched the offline analysis.”article, stated

Alternatives considered (roads not taken)

Use only within-vertical historical behavior

Rejected because: Sparse non-restaurant histories create cold-start behavior that restaurant-derived semantic affinities can bridge.

“The same customer may be well understood in restaurants yet effectively cold-start elsewhere. This asymmetry creates a modeling gap.”article, stated

Use all taxonomy levels equally

Rejected because: L1 is too generic and L4 too sparse, so the ranking model focuses on L2 and L3.

“For our multi task learning (MTL) ranking, we focus mainly on L2 and L3, as L1 is too generic to provide meaningful signals and L4 is often too sparse in real world data.”article, stated

Patterns

`rag-retrieval` (retrieval)

DoorDash uses an H-RAG pipeline to infer category affinities from behavior and taxonomy context.

“Hierarchical Retrieval Augmented Generation (H-RAG) pipeline that infers which product categories each user is most likely interested in”article, stated

`confidence-thresholding` (quality-control)

Low-confidence category predictions are dropped to stabilize LLM features.

“require per category confidence scores (and drop low confidence ones)”article, stated

`semantic-caching` (cost-control)

Static prompt parts such as instructions and taxonomy are cached to control inference cost.

“We cached the static part of the prompt (instructions + taxonomy).”article, stated

`retrieval-ranking` (ranking)

LLM-generated affinity features feed a production multi-task ranking model.

“plug them into your existing ranking models, especially to transfer signals from data rich to data sparse verticals.”article, stated

Tradeoffs

Taxonomy level used by ranker

Chose L2 and L3 category affinities over L1 and L4 as primary ranking features.

Rationale: L1 is too broad to help and L4 is too sparse in real data.

“we focus mainly on L2 and L3, as L1 is too generic to provide meaningful signals and L4 is often too sparse”article, stated

Inference cost control

Chose prompt design, caching, and small models over unoptimized large-model inference.

Rationale: DoorDash needed LLM features that were practical enough for production ranking workflows.

“keeping inference costs practical via prompt design, caching, and small language models.”article, stated

Gotchas (what bit them)

Cross-vertical customer understanding is sparse outside core restaurant behavior.

Root cause: Retail/grocery catalogs are enormous and user activity spreads thinly across them.

Resolution: Translate dense restaurant orders and search behavior into taxonomy-aligned cross-domain affinity features.

“restaurants have compact menus and high reorder frequency, which produce dense, clean behavioral data. Categories like grocery, retail, and convenience are a different story.”article, stated

Unconstrained LLM features can hallucinate or become unstable.

Root cause: The model needs explicit categories, rules, and confidence handling.

Resolution: Constrain outputs with taxonomy, chronological histories, confidence filtering, clear rules, and low temperature.

“provide an explicit taxonomy, require per category confidence scores (and drop low confidence ones), use chronological histories”article, stated

Evaluation

DoorDash evaluates the ranking model offline across overall, cold-start, and power-user cohorts, then validates the gains online against baseline AUC-ROC and MRR.

Metrics: AUC-ROC · MRR · cold-start cohort lift · power-consumer cohort lift · online AUC-ROC · online MRR

“We evaluated the performance of our model on the full user base and two key cohorts: cold start consumers”article, implied

Outcomes

Overall offline ranking improved 4.4 percent AUC-ROC and 4.8 percent MRR.
Online production ranking improved +4.3 percent AUC-ROC and +3.2 percent MRR.
Cold-start and power-consumer cohorts also improved.

The H-RAG pipeline was run on a 20 percent sample of three months of consumer data and targeted catalogs with tens to hundreds of thousands of SKUs.

“This confirms that LLM-generated taxonomic features deliver consistent, real-world improvements in personalization quality.”article, stated