Hyper-Personalized Merchandising Through Hybrid LLM and Deep Learning Systems

doordash · E-commerce · 2026 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: hyper-personalized local commerce merchandising and recommendation

DoorDash needed to personalize discovery across a large local-commerce catalog for millions of users whose needs shift by moment, event, location, and session intent, while keeping serving latency and cost low enough for consumer-facing feeds.

Constraints:

• DoorDash spans restaurants, grocery, convenience, alcohol, flowers, pets, electronics, and other local commerce categories
• classic recommender systems learn from engagement and product metadata but struggle with moment-aware intent
• new merchants and items may have no engagement history
• LLM calls are too costly and slow for all real-time serving paths
• attribute extraction can be safety-critical, such as allergen extraction
• hyper-personalized pages create too many variants for exhaustive human review
• consumer profiles must update with recency while avoiding full-history regeneration every time

Architecture

DoorDash built a hybrid personalization architecture that uses LLMs where language, world knowledge, and generalization matter, and keeps traditional deep-learning rankers for low-latency last-mile optimization. Product understanding uses LLMs, fine-tuning, RAG over DoorDash ontology/taxonomy/catalog, and agentic external information gathering to extract product attributes. Consumer understanding is represented as natural-language profile narratives organized into memory blocks such as dietary habits, household information, category preferences, item/brand preferences, and taxonomy preferences, with structured facets extracted for efficient targeting. LLMs generate weekly content blueprints for evergreen and event-driven merchandising; online serving blends those offline artifacts with real-time session embeddings, performs many embedding/lexical/semantic retrievals, and then uses two-tower/MTML models for final ranking. Evaluation combines quantitative metrics, LLM-as-judge, human textual feedback, dogfooding, and GEPA optimization inside DSPy to tune prompts, templates, retrieval logic, profile scope, and ranking objectives.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  Merchant_catalog_inputs["Merchant catalog inputs"] --> Product_understanding_layer["Product understanding layer"]
  Ontology_taxonomy_catalog_RAG["Ontology taxonomy catalog RAG"] --> Product_understanding_layer
  Agentic_enrichment_process["Agentic enrichment process"] --> Product_understanding_layer
  Product_understanding_layer --> Product_graph["Product graph"]
  Consumer_behavior_history["Consumer behavior history"] --> Consumer_profile_generator["Consumer profile generator"]
  Consumer_profile_generator --> Profile_memory_blocks["Profile memory blocks"]
  Product_graph --> Content_blueprint_generator["Content blueprint generator"]
  Profile_memory_blocks --> Content_blueprint_generator
  Content_blueprint_generator --> Feed_service["Feed service"]
  Personalized_homepage["Personalized homepage"] --> Consumer_behavior_history
  Feed_service --> DoorDash_Brain["DoorDash Brain"]
  Profile_memory_blocks --> DoorDash_Brain
  DoorDash_Brain --> Retrieval_layer["Retrieval layer"]
  Content_blueprint_generator --> Item_embedding_index["Item embedding index"]
  Item_embedding_index --> Retrieval_layer
  Retrieval_layer --> Final_ranking_models["Final ranking models"]
  Final_ranking_models --> Personalized_homepage
  Personalized_homepage --> Evaluation_framework["Evaluation framework"]
  Evaluation_framework --> GEPA_optimizer["GEPA optimizer"]
  GEPA_optimizer --> Content_blueprint_generator
  GEPA_optimizer --> Consumer_profile_generator
  GEPA_optimizer --> Final_ranking_models
  Profile_memory_blocks --> Agent_context_grounding["Agent context grounding"]

Components

Data flow

Merchant/product data flows into an LLM-powered product-understanding layer that extracts attributes, sometimes using external search and merchant-site scraping for sparse small-merchant spreadsheets. User behavior over roughly the past year initializes natural-language profile memory blocks, which are updated on different cadences through delta updates and recency weighting. LLMs create content blueprints and search terms offline for each consumer/use case. At request time, the feed service sends events to DoorDash Brain, retrieves pre-generated carousels, blends in real-time intent through embeddings rather than synchronous LLM calls, executes 20-30 retrievals over item embeddings plus lexical/semantic search, and applies deep-learning rankers before rendering the feed. Evaluation feedback flows back through GEPA to mutate prompts, search-term generation, profile generation, and ranking objectives.

Why this architecture

DoorDash keeps LLMs and deep learning in distinct roles because each solves a different part of personalization. LLMs understand messy behavior, world knowledge, product semantics, natural-language narratives, content ideation, and agentic enrichment. Deep-learning models remain better for concrete ranking objectives, CTR/CVR optimization, and low-latency serving. The architecture therefore does expensive LLM work offline or asynchronously, approximates real-time intent through embeddings, and uses traditional rankers for final serving.

Models used: LLMs · fine-tuned product-understanding models · two-tower embedding models · multi-task deep learning rankers · LLM-as-judge models · GEPA optimizer within DSPy

Scale

• DoorDash serves millions of users, thousands of merchants, and a massive catalog.
  “now we have millions of users, thousands of merchants, a massive catalog. We need to understand every user. We need to understand every item we carry.”article, stated
• The platform spans many local-commerce categories beyond restaurants.
  “Everything that's sold around you, be it restaurants or grocery, convenience, alcohol, flowers, pets, you name it, we can deliver it”article, stated
• One product extraction task moved from 28 days to 2 days.
  “There's a certain task that used to take 28 days, we do that exact same thing in 2 days today.”article, stated
• Initial profiles use roughly one year of user behavior.
  “there is an initialization part of it where you want to essentially dump everything that the consumer has done over the last year or so.”article, stated
• Online serving can require 20-30 embedding retrievals.
  “It becomes 20 to 30 different embedding retrievals that you would essentially have to do”article, stated
• The evaluation surface can include 50 brand-new carousels per consumer.
  “Right now, for every single consumer, there's 50 different carousels which are brand new and which are going to say different things.”article, stated

Alternatives considered (roads not taken)

Classic collaborative filtering and deep learning only

Rejected because: Classic systems learn from engagement and metadata but do not react quickly enough to moment-specific needs or use world knowledge for new catalog items.

Static merchandising and editorial carousels

Rejected because: Static merchandising targets an average consumer and cannot create unique dynamic paths for each person.

LLMs for everything in real time

Rejected because: The speakers say that end-to-end real-time LLM use becomes costly and introduces latency concerns.

Vanilla LLM extraction for product understanding

Rejected because: Vanilla LLMs delivered only part of the desired accuracy, and safety-critical attributes such as allergens required higher precision.

Per-user prompt modification for feedback optimization

Rejected because: The Q&A says per-user prompt modification would be too expensive; context selection does the personalization instead.

Patterns

agent-memory (context)

Consumer profiles are stored as natural-language memory blocks and reused across homepage, search, notifications, ranking, and agents.

rag-retrieval (retrieval)

RAG grounds product understanding in DoorDash ontology/taxonomy/catalog and later pulls profiles into agent contexts.

tool-use-orchestration (agent-architecture)

Agentic product enrichment searches Google, scrapes merchant websites, aggregates context, and lets an LLM reason over the combined information.

fine-tuning-adaptation (model-adaptation)

Product-understanding models needed fine-tuning to reach production-grade precision beyond vanilla LLM extraction.

retrieval-ranking (ranking)

Generated search terms and profile facets feed embedding, lexical, and semantic retrieval before two-tower/MTML rankers perform last-mile ranking.

cost-aware-routing (operations)

The system keeps expensive LLM narrative generation offline/asynchronous and uses embeddings for low-latency real-time intent blending.

llm-as-judge (evaluation)

DoorDash uses LLM-as-judge alongside quantitative metrics and human feedback to evaluate generated personalization content.

human-in-the-loop-gate (oversight)

Product extraction and evaluation retain human input through human-in-the-loop product workflows, annotators, and employee dogfooding.

eval-driven-development (quality)

Quantitative metrics, LLM judges, human textual feedback, dogfooding, and GEPA reward functions drive system optimization.

Tradeoffs

LLMs for understanding, deep learning for last-mile ranking

Chose hybrid LLM/deep-learning division of labor over replacing traditional recommenders with LLM ranking.

Rationale: LLMs are better at narratives, world knowledge, and content ideation, while deep learning remains better for concrete ranking metrics under latency constraints.

Offline LLM generation with online embedding blending

Chose pre-generated carousels plus real-time embedding signals over fully real-time LLM personalization.

Rationale: The architecture needs to react to session intent without paying LLM latency/cost on every homepage request.

Natural-language profiles plus structured facets

Chose plain-English narratives with extracted facets over only feature vectors or embeddings.

Rationale: Natural language is expressive and explainable, while structured facets keep retrieval and targeting efficient.

Search as the carousel population primitive

Chose LLM-generated search terms feeding retrieval over bespoke generated item lists for every carousel.

Rationale: Search terms are expressive enough for personalized content while remaining tractable and compatible with retrieval/ranking systems.

Shared primitives over bespoke use-case systems

Chose profiles, product graph, and evaluation frameworks reused across applications over tightly coupling each primitive to one feature.

Rationale: Shared primitives are more cost-effective and let multiple products reuse proven context and evaluation foundations.

Start with targeted experiments before generic frameworks

Chose real experiments with ROI before broad infrastructure over large up-front infra projects.

Rationale: DoorDash recommends learning from concrete experiments and extracting frameworks later to avoid over-engineering.

Gotchas (what bit them)

Classic personalization misses short-lived moment intent.

Root cause: Collaborative filtering and deep learning rankers mostly learn from long-term engagement and product metadata.

Resolution: Layer LLM-driven profiles, moment-aware content generation, and real-time embedding signals on top of classic ranking systems.

Safety-critical product attributes cannot rely on vanilla LLM extraction.

Root cause: Wrong allergen extraction could cause serious harm, so approximate LLM performance was insufficient.

Resolution: Use fine-tuning and RAG grounded in DoorDash's ontology, taxonomy, and product catalog.

Sparse small-merchant data is hard for humans and models to interpret.

Root cause: Mom-and-pop stores send abbreviated spreadsheets that lack enough direct product context.

Resolution: Use agentic external search and merchant-site scraping, then let an LLM reason over aggregated evidence.

Personalized evaluation does not scale to manual review.

Root cause: Each consumer can see dozens of brand-new carousels with different copy and items, making exhaustive human evaluation impossible.

Resolution: Combine quantitative metrics, LLM-as-judge, human textual feedback, dogfooding, and GEPA optimization.

Recommendation systems can trap users in their existing preference loop.

Root cause: The pigeonhole effect reinforces prior recommendations and purchases.

Resolution: Introduce exploration, potentially guided by LLM world knowledge to suggest adjacent categories.

Profile updates can become too expensive if they regenerate full history.

Root cause: Sending a full year of behavior for every update is costly and ignores the different cadence of profile facets.

Resolution: Use memory-block-specific cadences, delta updates, right-sized models, and recency weighting.

Evaluation

DoorDash evaluates hyper-personalized merchandising with three axes: quantitative metrics such as online behavior and click likelihood, LLM-as-judge rubric evaluation, and open-ended human textual feedback from annotators and dogfooding. These signals become a reward function for GEPA inside DSPy, which optimizes prompts/templates, search-term generation, profile generation scope, and ranking objectives.

Metrics: CTR · CVR · quantitative online metrics · LLM-as-judge rubric ratings · human textual feedback · GEPA reward function · exploration versus exploitation objective

Outcomes

• DoorDash reduced a product extraction task from 28 days to 2 days.
• Small merchant onboarding shifted from a purely human-driven bottleneck toward AI-driven workflows with humans in the loop.
• Consumer profiles became shared primitives for homepage carousels, notifications, ranking, search, and agents.
• The system can generate dynamic Black Friday-style merchandising that reflects user profile, current deals, merchant affinity, exploration, and session intent.
• DoorDash is extending the same primitives into consumer-facing agents, catalog-building agents, and Dasher shopping assistance.

The article/transcript is a detailed practitioner talk but does not provide aggregate production lift metrics for the personalization system. The strongest measurable result is the 28-day to 2-day extraction improvement; the rest of the value is architecture and operational lessons.