Context-Aware Item Recommendations Using Hybrid LLM and Embedding-Based Retrieval

doordash · E-commerce · 2025 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: consumer shopping recommendations

DoorDash needed in-app item recommendations that preserved a user's specific food intent and preferences instead of falling back to generic store menus or duplicate suggestions.

Constraints:

Recommendations had to work in a real-time consumer-facing experience.
The system needed to preserve nuanced preferences such as dietary restrictions and taste profiles.
LLM-only recommendation was too slow and expensive for production serving.
Embedding-only retrieval was fast but not personalized enough.

Architecture

DoorDash built a hybrid context-shopping engine: user profile and session context are compressed into retrieval signals, query expansion injects preference terms into embedding search, FAISS retrieves candidates quickly, and an LLM reranks/filters candidates and generates contextual carousel copy.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  shopping-query-and-session-context["shopping query and session context"] --> profile-context-compressor["profile/context compressor"]
  shopping-query-and-session-context --> query-expansion-layer["query expansion layer"]
  profile-context-compressor --> query-expansion-layer
  profile-context-compressor --> embedding-candidate-retriever["embedding candidate retriever"]
  query-expansion-layer --> embedding-candidate-retriever
  embedding-candidate-retriever --> llm-reranker-and-filter["LLM reranker and filter"]
  llm-reranker-and-filter --> carousel-generation-layer["carousel generation layer"]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`shopping query and session context`	Carries the user's current intent, such as a specific food craving, plus session-level preference signals.	`profile/context compressor`, `query expansion layer`
`profile/context compressor`	Compresses dietary preferences, historical interactions, and contextual session data into real-time retrieval context.	`shopping query and session context`, `query expansion layer`, `embedding candidate retriever`
`query expansion layer`	Injects user-specific preference terms into embedding vectors to improve personalized retrieval.	`shopping query and session context`, `profile/context compressor`, `embedding candidate retriever`
`embedding candidate retriever`	Uses FAISS and embedding similarity to retrieve candidate items quickly with business-rule filters.	`query expansion layer`, `LLM reranker and filter`
`LLM reranker and filter`	Evaluates semantic relevance and contextual alignment of retrieved candidates.	`embedding candidate retriever`, `carousel generation layer`
`carousel generation layer`	Generates contextual carousel titles and item descriptions for the in-store experience.	`LLM reranker and filter`

Data flow

A user issues a shopping query. The system fetches compressed profile/session context and embeddings, expands the query with user-specific preference terms, retrieves candidates through FAISS-based embedding search, then sends the candidate set to an LLM for reranking, filtering, carousel title generation, and explanatory item descriptions.

“After passing down the context signals, we use EBR for retrieval and rely on an LLM call for ranking. The results are then displayed in item carousels.”article, stated

“During this stage, we conducted real-time compression of user profiles, including dietary preferences, historical interactions, and contextual session data.”article, stated

Why this architecture

The hybrid design was chosen because EBR alone was fast but weak on personalization, while an LLM-only pipeline offered rich contextual output but exceeded the serving latency budget. Combining the two let DoorDash keep fast candidate retrieval and reserve LLM work for higher-value reranking and explanation.

“The last method — hybrid EBR+LLM pipeline — proved superior to the other options because it successfully addresses the shortcomings of the other single-method approaches.”article, stated

“Balances quick retrieval with LLM’s sophisticated context and relevance evaluation. Significantly enhances personalization, precision, and contextual depth while maintaining acceptable latency.”article, stated

Models used: large language model for reranking, filtering, titles, and descriptions · embedding model/vector representations · FAISS similarity retrieval

Scale

LLM-only recommendation was measured at more than 20 seconds of latency.
“Initial tests with single-method approaches revealed impractical latency of more than 20 seconds using LLM alone.”article, stated
The optimized hybrid solution reached about six seconds end-to-end.
“Our optimized hybrid solution achieved end-to-end latency of approximately six seconds”article, stated
Store pages stayed under two seconds, which DoorDash considered compliant with its performance guidelines.
“with store page load times consistently under two seconds, which is fully compliant with DoorDash's strict performance guidelines.”article, stated

Alternatives considered (roads not taken)

deep neural network retrieval

Rejected because: It would require extensive data and large resource investment to train and maintain at scale.

“Not practical because of extensive data requirements and significant resource investments needed to train and maintain such models at scale.”article, stated

LLM-only pipeline

Rejected because: It gave richer contextual outputs but had unacceptable latency, prompt size, and processing overhead for real-time serving.

“Severe latency issues of more than 20 seconds, making it unsuitable for real-time consumer-facing environments.”article, stated

embedding-based retrieval alone

Rejected because: It was near-instant but insufficiently personalized and could return duplicates or irrelevant items.

“Offers limited personalization, often returning duplicates or irrelevant items by failing to consider nuanced user preferences”article, stated

Patterns

`retrieval-ranking` (recommendation)

DoorDash retrieves candidates through embedding search and then reranks them with an LLM for contextual relevance.

“Combines rapid nearest-neighbor search using FAISS (Facebook AI Similarity Search) candidate retrieval with refined LLM-based reranking and filtering.”article, stated

`query-rewriting` (retrieval)

The system expands queries with user-specific preference terms before embedding retrieval.

“injecting user-specific terms such as dietary restrictions and taste preferences — for example, "vegan," "savory," or "spicy" — into embedding vectors”article, stated

`context-compaction` (context-management)

The shopping engine compresses profile and session context in real time before using it for retrieval.

“we conducted real-time compression of user profiles, including dietary preferences, historical interactions, and contextual session data.”article, stated

`latency-optimization` (serving)

DoorDash uses fast embedding retrieval before selective LLM work to stay within consumer-facing latency requirements.

“DoorDash’s strict latency constraints demand careful optimization. The hybrid approach requires parallelizing retrieval and LLM operations to maintain low latency.”article, stated

`cost-aware-routing` (serving)

The architecture limits expensive LLM usage by combining EBR with selective LLM calls.

“Through strategic use of EBR combined with selective LLM calls, we significantly minimized compute resources and associated costs”article, stated

Tradeoffs

Use hybrid EBR plus LLM rather than a single-method recommender.

Chose FAISS candidate retrieval followed by LLM reranking/filtering over DNN-only, LLM-only, or EBR-only recommendation.

Rationale: The hybrid approach kept retrieval fast while adding contextual relevance that the fast-only method lacked.

“The last method — hybrid EBR+LLM pipeline — proved superior to the other options because it successfully addresses the shortcomings of the other single-method approaches.”article, stated

Spend LLM budget on reranking and explanations instead of full retrieval.

Chose selective LLM calls after embedding retrieval over an LLM-only call for context understanding, candidate generation, and ranking.

Rationale: Selective LLM usage preserved contextual quality while reducing latency and compute cost.

“Through strategic use of EBR combined with selective LLM calls, we significantly minimized compute resources and associated costs”article, stated

Inject preference terms into retrieval rather than ranking generic candidates only after retrieval.

Chose query expansion with dietary and taste preference terms over plain semantic retrieval over the original query.

Rationale: Embedding user preferences directly into retrieval improved the candidate pool before reranking.

“implemented intelligent query expansion techniques to refine results by embedding user preferences directly into the retrieval process.”article, stated

Gotchas (what bit them)

LLM-only recommendation was too slow for the store-page experience.

Root cause: Putting context understanding, candidate generation, and ranking into one LLM call created large prompts and long processing overhead.

Resolution: Use embeddings/FAISS for candidate retrieval and reserve the LLM for reranking/filtering and copy generation.

“Relies solely on an LLM for context understanding, candidate generation, and ranking in a single call.”article, stated

“Practical deployment also constrained by extremely large prompt sizes and processing overhead.”article, stated

Embedding retrieval alone could not preserve nuanced preferences.

Root cause: Fast vector search did not capture user-specific context like dietary restrictions or taste profiles by itself.

Resolution: Add query expansion and LLM reranking to inject and evaluate preference context.

“EBR converts search queries and items into numerical vectors — embeddings — and, with simple business-rule filters, quickly fetches relevant items from that vector database.”article, stated

“failing to consider nuanced user preferences, such as dietary restrictions or taste profiles.”article, stated

The hybrid approach introduced orchestration, latency, cost, and context-balancing challenges.

Root cause: Multiple retrieval/model stages had to coordinate in a real-time product path.

Resolution: Parallelize retrieval and LLM operations, use selective LLM calls, and tune query expansion/embeddings for stable and real-time context.

“adopting this hybrid method introduced specific implementation challenges”article, stated

“It was challenging to ensure the system appropriately balanced real-time context signals and stable user preferences”article, stated

Evaluation

DoorDash compared retrieval methodologies, ran experimentation on latency and user experience, and evaluated hybrid retrieval against single-method approaches.

Metrics: LLM-only latency above 20 seconds · hybrid end-to-end latency of approximately six seconds · store page load times under two seconds · preliminary user satisfaction and perceived relevance feedback

“Our extensive experimentation validated the hybrid retrieval engine’s performance and impact.”article, stated

“Preliminary feedback indicated notable improvements in user satisfaction, with increased perceived relevance and ease of navigation”article, stated

Outcomes

The system transformed static menus into dynamic, personalized item carousels.
The hybrid solution met DoorDash's strict performance guidelines.
DoorDash reports improved engagement and reduced decision fatigue.

The article frames the work as a real-time personalization system for DoorDash's consumer platform, with end-to-end latency around six seconds and store-page loads under two seconds.

“User experience demonstrations clearly illustrated the transformation from static, generic menus to dynamic, personalized item carousels.”article, stated

“this innovative solution significantly improved user engagement and reduced decision fatigue.”article, stated