Conversational AI Shopping Assistant with Multi-Agent Architecture and Real-Time Grounding

doordash · E-commerce · 2026 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Local-commerce shopping assistant

DoorDash needed Ask DoorDash to handle food and grocery discovery through multi-turn conversation while staying grounded in local-commerce data that changes minute by minute. Menus, prices, hours, ETAs, delivery radii, inventory, personalization, and cart state cannot be trusted to static model weights.

Constraints:

Local commerce data changes minute to minute and depends on location.
Most sessions are multi-turn, so state and artifact continuity matter.
Consumer-visible claims must come from system-of-record tool calls.
Assistant latency must remain tolerable in a shopping flow.
Restaurant and grocery domains have separate teams and release schedules.

Architecture

DoorDash built Ask DoorDash on a shared assistant platform with clients and Gateway, an Orchestrator plus restaurant and grocery domain agents, Managed Agent Services for artifacts/session/memory, a shared MCP tool surface over DoorDash backend services, consumer memory reconciled with live catalog data, widgets for grounded interactions, and LLM-as-judge evaluation over transcripts, tool calls, outputs, and grounding context.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  consumer-client["Consumer client"] --> gateway["Gateway"]
  gateway --> orchestrator-agent["Orchestrator agent"]
  orchestrator-agent --> restaurant-discovery-agent["Restaurant discovery agent"]
  orchestrator-agent --> grocery-shopping-agent["Grocery shopping agent"]
  restaurant-discovery-agent --> mcp-tool-surface["MCP tool surface"]
  grocery-shopping-agent --> mcp-tool-surface
  mcp-tool-surface --> doordash-backend-services["DoorDash backend services"]
  restaurant-discovery-agent --> managed-agent-services["Managed Agent Services"]
  grocery-shopping-agent --> managed-agent-services
  managed-agent-services --> memory-platform["Memory platform"]
  managed-agent-services --> artifact-store["Artifact store"]
  managed-agent-services --> session-store["Session store"]
  artifact-store --> gateway
  evaluation-harness["Evaluation harness"] --> restaurant-discovery-agent
  evaluation-harness --> grocery-shopping-agent
  evaluation-harness --> doordash-backend-services

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Consumer client`	iOS client that sends text/image/voice input and receives SSE text and widget payloads.	`Gateway`
`Gateway`	Authenticates, preserves session continuity, translates UI streams into A2A streaming gRPC, and applies direct artifact edits.	`Consumer client`, `Orchestrator agent`, `Managed Agent Services`
`Orchestrator agent`	Routes turns to restaurant or grocery agents and pins follow-ups to the previous domain agent.	`Gateway`, `Restaurant discovery agent`, `Grocery shopping agent`
`Restaurant discovery agent`	Handles restaurant discovery and recommendation turns.	`Orchestrator agent`, `MCP tool surface`, `Managed Agent Services`
`Grocery shopping agent`	Builds and revises grocery lists and carts against live store inventory and consumer preferences.	`Orchestrator agent`, `MCP tool surface`, `Managed Agent Services`
`Managed Agent Services`	Shared platform for artifacts, session state, and consumer memory across DoorDash agent teams.	`Gateway`, `Restaurant discovery agent`, `Grocery shopping agent`, `Memory platform`, `Artifact store`, `Session store`
`Memory platform`	Stores long-term, in-session, and agentic consumer facts and reconciles preferences with live data.	`Managed Agent Services`, `MCP tool surface`
`Artifact store`	Stores widgets such as shopping lists and store cards as versioned objects with stable IDs.	`Managed Agent Services`, `Gateway`
`Session store`	Stores conversation turns, tool calls, tool results, and agent state per agent.	`Managed Agent Services`
`MCP tool surface`	Typed tools exposing business logic and grounding data over DoorDash backend services.	`Restaurant discovery agent`, `Grocery shopping agent`, `DoorDash backend services`, `Memory platform`
`DoorDash backend services`	Search, catalog, order history, cart, deals, and merchant pipelines used as systems of record.	`MCP tool surface`
`Evaluation harness`	Builds transcripts with tool calls and grounding context, applies LLM-as-judge rubrics, clusters failures, and validates fixes.	`Restaurant discovery agent`, `Grocery shopping agent`, `DoorDash backend services`

Data flow

A consumer asks for a restaurant or grocery task. The Gateway streams the request to the Orchestrator, which pins or routes the turn to a domain agent. The agent retrieves memory, calls MCP tools over search/catalog/cart/deals/order-history services, builds or updates a versioned artifact such as a shopping list, streams text and widget payloads back to the client, and waits for explicit consumer confirmation before committing.

“An Orchestrator agent decides which domain agent (restaurant discovery or grocery shopping) handles each turn.”article, stated

“Agents call tools through a shared MCP layer. The same MCP server backs both the Assistant and our external integrations, with each surface configured to see the tools it needs.”article, stated

Why this architecture

DoorDash separates domain agents while centralizing shared platform primitives. Real-time catalog and memory reconciliation happen through MCP tools on every turn, preventing the model from inventing local commerce facts while letting restaurant and grocery teams ship independently.

“DoorDash Assistant is a layer on top of the existing DoorDash platform. Four parts, shown in Figure 2:”article, stated

“The goal is for every consumer-visible claim to come from a tool call against the system of record on the turn it's made.”article, stated

Models used: Google Agent Development Kit agents · Orchestrator agent · Restaurant discovery agent · Grocery shopping agent · LLM-as-judge evaluators · Background failure-clustering agents

Scale

Around seven in ten messages are discovery traffic.
“Around seven in ten messages are some form of discovery: looking for a restaurant”article, stated
The example grocery turn uses 6-8 LLM calls and a handful of live-catalog tool calls.
“6-8 LLM calls and a handful of tool calls against the live catalog”article, stated
The example grocery turn carries low hundreds of thousands of input tokens.
“Low hundreds of thousands of input tokens in the model context once the candidate set is in”article, stated
The example grocery turn takes 20-30 seconds end to end.
“20-30 seconds end to end”article, stated
Weekly pull-request volume doubled early and roughly tripled by final pre-launch weeks with coding assistants in the loop.
“weekly pull-request volume doubled in the early sprints and roughly tripled by the final pre-launch weeks.”article, stated

Alternatives considered (roads not taken)

Rely on model weights for local commerce facts

Rejected because: Menus, prices, hours, ETAs, radii, and inventory are live, local, and not in model weights.

“Menus, prices, hours, ETAs, delivery radii, and inventory change minute to minute and depend on where the consumer is. None of it lives in a model's weights”article, stated

Commit assistant-built carts automatically

Rejected because: The UX keeps the consumer in control and requires confirmation before cart changes land.

“When the Assistant builds a shopping list, the consumer reviews and confirms before it lands in the cart.”article, stated

Build separate infrastructure for each domain agent

Rejected because: Shared services let teams reuse artifacts, session, memory, MCP, orchestration, cold-start handling, and evaluation.

“They share the platform: widgets, Managed Agent Services, MCP, the Orchestrator, cold-start handling, and the evaluation harness.”article, stated

Patterns

`multi-agent-orchestration` (orchestration)

An orchestrator routes turns between separate restaurant and grocery domain agents with agent pinning for follow-ups.

“Agent pinning keeps follow-up turns like "add to cart" routed to whichever agent answered the previous turn, until the consumer's intent shifts.”article, stated

`tool-use-orchestration` (orchestration)

Agents call typed MCP tools for search, catalog, order history, cart, deals, merchant data, and memory.

“Business logic (cart manipulation, store lookup, deal application) lives in the tools, separate from the prompts that call them.”article, stated

`agent-memory` (memory)

The assistant uses long-term, in-session, and agentic memory to personalize shopping while reconciling with live data.

“The Intelligence pillar adds the memory layer that lets each session pick up the consumer's context.”article, stated

`llm-as-judge` (evaluation)

DoorDash judges full session transcripts against rubrics calibrated with human-reviewed labels.

“A suite of LLM-as-judge, calibrated against human-reviewed labels, evaluates the transcript against the relevant rubric.”article, stated

`human-in-the-loop-gate` (workflow)

Consumers choose when to delegate, edit widgets manually, and confirm before assistant work lands in the cart.

“The consumer stays in the loop, choosing when to delegate to the Assistant and when to operate manually.”article, stated

Tradeoffs

Use separate domain agents

Chose restaurant and grocery agents on a shared platform over one monolithic shopping agent.

Rationale: Domains differ in answer shape, edit behavior, ownership, and deploy schedules.

“Grocery and restaurant discovery are different products in practice.”article, stated

Let widgets handle direct edits without LLM round trips

Chose artifact mutations through the Gateway over sending every item edit back through the model.

Rationale: Direct widget edits are faster and keep cart state grounded in live catalog data.

“These edits run against the artifact directly through the Gateway and never enter an LLM round trip.”article, stated

Keep architecture and model choices reversible

Chose simulation harness plus dynamic rollout controls over irreversible prompt/model/platform bets.

Rationale: DoorDash expects to reverse many decisions as data arrives.

“Architecture and model choices are reversible by design. Every meaningful change runs through the simulation harness before shipping”article, stated

Gotchas (what bit them)

Grounding failures are the largest potential production failure.

Root cause: The agent may claim stores are open, prices are current, or cart actions happened unless claims route through live systems.

Resolution: Route every consumer-visible claim through tool calls against systems of record.

“The largest potential production-failure category is grounding. Stores recommended as open when they're closed, prices that don't match the catalog, items the agent claims to have added that aren't in the cart.”article, stated

Memory can conflict with current inventory or price reality.

Root cause: Stored preferences may not be available or affordable at the chosen store right now.

Resolution: Reconcile memory with live search and cart tools on the turn.

“When memory conflicts with live data, the agent adjusts its plan accordingly.”article, stated

Minor changes can alter an entire multi-turn conversation.

Root cause: Agent interactions are stateful and tool actions shape later context.

Resolution: Evaluate complete transcripts with tool calls, outputs, and grounding context.

“A change that appears minor in isolation can alter how an entire conversation unfolds, making it difficult to reason about agent quality through pre-defined input-output mappings alone.”article, stated

Evaluation

DoorDash evaluates complete agent sessions with transcript construction, LLM-as-judge rubrics calibrated to human labels, guardrail evals, capability evals, online and offline alignment, failure clustering, synthetic simulation, and production-trace gating.

Metrics: Session integrity · Safety · Result quality · Execution quality · Rubric pass rate · Grounding failures · Prompt/code change regression

“the system constructs a transcript for each session, capturing user inputs, agent responses, tool calls, tool outputs, and grounding context.”article, stated

“a coding agent reads it, proposes a fix, and validates the fix against a shadow Assistant paired with the simulator. Changes are gated on the rubric pass rate staying clean.”article, stated

Outcomes

Ask DoorDash was rolling out in select U.S. iOS areas for restaurant search and grocery shopping.
Around 70% of messages were discovery-related.
Most users who send a first message continue iterating in the same session.
The team identifies grounding as the largest potential failure category and routes claims through tools.
The development team increased weekly PR volume while using AI coding assistants.

The example grocery task uses 6-8 LLM calls, low hundreds of thousands of input tokens, and 20-30 seconds end to end.

“The Assistant that powers Ask DoorDash is now rolling out to select areas in the U.S. on iOS, starting with restaurant search and grocery shopping.”article, stated

“Most consumers who send a first message keep iterating in the same session”article, stated