LLM-Powered Voice Assistant for Restaurant Operations and Personalized Alcohol Recommendations

doordash · E-commerce · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Restaurant operations and personalized commerce discovery

DoorDash needed more flexible merchant-hours verification than DTMF robocalls allowed, and more personalized alcohol item discovery than ranked store lists and category tiles provided.

Constraints:

• Legacy robocalls could only capture simple phone-tree responses.
• The hours-verification system had to remain backward compatible with existing scheduling, event, and downstream systems.
• Natural-language conversations needed to become deterministic structured data before mutating store hours.
• Rollout needed feature-flagged fallback to legacy robocalls.
• Alcohol recommendations had sparse user-behavior data compared with food orders.
• Generated carousel titles needed to be engaging without misrepresenting the actual recommended items.

Architecture

DoorDash built a voice AI path behind the existing robocall interface. A RobocallCreatorFactory selects legacy DTMF or AI voice providers by store configuration and feature flags; Alex, the voice agent, collects natural-language hours; Vapi sends structured responses to a webhook; the webhook validates data, creates special hours through MenuDataService/SpecialHoursRepository, and publishes the same VoiceCallStatusUpdateEvent as the legacy flow. Separately, an alcohol recommendation pipeline uses user history and semantic search to write item recommendations to a database, then renders them through CSF, Discovery SDK, and VLP while a two-stage LLM pipeline generates and judges carousel titles.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  Scheduler["Scheduler"] --> Factory["RobocallCreatorFactory"]
  Factory --> Legacy["Legacy DTMF provider"]
  Factory --> Client["Voice AI provider client"]
  Client --> Alex["Voice agent Alex"]
  Alex --> Webhook["Webhook handler"]
  Webhook --> Hours["SpecialHoursRepository"]
  Webhook --> Event["VoiceCallStatusUpdateEvent"]
  Recs["Recommendation generator"] --> DB["Recommendation database"]
  DB --> CSF["Carousel Serving Framework"]
  CSF --> SDK["Discovery SDK"]
  SDK --> VLP["VLP Layout Framework"]
  Recs --> TitleGen["Title generation LLM"]
  TitleGen --> TitleJudge["Title judge LLM"]

Components

Data flow

For store hours, the scheduler asks the factory for a provider, the AI voice agent talks to the merchant, structured response data goes to a webhook, valid special-hours data triggers a MenuDataService update, and Kafka status events notify downstream systems. For alcohol recommendations, user history and search terms feed semantic search over orderable catalog products, recommendations are written to a database, CSF fetches them into the VLP, and LLM candidate generation plus a title judge produce carousel titles.

"Scheduler picks the provider via factory

2. Voice agent "Alex" talks to the merchant
3. VAPI sends structured response to webhook
4. Webhook interprets and triggers special hours update in MenuDataService"

Why this architecture

DoorDash used abstractions and existing framework contracts to add LLM capabilities without broad rewrites. Factory selection and feature flags let the voice system roll forward or back safely, while the old digits/status contract keeps downstream systems unchanged. For recommendations, CSF, Discovery SDK, and VLP separate recommendation content from presentation; the two-stage title pipeline fixes a specific observed title-quality failure mode.

Models used: LLMs · Vapi voice AI agent

Scale

• The voice rollout could target 1% of stores and roll back to 0% by flag.
  “on Monday, we can run 1% of stores on the AI path, but on Tuesday, we can roll back to 0% if something looks off”article, stated
• The alcohol title generator proposes 3-5 candidate titles per carousel.
  “**Candidate Generation:**The LLM proposes 3–5 short (3–5 word) titles based on the recommended items and its own analysis.”article, stated
• The legacy dispute system elsewhere in the same internship post showed the platform context for seconds-level operational pipelines.
  “We achieved a 99.9%+ reduction in processing delay, taking case creation time from more than 24 hours to mere seconds.”article, stated

Alternatives considered (roads not taken)

Keep same-day DTMF robocalls as the only merchant-hours workflow

Rejected because: DTMF calls could not capture detailed schedules or advanced special-hours information.

Bind the scheduler directly to one AI voice vendor

Rejected because: DoorDash wanted provider flexibility and future support for Vapi, Twilio, or Amazon Connect without changing business logic.

Create new downstream voice-call contracts

Rejected because: Reusing existing events and digits semantics kept alerting, dashboards, and consumers unchanged.

Use the first LLM-generated carousel title

Rejected because: The first title was not always natural or accurate, so a second LLM evaluates, filters, and re-ranks candidates.

Patterns

tool-use-orchestration (orchestration)

The voice assistant coordinates provider calls, webhook processing, special-hours mutations, and Kafka status events around a natural-language conversation.

5. Kafka events notify systems downstream"

— article, stated

structured-output-contract (integration)

The voice prompt always returns legacy-compatible digits plus normalized timestamps, and the webhook only reads the structured fields it needs.

fallback-degradation (reliability)

Factory selection and feature flags provide deterministic fallback from AI voice calls to legacy robocalls.

canary-rollout (release)

DoorDash can send a small percentage of stores through the AI path and roll back by changing feature flags.

prompt-chaining (prompting)

The carousel title workflow first generates candidates after category reasoning, then uses another LLM step to filter and re-rank them.

"For this, we developed a two-stage pipeline:

1. **Candidate Generation:**The LLM proposes 3–5 short (3–5 word) titles"

llm-as-judge (evaluation)

A second LLM judges title candidates, removes generic or editorial titles, and re-ranks the rest.

observability-tracing (operations)

Both systems rely on instrumentation or framework metrics to debug latency, fallback, and job-level failures.

Tradeoffs

Route through an abstraction layer

Chose RobocallCreator and provider factories selected by flags over direct coupling to one voice provider or rewriting downstream systems.

Rationale: The abstraction reduces vendor lock-in, blast radius, and rollout risk while preserving backward compatibility.

Preserve the legacy digits field

Chose dual output with digits plus normalized timestamps over a new voice-only schema that would force downstream changes.

Rationale: The old field keeps legacy semantics intact while new timestamps enable richer special-hours automation.

Use a two-stage LLM title pipeline

Chose candidate generation followed by LLM judging and re-ranking over taking the first generated title.

Rationale: The second stage improves naturalness and contextual accuracy of carousel titles.

Use existing carousel and discovery frameworks

Chose CSF, Discovery SDK, and VLP Layout Framework over building a bespoke carousel serving stack.

Rationale: Framework reuse lets engineers focus on recommendation content while preserving latency budgets and observability.

Gotchas (what bit them)

Legacy AI integration can break downstream assumptions if old contracts are not preserved.

Root cause: Downstream systems expected status events and digits semantics from DTMF robocalls.

Resolution: Return compatible digits, publish the same VoiceCallStatusUpdateEvent, and keep consumers unaware of whether the call was DTMF or natural language.

Open-ended conversations are unsafe unless converted into validated structured data.

Root cause: A merchant can phrase hours in many ways, but downstream systems need deterministic fields for business mutations.

Resolution: Guide the conversation, confirm back the interpreted schedule, normalize time formats, and validate preconditions in the webhook.

Incomplete voice data can corrupt store-hour source-of-truth systems.

Root cause: Webhook payloads may lack sufficient opening/closing times or valid digits.

Resolution: Make the webhook idempotent, schema-driven, strict about preconditions, and avoid partial writes while publishing observable status events.

LLM-generated titles can be misleading when they overfocus on one item category.

Root cause: A title like 'Unwinding with Wine' can be inaccurate if the carousel includes non-wine products.

Resolution: Have the model first summarize beverage categories, then judge and re-rank candidate titles.

Evaluation

The voice assistant is controlled through feature flags and observability for rollouts, fallbacks, and latency. The recommendation system used launch/employee testing plus LLM self-judging for title quality; the source does not provide quantitative customer-facing A/B metrics for these LLM systems.

Metrics: feature-flag rollout percentage · rollback percentage · cache hit rates · fallbacks · end-to-end latency · job-level metrics · error logs · employee testing feedback

Outcomes

• The voice system can switch between legacy robocalls and AI-powered conversations by merchant preferences and feature flags.
• Natural-language merchant responses can automatically update store hours through structured webhook processing.
• The alcohol recommendation email launched, and early employee testing showed the in-app carousel could support LLM-driven personalization.
• The title pipeline produced more accurate and contextual carousel titles after adding the judge-and-rerank step.

The source emphasizes controlled rollout rather than broad production metrics for the LLM systems; the voice path supports 1% to 0% feature-flag rollbacks, and the recommendation path integrates with DoorDash's New Verticals serving frameworks.