Building a High-Quality RAG-based Support System with LLM Guardrails and Quality Monitoring

doordash · E-commerce · 2024 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Dasher support automation

DoorDash needed to improve automated support for Dashers beyond brittle flow-based resolutions while preventing RAG hallucinations, compliance failures, and low-quality answers in a high-volume support setting.

Constraints:

• The existing flow-based system could resolve only a small subset of Dasher issues quickly.
• Knowledge-base articles were hard to find, hard to extract from, and English-only for many non-English-preferring Dashers.
• LLM responses had to be grounded, coherent with conversation history, and compliant before reaching end users.
• Guardrail checks had to balance quality against latency and token cost.
• Human support still had to handle complex or risky cases.

Architecture

DoorDash built a RAG-based Dasher support chatbot with online guardrails and offline quality monitoring. The RAG flow condenses a multi-turn conversation into a core issue, retrieves top-N similar historical cases mapped to KB articles, injects article content into prompt templates, and generates a tailored response. An LLM Guardrail applies a shallow semantic-similarity check first and only escalates failures to an LLM evaluator. LLM Judge, rules, and human review provide continuous quality monitoring and improvement.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  dasher-conversation[Dasher conversation] --> conversation-summarizer[Conversation summarizer]
  conversation-summarizer --> historical-case-retriever[Historical case retriever]
  historical-case-retriever --> kb-prompt-template[KB prompt template]
  kb-prompt-template --> llm-chatbot[LLM chatbot]
  llm-chatbot --> llm-guardrail[LLM Guardrail]
  llm-guardrail --> human-support-agents[Human support agents]
  llm-guardrail --> llm-judge-monitoring[LLM Judge monitoring]
  llm-judge-monitoring --> human-review-team[Human review team]

Components

Data flow

A Dasher issue is summarized from the conversation, used to search historical cases, and mapped to relevant KB articles. The article content is integrated into a prompt template for generation. The output must pass a semantic-similarity guardrail or an LLM evaluator before being shown; monitoring then uses LLM Judge, rules, and human samples to feed improvements back into prompts and retrieval.

Why this architecture

DoorDash chose RAG because KB articles contained the support knowledge, but discoverability, extraction, and language barriers made the old flow-based system inadequate. The two-tier guardrail exists because always using an LLM evaluator would be too expensive and slow, while semantic similarity can cheaply catch grounded answers and reserve LLM evaluation for uncertain cases.

Models used: GPT-4 · Claude-3 · LLM Guardrail · LLM Judge · RAG chatbot

Scale

• The system autonomously assists thousands of Dashers each day.
  “Each day, this system autonomously assists thousands of Dashers, streamlining basic support requests while maximizing the value of human contributions.”article, stated
• The guardrail reduced hallucinations by 90%.
  “This guardrail system has successfully reduced overall hallucinations by 90% and cut down potentially severe compliance issues by 99%.”article, stated
• The guardrail reduced severe compliance issues by 99%.
  “reduced overall hallucinations by 90% and cut down potentially severe compliance issues by 99%.”article, stated
• DoorDash manually reviewed thousands of chat transcripts to build the monitoring pipeline.
  “So we manually reviewed thousands of chat transcripts between the LLM and Dashers to develop an iteration pipeline for monitoring LLM quality.”article, stated

Alternatives considered (roads not taken)

Continue relying on pre-built flow-based support paths.

Rejected because: The flow system resolved only a small subset of Dasher issues quickly.

Use LLM evaluation for every guardrail check.

Rejected because: The article describes LLM Guardrail latency as a drawback and uses semantic similarity as the first shallow check.

Force automation for all problematic responses.

Rejected because: DoorDash defaults some problematic responses to human agents to preserve support quality.

Patterns

rag-retrieval (retrieval)

DoorDash retrieves similar historical cases and KB articles, then injects the article into a prompt template for the support response.

guardrails-io-filtering (governance)

Responses must pass semantic similarity and, when needed, an LLM evaluator before reaching the Dasher.

llm-as-judge (evaluation)

LLM Judge monitors retrieval correctness, response accuracy, grammar/language accuracy, context coherence, and relevance.

human-in-the-loop-gate (governance)

Human support agents handle complex fallback cases and a dedicated review team calibrates automated quality monitoring.

Tradeoffs

Guardrail depth

Chose A two-tier semantic-similarity then LLM-evaluator guardrail. over Running the expensive LLM evaluator on every response..

Rationale: Semantic similarity cheaply catches responses grounded in KB article segments, while LLM evaluation handles flagged responses.

Automation versus support quality

Chose High automation with human-agent fallback. over Forcing the LLM to answer every support request..

Rationale: Human defaulting preserves quality for the smaller number of problematic responses.

Open-ended quality discovery

Chose LLM Judge open-ended questions summarized into common issues and then converted into prompts/rules. over Only tracking business-level metrics such as feedback and speed..

Rationale: The open-ended monitor produces actionable categories for prompt, retrieval, and KB improvement.

Gotchas (what bit them)

RAG answers can sound fluent while diverging from DoorDash support context.

Root cause: Foundation models may rely on stale or public DoorDash-related training data instead of the intended KB article.

Resolution: DoorDash checks groundedness and relevance with a two-tier LLM Guardrail before showing responses.

Online guardrails add latency to the user path.

Root cause: The full flow may generate a response, apply guardrails, and retry with a new check.

Resolution: DoorDash uses a shallow semantic check first and defaults some problematic responses to human agents.

Aggregate support metrics were not actionable enough to improve the chatbot.

Root cause: Dasher feedback, human engagement rate, and delivery speed did not identify concrete failure modes.

Resolution: DoorDash manually reviewed transcripts and categorized quality into monitored aspects.

Evaluation

DoorDash evaluated the system with online guardrail pass/fail checks, LLM Judge quality monitors, human transcript review, hallucination/compliance reductions, and operational support volume.

Metrics: 90% reduction in overall hallucinations · 99% reduction in potentially severe compliance issues · Thousands of chat transcripts reviewed for monitoring design · Thousands of Dashers assisted daily · Quality dimensions: retrieval correctness, response accuracy, grammar/language accuracy, coherence to context, relevance

Outcomes

• Reduced overall hallucinations by 90%.
• Reduced potentially severe compliance issues by 99%.
• Autonomously assists thousands of Dashers daily.
• Allows human support representatives to focus on more complex Dasher problems.

The support system handles thousands of Dasher interactions daily, was built from thousands of transcript reviews, and reports 90% hallucination and 99% severe-compliance reductions.