Generative AI Contact Center Solution with Amazon Bedrock and Claude

doordash · E-commerce · 2024 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: contact-center self-service support

DoorDash wanted to enhance an existing voice AI assistant for Dasher phone support with generative AI responses that were fast enough for voice interactions, accurate enough for self-service, safe against hallucination and abuse, and testable at production A/B scale.

Constraints:

• phone support required response latency of 2.5 seconds or less
• AI-generated phone responses had to improve answer quality while preserving fast response times
• the system needed access to a deep and diverse knowledge base
• DoorDash did not provide personally identifiable information to the generative AI solution
• the solution had to mitigate hallucinations, prompt injection, and abusive language
• manual test capacity depended on pulling contact-center agents off help queues
• the architecture had to be suitable for production A/B testing

Architecture

DoorDash worked with the AWS Generative AI Innovation Center to build an Amazon Bedrock-based contact-center assistant using Anthropic Claude, Knowledge Bases for Amazon Bedrock, public help-center content, and a SageMaker-based test and evaluation framework. The system enriches prompts with retrieved DoorDash help-center knowledge, uses Claude 3 Haiku for low-latency voice responses, keeps PII out of the generative AI solution, and evaluates responses against ground-truth data at scale before production A/B testing.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  dasher_phone_support_user["Dasher phone-support user"] --> voice_ai_assistant["Voice AI assistant"]
  public_help_center_content["Public help-center content"] --> knowledge_bases_for_amazon_bedrock["Knowledge Bases for Amazon Bedrock"]
  knowledge_bases_for_amazon_bedrock --> amazon_bedrock["Amazon Bedrock"]
  voice_ai_assistant --> amazon_bedrock
  amazon_bedrock --> anthropic_claude_3_haiku["Anthropic Claude 3 Haiku"]
  anthropic_claude_3_haiku --> voice_ai_assistant
  voice_ai_assistant --> dasher_phone_support_user
  sageMaker_evaluation_framework["SageMaker evaluation framework"] --> ground_truth_response_data["Ground-truth response data"]
  sageMaker_evaluation_framework --> voice_ai_assistant
  live_contact_center_agents["Live contact-center agents"] --> dasher_phone_support_user

Components

Data flow

DoorDash indexes publicly available help-center content with Knowledge Bases for Amazon Bedrock. The voice AI assistant uses RAG to fetch relevant help content and enrich the prompt sent to Anthropic Claude in Amazon Bedrock. Claude generates a response for the Dasher support interaction, while a SageMaker test-and-evaluation framework runs automated test cases, draws insights from A/B testing, and semantically evaluates generated responses against ground-truth data.

Why this architecture

DoorDash chose Bedrock and Claude because the contact-center use case needed fast, safe, and privacy-aware model access. Knowledge Bases for Bedrock supplied managed RAG over help-center content, Claude 3 Haiku met the speed and accuracy requirements for a voice application, and SageMaker let the team evaluate responses at a scale that manual agent testing could not support.

Models used: Anthropic Claude models · Claude 3 Haiku

Scale

• The AWS and DoorDash teams iterated over the design and implementation for 8 weeks.
  “Over 8 weeks, teams from DoorDash and the GenAIIC iterated over design and implementation”article, stated
• The Bedrock-based solution reduced generative AI application development time by 50 percent.
  “The Amazon Bedrock–based solution reduced generative AI application development time by 50 percent.”article, stated
• Claude 3 Haiku met a 2.5-second-or-less response latency target for the voice application.
  “achieving a response latency of 2.5 seconds or less.”article, stated
• The automated evaluation framework completes thousands of tests per hour.
  “complete thousands of automated tests per hour—a 50x increase in capacity”article, stated

Alternatives considered (roads not taken)

Manual contact-center agent testing

Rejected because: Manual tests required pulling agents off help queues and produced only a small number of test cases per hour.

Build custom RAG ingestion and prompt-augmentation integrations

Rejected because: Knowledge Bases for Amazon Bedrock handled ingestion, retrieval, prompt augmentation, and backend data-flow management.

Expose personally identifiable information to the generative AI solution

Rejected because: DoorDash explicitly withheld PII from these generative AI solutions while using Bedrock security and encryption.

Route routine Dasher questions to live agents by default

Rejected because: Automating common inquiries improved self-service and freed live agents for more complex troubleshooting.

Patterns

rag-retrieval (support knowledge grounding)

DoorDash uses Knowledge Bases for Amazon Bedrock to retrieve public help-center content and enrich Claude prompts for Dasher support answers.

eval-driven-development (contact-center testing)

DoorDash built an automated SageMaker evaluation framework to support A/B testing, evaluate key success metrics, and semantically compare responses with ground truth.

guardrails-io-filtering (response safety)

DoorDash chose Claude partly for its ability to mitigate hallucinations, prompt injection, and abusive language in the contact-center flow.

privacy-preserving-processing (customer data handling)

DoorDash kept PII out of the generative AI solution and used Bedrock's encryption and application-specific data handling boundaries.

Tradeoffs

Use Claude 3 Haiku for the voice assistant.

Chose Claude 3 Haiku through Amazon Bedrock over A slower or less safety-capable model choice.

Rationale: DoorDash needed both low response latency and mitigation of hallucination, prompt injection, and abusive language for phone support.

Use managed Bedrock Knowledge Bases for RAG.

Chose Knowledge Bases for Amazon Bedrock over Custom ingestion, retrieval, prompt augmentation, and data-flow integrations.

Rationale: The managed service handled the RAG backend work, letting DoorDash focus on solution behavior and production testing.

Automate evaluation instead of depending on live support agents.

Chose A SageMaker automated test-and-evaluation framework over Manual test cases performed by contact-center agents.

Rationale: Automated testing increased capacity 50x and avoided pulling agents away from help queues.

Use GenAI for routine self-service while preserving live agents for complex issues.

Chose Automated handling of common Dasher inquiries over Routing all routine questions to live agents.

Rationale: Routine automation improved self-service and issue resolution while allowing humans to handle higher-complexity troubleshooting.

Gotchas (what bit them)

Phone support leaves little room for slow model responses.

Root cause: Voice interactions require low latency while still producing accurate and safe generated responses.

Resolution: DoorDash used Claude 3 Haiku through Bedrock to reach the speed and accuracy needed for the voice application.

Support responses need deep domain knowledge, not only a base model.

Root cause: Self-service contact-center quality depends on access to a detailed and diverse knowledge base.

Resolution: DoorDash indexed public help-center content with Knowledge Bases for Amazon Bedrock and used RAG to enrich prompts.

Manual testing could not keep up with production-quality evaluation needs.

Root cause: Manual contact-center tests required agents to leave support queues and yielded only a small number of cases per hour.

Resolution: DoorDash built a SageMaker test-and-evaluation framework that runs thousands of automated tests per hour and supports A/B insights.

Contact-center GenAI needs explicit privacy boundaries.

Root cause: Support systems may involve customer data, but DoorDash did not want PII accessible through the generative AI solution.

Resolution: DoorDash withheld PII from the solution and relied on Bedrock encryption and application-specific data use controls.

Evaluation

DoorDash developed a SageMaker test-and-evaluation framework for production A/B testing, key success metrics, and semantic evaluation against ground-truth data. The source reports automation capacity and latency outcomes rather than a full metric table.

Metrics: response latency of 2.5 seconds or less · thousands of automated tests per hour · 50x testing capacity increase · key success metrics from A/B testing · semantic match against ground-truth data · generative AI application development time reduced by 50 percent

Outcomes

• The Bedrock-based solution reduced generative AI application development time by 50 percent.
• DoorDash achieved 2.5-second-or-less response latency with Claude 3 Haiku.
• The automated evaluation framework increased testing capacity 50x to thousands of tests per hour.
• The self-service assistant improved issue resolution speed and Dasher productivity and satisfaction.
• Routine automation freed live agents to focus on higher-complexity troubleshooting.

The case is a production contact-center application prepared for A/B testing after an 8-week joint build with AWS GenAIIC. The strongest reported scale number is the test framework's thousands of automated tests per hour and 50x capacity increase.