Automating Merchant Onboarding with Reinforcement Learning

doordash · E-commerce · 2026 · source Agentic: no · promotional bias: high · summary-only (no fetched source) · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Merchant onboarding and menu digitization

DoorDash needed to improve merchant menu-generation accuracy for messy real-world menu inputs without relying only on prompt tweaks or manually encoded standard operating procedures.

Constraints:

• menus arrive as photos, PDFs, and text documents
• menu formats have a long tail of messy styles
• human experts disagree on some optimal menu outputs
• production deployment must satisfy latency, cost, reliability, and security constraints
• source cache is unavailable, so this extraction is summary-only

Architecture

DoorDash and Applied Compute used a grader-first reinforcement learning approach. They built an automated grader aligned with internal expert standards, used it as a reward function for a menu error-correction model, trained against DoorDash's existing structured menu outputs, validated with automated, human, and live A/B tests, and integrated the resulting correction model as a production-ready library in DoorDash's codebase.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  menu-input-processor[Menu input processor] --> menu-correction-model[Menu correction model]
  menu-correction-model --> automated-grader[Automated grader]
  automated-grader --> rl-training-loop[RL training loop]
  rl-training-loop --> menu-correction-model
  menu-correction-model --> human-validation-process[Human validation process]
  human-validation-process --> production-menu-library[Production menu library]

Components

Data flow

Existing menu inputs are converted into structured menu output by DoorDash's baseline system. The correction model improves that structured output, the automated grader scores errors against DoorDash menu style guidelines, and reinforcement learning uses the grader signal to improve model behavior. Human graders and A/B tests validate the trained model before rollout to USA menu traffic.

Why this architecture

DoorDash reframed the problem around grading because specifying perfect generation rules was harder than judging outputs against ground truth. A calibrated grader gave reinforcement learning a domain-specific reward signal, while human validation and A/B testing prevented the team from trusting the grader alone.

Models used: menu error correction model · automated grader · reinforcement learning training process

Scale

• DoorDash rolled the model out to all USA menu traffic.
  “The case study states that DoorDash has rolled out the error correction model to all menu traffic in the USA, indicating a full production deployment rather than a limited pilot.”summary, stated
• The A/B test reported a roughly 30% relative reduction in low-quality menus.
  “The test showed that the share of low-quality menus fell by approximately 30% relative to the baseline—the key result highlighted throughout the case study.”summary, stated
• The project moved from initial development to full production rollout across all USA traffic.
  “This case represents a mature LLMOps deployment that went from initial development through rigorous validation to full production rollout across all USA traffic.”summary, stated

Alternatives considered (roads not taken)

Create a unified standard operating procedure for menu generation

Rejected because: The summary says even human experts disagreed on some optimal outputs.

Incremental prompt or fine-tuning improvements

Rejected because: The team saw diminishing returns and moved to reward-based optimization.

Trust the automated grader alone

Rejected because: DoorDash added human offline validation and live A/B testing.

Patterns

llm-as-judge (evaluation)

A calibrated automated grader scores menu quality and becomes the foundation for training and validation.

eval-driven-development (evaluation)

Model improvement is driven by grader alignment, offline human grading, and live A/B testing.

data-quality-loop (data)

Production corrections can feed future training, creating a repeatable improvement loop.

human-in-the-loop-gate (workflow)

Human graders validated model outputs on held-out data and live A/B samples.

Tradeoffs

Invest in evaluation before model optimization

Chose grader-first reinforcement learning over more prompt engineering or direct fine-tuning first.

Rationale: Reliable evaluation was easier to specify than a unified generation policy and enabled domain-specific rewards.

Use a vendor partnership model

Chose Applied Compute working onsite with DoorDash over arms-length tool use.

Rationale: Close collaboration likely supported domain alignment but may create dependency and knowledge-transfer concerns.

Optimize for production constraints

Chose accuracy balanced with latency, cost, and reliability over offline accuracy alone.

Rationale: The menu correction model had to be viable inside merchant onboarding traffic.

Gotchas (what bit them)

The grader becomes a hidden dependency.

Root cause: If the grader is misaligned, reinforcement learning optimizes the wrong behavior.

Resolution: DoorDash emphasized alignment, self-consistency, and human validation before production rollout.

Promotional reporting omits important deployment details.

Root cause: The case study does not disclose absolute error rates, sample sizes, or statistical confidence intervals.

Resolution: Treat the 30% relative reduction as directionally useful but incomplete.

USA rollout may not generalize globally.

Root cause: International menu formats, languages, and cuisines may differ from USA traffic.

Resolution: The summary identifies retraining or fine-tuning as likely needed for other markets.

Evaluation

DoorDash used automated grader scores, human offline grading, and production A/B testing with human reviewers grading both baseline and new outputs.

Metrics: low-quality menu share · automated-human grader alignment · latency · cost · reliability · production A/B outcome

Outcomes

• Low-quality menus fell by about 30% relative to baseline.
• The correction model was rolled out to all USA menu traffic.
• The case demonstrates encoding reviewer judgment into automated training signals.
• The summary flags missing statistics and cost-benefit details.

Summary-only evidence supports a USA-traffic rollout and relative quality reduction, but not absolute error rates or sample sizes.