Scaling Order Processing Automation Using Modular LLM Architecture

choco · E-commerce · 2025 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: food distributor order automation

Choco needed to automate distributor order intake from unstructured channels such as email, voicemail, SMS, WhatsApp, and fax, while still identifying customer-specific products accurately enough to sync orders into ERP systems.

Constraints:

• Orders arrive in varied, unstructured formats and often omit exact catalog identifiers.
• Distributors need ERP-ready structured order details, not just extracted text.
• The system has to learn from corrections while keeping humans in the loop for early or uncertain orders.

Architecture

Choco AI consolidates incoming orders into a review UI, uses multiple LLMs and ML models for task-specific steps such as transcription, correction, extraction, and product matching, displays predicted order details beside the raw input, lets distributors correct mispredictions, and feeds user/customer-success/internal-labeling corrections back into evaluation and improvement loops.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["Order channels"] --> B["Unified order interface"]
  B --> C["Task-specific LLM pipeline"]
  C --> D["Product matching models"]
  E["Distributor catalog"] --> D
  D --> F["Human review UI"]
  F --> G["ERP sync"]
  F --> H["Evaluation pipeline"]
  I["Internal labeling tools"] --> H
  H --> C

Components

Data flow

Orders enter from email, voicemail, SMS, WhatsApp, or fax; the system transcribes or extracts text, identifies details such as customer, product, and quantity, matches them to distributor catalog items, presents predictions in a review UI, accepts edits, syncs approved orders to ERP, and stores corrections/labels for evaluation and future improvement.

Why this architecture

Choco split the workflow into narrow model responsibilities because modularity made the system easier to evaluate, debug, optimize, and scale than a single multimodal call over the whole order.

Models used: multiple LLMs · speech-to-text models · custom ML product matching models · GPT-4o

Scale

• One distributor reduced manual order entry time by 60%.
  “Choco AI helped a distributor reduce manual order entry time by 60%, allowing them to process 50% more orders daily”article, stated
• Choco AI achieved over 95% correct predictions at the time of the article.
  “This modular approach ensures high accuracy, with Choco AI currently achieving over 95% correct predictions.”article, stated
• Choco manually labeled tens of thousands of examples for evaluation and training.
  “Over time, we’ve manually labeled tens of thousands of examples for different tasks while maintaining stringent data privacy practices.”article, stated
• GPT-4o was tested and moved to production within a week for a targeted large-order task.
  “When GPT-4o was released, we had it running in production within a week.”article, stated

Alternatives considered (roads not taken)

One-step multimodal model for voicemail orders

Rejected because: Choco preferred separate testable responsibilities for transcription, correction, and extraction.

Outsource ground-truth labeling broadly

Rejected because: External agencies lacked domain expertise and produced unreliable labels for ambiguous distributor orders.

Use LLMs to replace human judgment and labeling

Rejected because: Evaluator/generator automation still requires oversight and labeled-data evaluation; Choco prioritized the primary task first.

Patterns

tool-use-orchestration (application-architecture)

Multiple model and integration steps are orchestrated to transform unstructured orders into ERP-ready structured entries.

human-in-the-loop-gate (quality-control)

Customers review and correct predicted order entries before ERP sync, especially during initial learning.

eval-driven-development (evaluation)

Component and end-to-end test datasets drive model changes, including rapid GPT-4o evaluation.

context-engineering (context-management)

The system uses personalized distributor/customer context to disambiguate vague orders like tomato requests.

Tradeoffs

Model architecture

Chose modular multi-step LLM/ML pipeline over single end-to-end multimodal model call.

Rationale: Modularity made responsibilities independently testable and easier to optimize.

Evaluation strategy

Chose large labeled test datasets and offline evaluation over online evaluation alone.

Rationale: Online evaluation was too slow or costly to rely on exclusively for iteration.

Improvement loop

Chose productized user corrections plus internal tools over manual AI-engineering intervention for every failure.

Rationale: Self-service corrections reduce disruptions as new distributors onboard weekly.

Gotchas (what bit them)

Ambiguous natural-language orders need customer-specific context.

Root cause: A phrase like “2 kilos of tomatoes” can map to many brands, packages, and customer expectations.

Resolution: Use distributor catalogs, customer preference context, and product matching models.

Labeling without domain expertise produced unreliable ground truth.

Root cause: Ambiguous speech, regional accents, and distributor-specific operations require expert judgment.

Resolution: Shift labeling toward internal Customer Success teams and internal tools.

Every model failure could become an engineering support ticket.

Root cause: New distributors and edge cases generate ongoing correction needs.

Resolution: Build self-service review, labeling, training, and observability mechanisms into the product.

Evaluation

Choco evaluates individual tasks and end-to-end order quality with labeled test datasets, task-specific metrics such as WER, Day-0 performance, learning-curve metrics, and customer-level edit reduction.

Metrics: word error rate · correct prediction rate · Day-0 performance · learning curve · edits required per order · manual order entry time

Outcomes

• Choco AI reduced manual order entry time by 60% for one distributor.
• The same distributor processed 50% more orders daily without additional staffing.
• The system achieved over 95% correct predictions.
• Choco could evaluate and ship GPT-4o for a targeted task within a week.

The article is a Choco engineering/product story and reports strong operational results; it is still grounded in production customer workflow rather than an independent benchmark.