RoBERTa for Large-Scale Merchant Classification

square · Finance · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Merchant categorization for payments and business services

Square needed more accurate merchant categorization because self-selected onboarding categories were noisy and downstream product, marketing, compliance, forecasting, and interchange-fee decisions depended on knowing what business each seller actually ran.

Constraints:

• Self-selected categories are often inaccurate because sellers rush onboarding or misunderstand granular options.
• Catalog inputs can be noisy because Square auto-creates services from the seller's selected category.
• Some merchants have catalogs with tens of thousands of items, exceeding transformer token limits.
• Daily inference must cover tens of millions of sellers without taking weeks.
• The system needs historical partitions for drift monitoring and latest predictions for business users.

Architecture

Square built a RoBERTa-large merchant categorization model trained on more than 20,000 manually reviewed sellers. The pipeline removes misleading auto-created services, ranks catalog items by purchase frequency to fit token limits, formats business/category/catalog signals into a structured input string, fine-tunes RoBERTa for sequence classification on GPU-backed Databricks, and runs daily batch inference with multiple GPU workers, PySpark parallelism, batch-size tuning, and incremental predictions.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["Merchant profile signals"] --> B["Data preprocessing pipeline"]
  C["Manual review labels"] --> D["RoBERTa training job"]
  B --> D
  D --> E["Trained merchant classifier"]
  B --> F["Daily inference job"]
  E --> F
  F --> G["Historical prediction table"]
  F --> H["Latest prediction table"]
  G --> I["Business analytics systems"]
  H --> I

Components

Data flow

Merchant business names, self-selected categories, and catalog items feed preprocessing. Auto-created services that were never manually modified are removed, catalog items are ranked by purchase frequency and truncated under token limits, and the remaining signals are formatted into an input string. A Hugging Face RoBERTa sequence classifier is trained on manually reviewed labels, then daily inference runs across merchants on GPU/PySpark infrastructure and writes historical and latest prediction tables.

Why this architecture

Square chose a supervised RoBERTa classifier because the problem was language understanding over merchant text, not generation. Manually reviewed labels fixed the weak-label problem, post-onboarding signals gave richer evidence than onboarding self-selection, and GPU/PySpark batch infrastructure made daily inference practical at tens-of-millions scale.

Models used: RoBERTa-large · AutoModelForSequenceClassification

Scale

• The model was trained on more than 20,000 manually reviewed sellers.
  “The model is trained on a random sample of over 20,000 manually reviewed sellers, ensuring high-quality ground truth labels for our training and evaluation data.”article, stated
• Some merchants can have catalogs with tens of thousands of items.
  “Some merchants sell a lot of things (e.g. possibly tens of thousands), and so we need to truncate their catalog items”article, stated
• Daily inference runs over tens of millions of sellers.
  “This model performed inference daily and, given the vast number of sellers on the platform (in the tens of millions!), was another area that demanded even more GPU support.”article, stated
• The classifier improved category accuracy by roughly 30 absolute percentage points.
  “The model is estimated to have an absolute ~30% improvement in accuracy of determining seller business category than existing methods.”article, stated

Alternatives considered (roads not taken)

Rely on merchant self-selection

Rejected because: Self-selected categories are frequently wrong and misrepresent a meaningful portion of sellers.

Use auto-created catalog services as ordinary product evidence

Rejected because: Auto-created services can reflect the wrong onboarding category rather than the merchant's real offerings.

Run every merchant through inference every day

Rejected because: Incremental prediction skipping reduces inference time by avoiding unchanged inputs.

Use one fixed inference batch size

Rejected because: Batch size improves GPU efficiency only up to a point and can create memory issues when too large.

Patterns

fine-tuning-adaptation (model-adaptation)

Square fine-tuned RoBERTa for merchant category sequence classification using manually reviewed merchant labels and domain-specific inputs.

data-quality-loop (data)

The team replaced noisy self-selected labels with manually reviewed ground truth and cleaned catalog signals before training.

batch-offline-processing (serving)

Square runs daily large-scale inference jobs with GPU workers, PySpark parallelism, batch tuning, and incremental prediction skipping.

observability-tracing (operations)

Historical daily prediction partitions support troubleshooting, monitoring, and model-drift validation.

structured-output-contract (outputs)

The production model writes category predictions into defined historical and latest prediction tables for business systems.

Tradeoffs

Ground truth strategy

Chose manual review labels over self-selected onboarding categories.

Rationale: Manual review costs more up front but avoids training on inaccurate seller self-selection.

Catalog truncation signal

Chose purchase-frequency-ranked catalog items over arbitrary catalog truncation or all items.

Rationale: Purchase frequency preserves the items most representative of what the merchant actually does while staying under token limits.

Inference optimization

Chose incremental predictions for changed inputs only over rerunning every seller every day.

Rationale: Skipping unchanged inputs reduces daily inference time at tens-of-millions scale.

Gotchas (what bit them)

Self-selected merchant categories were unreliable training targets.

Root cause: Sellers can rush onboarding, misunderstand granular choices, or select the wrong category.

Resolution: Train and evaluate on a manually reviewed seller sample instead of treating self-selection as truth.

Auto-created catalog items created false evidence.

Root cause: Square creates default services from the selected onboarding category, so a wrong selection can create catalog items that do not represent the business.

Resolution: Remove auto-created services that merchants never manually modified.

Catalog length could exceed transformer token limits.

Root cause: Some merchants sell very large numbers of items.

Resolution: Rank catalog items by purchase frequency and truncate after the token threshold.

Daily inference at tens-of-millions scale could take too long.

Root cause: Running predictions for all sellers without parallelism or skipping unchanged inputs would create large delays.

Resolution: Use multiple GPU workers, PySpark, batch-size optimization, and incremental predictions.

Evaluation

Square evaluated the classifier against held-out manually reviewed samples and compared category accuracy gains to merchant self-selection, while also storing daily historical predictions for monitoring and drift validation.

Metrics: Absolute approximately 30% accuracy improvement over existing methods · Accuracy gains summarized across major business categories · Daily historical partitions for troubleshooting and model drift checks

Outcomes

• The RoBERTa classifier achieved an estimated absolute 30% category-accuracy improvement.
• Daily predictions are used for merchant segmentation across business metrics.
• The output supports product strategy, go-to-market targeting, email campaigns, and future interchange-fee use cases.
• Historical prediction partitions create an operational record for troubleshooting and drift monitoring.

The system performs daily inference for tens of millions of sellers and handles merchants with catalogs that can contain tens of thousands of items.