AI Agent for Automated Merchant Classification and Transaction Matching

ramp · Finance · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.2 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Financial transaction enrichment

Ramp needed to fix ambiguous merchant matches at transaction scale without hours of manual support and engineering intervention.

Constraints:

• cryptic processor data
• user-provided context
• policy impact
• low-latency customer workflow

Architecture

Ramp built a merchant-classification correction agent around user-supplied correction requests. A context builder combines card-acceptor fields, MCCs, receipt-derived merchant names/addresses/line items, user memos, and related merchants retrieved by embeddings plus name matching. The primary LLM maps the request to one of a small set of actions, guardrails reject hallucinated or unsafe actions and force retries, a second LLM can rewrite rejection reasoning for users, and evaluation progresses from manual review to follow-up/rejection signals, LLM judging, and shadow-mode testing.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  correction-request-ui[Correction request UI] --> context-builder[Context builder]
  context-builder --> related-merchant-retriever[Related merchant retriever]
  related-merchant-retriever --> merchant-database[Merchant database]
  context-builder --> primary-llm-decision-agent[Primary LLM decision agent]
  primary-llm-decision-agent --> guardrail-validator[Guardrail validator]
  guardrail-validator --> primary-llm-decision-agent
  primary-llm-decision-agent --> merchant-action-executor[Merchant action executor]
  merchant-action-executor --> merchant-database
  merchant-action-executor --> correction-request-ui
  primary-llm-decision-agent --> rejection-explanation-llm[Rejection explanation LLM]
  evaluation-harness[Evaluation harness] --> primary-llm-decision-agent

Components

Data flow

A user reports a wrong merchant by giving a proposed merchant name, website, and category. Ramp builds LLM context from transaction card-acceptor metadata, receipt/memo evidence, and related merchants fetched by vector and name search. The LLM decides whether to create, update, reassign, or reject; guardrails ensure the selected action and target are allowed, retrying hallucinated responses. If the action changes the merchant classification, the UI updates within seconds; if rejected, another LLM rewrites the reason for user-facing explanation. Evaluation uses manual review early, weak production signals, and an LLM judge at rollout scale.

Why this architecture

Ramp needed more than name matching because payment-processor data is cryptic and user context may be the only disambiguating signal. The architecture keeps the LLM useful by giving it targeted merchant context rather than every record, then boxing actions inside guardrails because changing merchant records incorrectly can affect policy enforcement and customer workflows.

Models used: LLM · second LLM for rejection explanation · LLM judge

Scale

• Manual correction time was hours, while the agent unblocks customers in less than 10 seconds.
  “Instead of waiting for hours for manual human intervention, our customers are unblocked by our agent in less than 10 seconds.”article, stated
• Manual teams handled only 3% of requests in 2023 and 1.5% in 2024, while the agent handles close to 100%.
  “Before: Customer support and engineering teams were able to service only 3% of requests in 2023 and 1.5% in 2024. As Ramp grows, these percents will only go down.”article, stated
• The LLM judge reports nearly 99% transaction-classification improvement and nearly two-thirds reasonable rejections.
  “According to our judge, our agent improves nearly 99% of transaction classifications and nearly two thirds of rejections are reasonable.”article, stated
• The LLM-judge evaluation phase was chosen because rollout required evaluating at least 100 requests per day.
  “Once we were confident with our agent and began to roll out to more and more customers, we needed an evaluation strategy that satisfied two requirements:”article, stated

Alternatives considered (roads not taken)

Manual support/finance/engineering correction for most requests

Rejected because: The workload scales with card transactions and hours-long manual handling could not keep up with growth.

Use only the user-submitted merchant information

Rejected because: The LLM would not know whether an existing merchant record already exists or whether the requested category is reasonable.

Put every Ramp merchant into the prompt

Rejected because: It would not scale and would overload the LLM context window, so Ramp retrieves K related merchants instead.

Unconstrained free-form LLM actions

Rejected because: Guardrails require the LLM to choose only supplied actions/targets and retry when hallucinating.

Patterns

rag-retrieval (retrieval)

The agent combines transaction metadata, receipts, memos, and related merchant signals through RAG.

guardrails-io-filtering (safety)

Ramp explicitly uses constructed guardrails around the agent’s classification changes.

Tradeoffs

How to serve misclassification requests at scale.

Chose automated agentic correction over manual support escalation for most requests.

Rationale: Manual handling did not scale with transaction volume; agentic handling reduced wait time and support cost.

Merchant context retrieval.

Chose RAG over K related merchants from embeddings and name matching over placing every Ramp merchant in the LLM context.

Rationale: The retrieved subset gives the LLM enough candidates without overloading the context window.

Action safety.

Chose allowed action sets and post-processing guardrails with retries over free-form LLM mutations to merchant records.

Rationale: Constrained actions make guardrails enforceable and prevent unsafe merchant changes or hallucinated reassignment targets.

Gotchas (what bit them)

Processor merchant fields are ambiguous and misleading.

Root cause: Card acceptor names and categories often do not map cleanly to the real merchant or user intent.

Resolution: Use user context, receipt-derived signals, embeddings, and OLAP lookup instead of relying on processor metadata alone.

Evaluation

Ramp monitored production behavior and used LLM-based evaluation for accuracy claims.

Metrics: nearly all reports handled · less than 10 seconds · 99% accuracy in summary

Outcomes

• Customers are reportedly unblocked in under 10 seconds instead of waiting hours.

This source is a Ramp engineering post with strong self-reported gains.