Using RAG to Improve Industry Classification Accuracy

ramp · Finance · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Fintech LLM platform portfolio

Ramp needed production LLM capabilities across financial data analysis, merchant matching, expense approvals, industry classification, resource integration, and retrieval infrastructure.

Constraints:

The source is a portfolio-style summary of multiple Ramp engineering posts rather than one full implementation deep dive.
Financial applications require trust, accuracy, compliance, and auditability.
Merchant matching and classification depend on retrieval over current business data.
Reusable infrastructure matters because Ramp is deploying LLMs across several product surfaces.
Company-published claims such as state-of-the-art classification should be treated cautiously.

Architecture

Ramp appears to be building a portfolio LLMOps platform: an agentic data analyst answering data questions, smart RAG for merchant matching, trustworthy agents for expense approvals, RAG-based industry classification, transaction embeddings for retrieval, an MCP server for structured resource access, and ML infrastructure such as YAML configuration and Metaflow to support iteration and deployment.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  ramp-product-workflows["Ramp product workflows"] --> llm-application-portfolio["LLM application portfolio"]
  llm-application-portfolio --> rag-and-embedding-layer["RAG and embedding layer"]
  rag-and-embedding-layer --> transaction-embeddings["Transaction embeddings"]
  llm-application-portfolio --> mcp-resource-server["MCP resource server"]
  mcp-resource-server --> ramp-resources["Ramp resources"]
  llm-application-portfolio --> ml-infrastructure["ML infrastructure"]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Ramp product workflows`	Expense, merchant, data analysis, and classification surfaces using LLM capabilities.	`LLM application portfolio`
`LLM application portfolio`	Includes data analyst agents, merchant matching, expense approvals, and industry classification.	`Ramp product workflows`, `RAG and embedding layer`, `MCP resource server`, `ML infrastructure`
`RAG and embedding layer`	Retrieves merchants, transactions, industry examples, categories, and definitions for LLM decisions.	`LLM application portfolio`, `Transaction embeddings`
`Transaction embeddings`	Improve retrieval and accounting functionality through learned transaction similarity.	`RAG and embedding layer`
`MCP resource server`	Gives LLMs structured access to company resources and tools.	`LLM application portfolio`, `Ramp resources`
`Ramp resources`	Company data and operational resources exposed through MCP.	`MCP resource server`
`ML infrastructure`	Provides configuration, Metaflow workflows, reproducibility, and deployment support.	`LLM application portfolio`

Data flow

Financial-product workflows call into shared LLM infrastructure. Retrieval and embedding systems surface merchants, transactions, industry definitions, or company resources; LLMs or agents perform matching, classification, analysis, or approval reasoning; MCP standardizes tool/data access; and underlying ML infrastructure supports configuration, experimentation, and deployment across use cases.

3 supporting quotes

“Their work spans multiple aspects of LLMOps including agentic systems, RAG implementations, embedding-based retrieval, infrastructure for LLM integration, and production ML systems.”summary, stated

“Building an MCP server represents significant infrastructure investment in LLMOps.”summary, stated

“Good LLMOps requires solid MLOps foundations.”summary, stated

Why this architecture

Ramp's architecture is platform-shaped because multiple financial workflows need consistent, secure LLM access to company data and retrieval systems. Reusable infrastructure such as MCP, embeddings, and ML configuration lets one investment support several product applications.

“Ramp is not just deploying individual LLM features but building foundational infrastructure that can support multiple LLM-powered applications.”summary, stated

“This portfolio approach to LLMOps can accelerate learning and allow infrastructure investments to pay off across multiple applications.”summary, stated

Models used: LLMs · RAG systems · Transaction embeddings · MCP server

Scale

Ramp Research handles over 1,000 data questions per month.
“this AI agent handles over 1,000 data questions per month.”summary, stated
The merchant matching system resolves issues in under 10 seconds.
“resolving merchant matching issues in under 10 seconds using smart RAG and LLMs”summary, stated

Alternatives considered (roads not taken)

Rely on LLMs alone for merchant and industry knowledge

Rejected because: Ramp combines LLMs with retrieval over current company data and examples.

“This approach addresses a key limitation of pure LLM solutions (lack of access to current merchant databases)”summary, stated

Build one-off LLM integrations per feature

Rejected because: Ramp invested in MCP and ML infrastructure for reusable access and deployment patterns.

“building reusable infrastructure (MCP server) rather than one-off solutions”summary, stated

Patterns

`rag-retrieval` (retrieval)

Ramp uses RAG for merchant matching and industry classification so LLM decisions are grounded in current records, examples, and definitions.

“The system uses what they describe as "smart RAG" (Retrieval-Augmented Generation) combined with LLMs to resolve merchant matching problems in under 10 seconds.”summary, stated

“Using RAG for classification tasks is an interesting architectural choice. It suggests they may be retrieving relevant examples or industry definitions to augment the classification decision, rather than relying solely on a fine-tuned model or pure prompt engineering.”summary, stated

`tool-use-orchestration` (integration)

Ramp's MCP server provides structured tool and resource access for multiple LLM applications.

“In March 2025, Ramp documented their work building an MCP (Model Context Protocol) server that allows LLMs to interact with their resources. MCP is an emerging standard for enabling LLMs to access external tools and data sources in a structured way.”summary, stated

“This approach enables consistent access patterns for LLMs across different use cases, centralized management of permissions and security, and the ability to evolve LLM capabilities without modifying individual applications.”summary, stated

`agent-memory` (data analysis)

The data analyst use case implies ongoing schema/data context management for repeated data questions.

“Building an agent that can reliably answer data questions at this scale presents significant LLMOps challenges including ensuring accuracy and preventing hallucinations”summary, implied

`retrieval-ranking` (embeddings)

Transaction embeddings improve retrieval for accounting, merchant, category, and historical-pattern matching.

“These embeddings likely power the retrieval components of their RAG systems, enabling more accurate matching of transactions to categories, merchants, or historical patterns.”summary, stated

`safety-policy-enforcement` (financial automation)

Expense approval agents require transparency, edge-case handling, audit trails, and escalation because they make financial decisions.

“Building trust in AI agents for financial automation requires addressing concerns about transparency in decision-making, handling edge cases appropriately (escalating to humans when uncertain), maintaining audit trails”summary, stated

Tradeoffs

Reusable LLM infrastructure

Chose Build MCP and shared ML infrastructure over Independent point integrations.

Rationale: Shared infrastructure supports multiple LLM-powered financial workflows consistently.

“centralized management of permissions and security, and the ability to evolve LLM capabilities without modifying individual applications.”summary, stated

RAG for classification

Chose Retrieve examples or definitions to support classification over Pure fine-tuned or pure prompt-only classification.

Rationale: RAG can adapt to new industries or changing definitions without full retraining.

“This approach can offer better adaptability to new industries or changing definitions without full model retraining.”summary, stated

Gotchas (what bit them)

Portfolio summaries can overstate technical certainty.

Root cause: The source aggregates company blog titles and brief descriptions rather than full implementation details.

Resolution: Treat claims as company-published and avoid inferring internal mechanisms not described.

“While the full technical details of each implementation are not provided in the excerpts, the titles and summaries reveal a comprehensive approach to LLMOps across multiple use cases.”summary, stated

“It's important to note that these are company-published materials, so claims should be considered in that context”summary, stated

Financial LLM agents need trust and auditability.

Root cause: Expense approvals and finance decisions carry real monetary and compliance consequences.

Resolution: Ramp explicitly frames trustworthy agents around transparency, edge cases, audit trails, and escalation.

“This is a particularly sensitive application of LLMs in production because it involves financial decisions with real monetary implications.”summary, stated

“maintaining audit trails for compliance, and managing the consequences of errors.”summary, stated

RAG classification depends on retrieval quality.

Root cause: Industry classification and merchant matching need relevant examples or records before the LLM can make a grounded decision.

Resolution: Ramp invests in embeddings and retrieval systems alongside LLM reasoning.

“It suggests they may be retrieving relevant examples or industry definitions to augment the classification decision, rather than relying solely on a fine-tuned model or pure prompt engineering.”summary, stated

Evaluation

The source reports production metrics for selected applications but does not provide full evaluation methodology for each system.

Metrics: 1,000+ data questions per month · Under 10 seconds merchant matching · State-of-the-art industry classification claimed but not detailed

“The 10-second resolution time indicates attention to production performance requirements. Merchant matching is inherently a retrieval-heavy problem—the system must search through potentially millions of merchant records to find the correct match.”summary, stated

“we should be cautious about specific claims like "state-of-the-art" industry classification or the exact performance metrics mentioned.”summary, stated

Outcomes

Ramp reports an agentic data analyst handling over 1,000 data questions monthly.
Ramp reports smart-RAG merchant matching resolving issues in under 10 seconds.
Ramp has invested in MCP, embeddings, RAG, and ML infrastructure for multiple LLM applications.
The public source supports portfolio-level conclusions rather than detailed per-system outcome claims.

The strongest quantitative claims are over 1,000 data questions per month and under-10-second merchant-matching resolution.

“The breadth of their LLM applications—from data analysis to merchant matching to expense approval automation—also suggests a systematic approach”summary, stated