Building Trustworthy LLM Agents for Automated Expense Management

ramp · Finance · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: expense-management agents

Ramp needed LLM agents for finance workflows that could make ambiguous policy decisions at high volume while remaining explainable, controllable, and safe enough for users to trust.

Constraints:

• The domain is finance, where low-quality or unexpected outputs quickly damage trust.
• Expense approval is ambiguous enough that simple heuristics are insufficient.
• The agent must cite user policy context rather than relying on ungrounded reasoning.
• Uncertain cases must be escalated as valid outcomes.
• Autonomy must be configurable with deterministic hard stops.

Architecture

Ramp's policy agent combines retrieved expense-policy context, LLM reasoning, source citations, categorical decisions, an uncertainty escape hatch, workflow-builder autonomy controls, deterministic guardrails, collaborative policy editing, and eval datasets built from edge cases and user-flagged failures.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  expense-approval-workflow["expense approval workflow"] --> policy-agent["policy agent"]
  policy-agent --> policy-context["policy context"]
  policy-context --> policy-editor["policy editor"]
  policy-agent --> citation-renderer["citation renderer"]
  citation-renderer --> finance-users["finance users"]
  policy-agent --> uncertainty-router["uncertainty router"]
  uncertainty-router --> human-review["human review"]
  human-review --> finance-users
  policy-agent --> workflow-builder["workflow builder"]
  workflow-builder --> deterministic-rules["deterministic rules"]
  workflow-builder --> finance-users
  finance-users --> feedback-loop["feedback loop"]
  feedback-loop --> policy-editor
  feedback-loop --> eval-suite["eval suite"]
  eval-suite --> policy-agent

Components

Data flow

An expense is evaluated against relevant policy sections; the LLM emits reasoning and an Approve/Reject/Needs review decision; citations point users to the policy context; workflow rules and guardrails define where action is allowed; uncertain cases go to human review; feedback and policy edits feed evals and context improvements.

Why this architecture

The system is built around trust rather than raw autonomy: reasoning and citations make decisions auditable, escape hatches prevent forced hallucination, collaborative policy editing improves context, and workflow controls let customers decide how much autonomy fits their risk tolerance.

Models used: LLMs

Scale

• The policy agent handles more than 65% of approvals fully.
  “Since enabling the policy agent at Ramp, we’ve seen more than 65% of approvals be fully handled by the agent.”article, stated
• The conservative autonomy example requires human review above $50.
  “The more conservative end of the autonomy slider, requiring human review on every expense above $50.”article, stated
• Ramp targets high-volume finance tasks including expense approval, merchant identification, and receipt parsing.
  “These all manifest as tedious time sinks for users with low chance of catastrophic failure”article, stated

Alternatives considered (roads not taken)

Ask the LLM for numerical confidence.

Rejected because: Ramp rejects this because LLM confidence scores are not numerically meaningful or reproducible.

Force an answer when the model is unsure.

Rejected because: Ramp treats uncertainty as a valid product state and routes to human escalation.

Keep policies outside the platform as static PDFs.

Rejected because: Ramp brought policies into the platform and built an editor so context stays current and user-correctable.

Use user actions directly as ground truth.

Rejected because: Finance teams can be lenient, so Ramp created reviewed golden datasets to avoid teaching the agent excessive leniency.

Patterns

structured-output-contract (decisioning)

The policy agent uses predefined categories for decisions rather than numerical confidence.

human-in-the-loop-gate (control)

The system escalates uncertain or policy-restricted expenses to humans.

guardrails-io-filtering (safety)

Ramp layers deterministic limits and blocklists over the agent so some actions are categorically unavailable.

eval-driven-development (evaluation)

Ramp uses evals as regression tests, especially around ambiguous edge cases and production failures.

Tradeoffs

Use categorical uncertainty instead of confidence scores.

Chose Approve/Reject/Needs review over Numerical confidence.

Rationale: The categories are more actionable and avoid false precision.

Use the existing workflow builder for autonomy.

Chose Workflow-builder controls over A separate one-size-fits-all AI control surface.

Rationale: Customers already customize processes there, and it can define exactly where agents may act.

Introduce autonomy gradually.

Chose Suggestions before autonomous actions over Immediate full-agent behavior.

Rationale: Progressive rollout let users learn the agent's patterns and build confidence.

Gotchas (what bit them)

Confidence numbers from LLMs are misleading.

Root cause: LLMs provide plausible-looking percentages without calibration or reproducibility.

Resolution: Use categorical outcomes that map directly to user actions.

Reasoning can build trust only if users can audit it.

Root cause: LLM reasoning alone can be flawed or hallucinated.

Resolution: Cite product or user-provided policy context for facts and figures.

Production feedback can encode human leniency.

Root cause: Finance teams sometimes approve reasonable but technically noncompliant expenses.

Resolution: Use curated golden datasets rather than raw user actions as objective labels.

Evaluation

Ramp runs evals like unit tests: start small, focus on edge cases, turn user failures into tests, and verify feedback labels through curated golden datasets.

Metrics: More than 65% autonomous approval handling · Unsure reasons tracked over time · User-flagged failures become eval candidates · Golden datasets reviewed by Ramp's team

Outcomes

• The policy agent handles more than 65% of approvals fully.
• Ramp established reusable production patterns for explainability, citations, uncertainty, autonomy controls, and evals.
• The same source describes agents across expense approval, merchant identification, and receipt parsing.

The article discloses the 65% approval automation figure and a $50 conservative review threshold, but not error rates, costs, or latency.