Building Trustworthy AI Agents for Automated Expense Management

ramp · Finance · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: finance workflow automation

Ramp needed LLM-backed expense-management agents that could automate high-volume ambiguous finance decisions without eroding user trust through unexplained, overconfident, or policy-ungrounded outputs.

Constraints:

• Finance users lose trust quickly when outputs are low-quality or unexpected.
• Expense approval involves ambiguous policy interpretation where simple heuristics are insufficient.
• The system must ground decisions in user-provided product context and expense-policy sections.
• Uncertain cases need a valid path back to human review rather than forced decisions.
• Customers need configurable autonomy because risk tolerance varies by company and workflow.

Architecture

Ramp built LLM-backed agents around a policy agent that receives expense and policy context, produces categorical decisions with reasoning and citations, defers uncertain cases to existing human review, lets users edit policies inside Ramp, and uses workflow-builder controls plus deterministic guardrails to define when agents may act autonomously.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  expense-intake["expense intake"] --> policy-agent["policy agent"]
  policy-agent --> policy-context-retriever["policy context retriever"]
  policy-context-retriever --> policy-editor["policy editor"]
  policy-agent --> reasoning-and-citation-renderer["reasoning and citation renderer"]
  reasoning-and-citation-renderer --> finance-users["finance users"]
  policy-agent --> uncertainty-escape-hatch["uncertainty escape hatch"]
  uncertainty-escape-hatch --> human-review-queue["human review queue"]
  human-review-queue --> finance-users
  policy-agent --> workflow-builder-controls["workflow builder controls"]
  workflow-builder-controls --> deterministic-guardrails["deterministic guardrails"]
  finance-users --> workflow-builder-controls
  finance-users --> policy-editor
  finance-users --> user-feedback-loop["user feedback loop"]
  user-feedback-loop --> policy-editor
  user-feedback-loop --> evaluation-dataset["evaluation dataset"]
  evaluation-dataset --> policy-agent

Components

Data flow

An expense enters Ramp, relevant policy context is retrieved from the in-platform policy editor, the policy agent decides Approve/Reject/Needs review with reasoning and citations, deterministic workflow rules enforce dollar/vendor/category hard stops, unsure cases fall back to human escalation, and user corrections or policy edits improve future context.

Why this architecture

Ramp combined transparent reasoning, citations, categorical uncertainty, collaborative policy context, user-configurable autonomy, deterministic rules, and evals because trust in finance depends on controllability and auditability as much as raw model accuracy.

Models used: LLMs

Scale

• The policy agent handles more than 65% of expense approvals autonomously at Ramp.
  “Since enabling the policy agent at Ramp, we’ve seen more than 65% of approvals be fully handled by the agent.”article, stated
• Ramp explicitly targeted high-volume finance tasks such as approving expenses, merchant identification, and receipt parsing.
  3 supporting quotes

  “Approving expenses”article, stated

  “Identifying credit card merchants”article, stated

  “Parsing through receipts”article, stated
• A conservative autonomy setting can require human review on every expense above $50.
  “The more conservative end of the autonomy slider, requiring human review on every expense above $50.”article, stated

Alternatives considered (roads not taken)

Force the LLM to answer every approval question.

Rejected because: Ramp treats uncertainty as a valid state and escalates uncertain decisions instead of forcing hallucinated answers.

Use numeric LLM confidence scores.

Rejected because: Ramp found LLM confidence numbers misleading and not reproducible, so it uses categorical actions.

Treat context as a static prompt input.

Rejected because: Ramp makes context collaborative so customers can refine policies when agent reasoning exposes ambiguity.

Jump directly to full autonomy.

Rejected because: Ramp first used suggestion mode, then subsets, then full autonomy to build user trust.

Patterns

structured-output-contract (decisioning)

The agent emits actionable categories rather than free-form confidence: Approve, Reject, or Needs review.

human-in-the-loop-gate (control)

Uncertain or user-configured cases route to human review instead of autonomous action.

“When the agent isn’t sure, we fall back to the pre-agent escalation process users are accustomed to.”article, stated

“These guardrails aren't just safety nets, they're how users tell the agent "I'll never be comfortable with you touching this.” The more conservative end of the autonomy slider, requiring human review on every expense above $50. !”article, stated

safety-policy-enforcement (guardrails)

The workflow builder layers deterministic limits, vendor blocklists, and category restrictions over the LLM decision.

observability-tracing (llmops)

Reasoning output and unsure reasons become observability signals for prompt, context, and system improvements.

eval-driven-development (evaluation)

Ramp treats evals as unit tests, grows them from user-flagged errors, and uses golden datasets to avoid learning user leniency.

Tradeoffs

Return categorical decisions instead of confidence scores.

Chose Approve, Reject, Needs review over Numeric model confidence.

Rationale: Categories are more explainable, actionable, and less misleading than uncalibrated LLM confidence numbers.

Make context collaborative inside Ramp.

Chose In-platform editable policy context over External static policy PDFs.

Rationale: Editable policy context lets users fix ambiguity at the source, improving both human and agent decisions.

Let customers tune autonomy.

Chose Workflow-builder autonomy slider plus hard stops over One-size-fits-all AI autonomy.

Rationale: Different customers have different comfort levels, and some decisions are better enforced by deterministic rules than LLM judgment.

Roll out autonomy progressively.

Chose Suggestions, then subsets, then full autonomy over Immediate autonomous action.

Rationale: Suggestion mode let users learn patterns, catch mistakes, and build confidence before allowing real action.

Gotchas (what bit them)

LLM confidence scores create false precision.

Root cause: LLMs readily output numerical confidence, often clustering around 70-80% without statistical calibration.

Resolution: Use actionable categories rather than numeric confidence.

User actions are not always ground truth.

Root cause: Finance teams may approve reasonable but technically out-of-policy expenses, which would teach an overly lenient agent.

Resolution: Create golden datasets reviewed against information available in the system.

Reasoning without citations can hallucinate.

Root cause: LLM-generated explanations may invent facts unless grounded in product or user context.

Resolution: Link facts and policy references directly to relevant expense-policy sections.

Ambiguous policy harms both the agent and humans.

Root cause: If an LLM is confused by policy wording, a human reviewer may be confused too.

Resolution: Expose policy editing in-platform so disagreements clarify source context.

Evaluation

Ramp treats evals as unit tests, starts simple, prioritizes ambiguous edge cases, turns production failures into eval cases, and maintains golden datasets reviewed by internal teams rather than blindly treating finance-team actions as ground truth.

Metrics: More than 65% of approvals fully handled by the policy agent · Unsure reasons tracked over time · User-flagged errors converted into eval candidates · Golden datasets reviewed by Ramp's team · No public false positive or false negative rates disclosed

Outcomes

• The policy agent handles more than 65% of approvals fully autonomously at Ramp.
• Ramp created a trust-oriented agent design around reasoning, citations, escape hatches, collaborative context, autonomy controls, and evals.
• The approach supports a broader suite of agents for expense approval, merchant identification, and receipt parsing.

The source reports more than 65% autonomous approval handling and examples of guardrail thresholds such as human review above $50, but does not disclose accuracy, override, latency, or cost metrics.