AI-Powered Tour Guide for Financial Platform Navigation

ramp · Finance · 2024 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Financial platform navigation

Ramp needed to help users navigate a complex financial operations product without forcing them into risky full automation or making support teams answer routine navigation questions.

Constraints:

• financial workflows are sensitive enough that users must stay able to observe and override actions
• web UI actions have to be grounded in the current application state
• model inputs need to be compact enough for interactive latency
• the agent must avoid high-risk or unsupported pages
• the case study does not disclose quantitative accuracy or support-deflection metrics

Architecture

Ramp built Tour Guide as a UI-navigation agent that turns the current web session into one constrained next action at a time. The system annotates interactive HTML elements, incorporates accessibility descriptions, simplifies the DOM, labels a fixed option set, and prompts the model to choose the next scroll, click, or text-fill action. A routing layer decides when Tour Guide should be invoked, while guardrails restrict operation on high-risk pages and leave users in a visible, interruptible loop.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  tour-guide-router["tour-guide-router"] --> prompt-builder["prompt-builder"]
  dom-annotation-script["dom-annotation-script"] --> prompt-builder["prompt-builder"]
  prompt-builder["prompt-builder"] --> single-call-action-agent["single-call-action-agent"]
  single-call-action-agent["single-call-action-agent"] --> ui-action-executor["ui-action-executor"]
  single-call-action-agent["single-call-action-agent"] --> guardrail-layer["guardrail-layer"]
  ui-action-executor["ui-action-executor"] --> prompt-builder["prompt-builder"]
  ui-action-executor["ui-action-executor"] --> human-user["human-user"]
  guardrail-layer["guardrail-layer"] --> human-user["human-user"]

Components

Data flow

A user asks for help in the Ramp product. A classifier routes relevant requests to Tour Guide. The current screen, annotated DOM, accessibility tags, and fixed letter-labeled interactable choices are packaged into the prompt. The model chooses one next action, the UI executes it, and the changed session state becomes the input for the next step until the user stops or the task is complete.

Why this architecture

Ramp favored an iterative, constrained action loop because it made UI automation safer and more predictable than full-plan generation. The team collapsed an earlier two-call plan-and-ground design into a single call to reduce latency, then compensated by simplifying the DOM and forcing the model to choose from explicit options. Guardrails and visible user control were especially important because the product handles finance operations.

Models used: LLM · Vimium-style element labeling · DOM accessibility tags

Scale

• The prompt labels a fixed set of options with letters from A to Z.
  “Instead of letting the model pick from a lengthy list of interactable elements, they found that labeling a fixed set in the prompt with letters (A to Z) made it clear to the model what options were available to process.”article, stated

Alternatives considered (roads not taken)

Full workflow planning up front

Rejected because: Planning entire tours would require many sequential actions and create more error propagation than generating one verifiable next action at a time.

Separate planning and grounding LLM calls

Rejected because: The two-call design improved accuracy but made the interactive experience too slow.

Context stuffing

Rejected because: Extra context increased prompt length and latency; enriched but concise interactions worked better.

Patterns

tool-use-orchestration (agent architecture)

The agent reduces browser navigation to a constrained tool-like action space of scrolling, clicking, and filling text.

latency-optimization (production)

Ramp collapsed separate planning and grounding calls into one LLM call because user-facing navigation could not tolerate the extra latency.

guardrails-io-filtering (safety)

The system limits where the agent can act, avoiding high-risk pages and UI states likely to fail.

human-in-the-loop-gate (safety)

Tour Guide is designed as visible assistance where users can watch, pause, or override the agent rather than surrendering full control.

Tradeoffs

Use a single action-generation call.

Chose One LLM call for planning plus action generation over Separate planning and grounding calls.

Rationale: Latency mattered more for a guided UI experience than the extra accuracy from splitting the steps.

Constrain the visible action space.

Chose Simplified DOM plus fixed labeled options over Letting the model choose from a long list of interactable elements.

Rationale: A smaller, explicit action space improved accuracy and made the model's decision more controllable.

Keep the user in control instead of pursuing full automation.

Chose Visible guided assistance over Full automation of financial product workflows.

Rationale: Finance users need trust and the ability to intervene when an agent is manipulating product state.

Gotchas (what bit them)

Large DOMs and many similar interactable elements degrade model choices.

Root cause: LLMs struggle to choose the best option among many similar UI elements.

Resolution: Prune irrelevant DOM objects and label a fixed set of interactable choices with letters.

Better separated reasoning can be too slow for interactive UI help.

Root cause: The initial two-step planning/grounding design added latency even though it improved accuracy.

Resolution: Use one prompt that combines planning and action generation, then constrain the input space more aggressively.

The case lacks disclosed automated evaluation and quantitative outcome metrics.

Root cause: The public writeup emphasizes qualitative iteration, prompt optimization, and UX design rather than benchmark numbers.

Resolution: Treat the implementation as a useful architecture pattern but discount claims about impact until adoption or accuracy data is available.

Evaluation

The public case describes iterative prompt, DOM, and UX refinement rather than a disclosed benchmark pipeline. Evaluation appears to rely on whether the constrained inputs improve action accuracy and whether the system stays responsive enough for interactive use.

Metrics: output accuracy · latency · prompt length · support-call reduction not quantified

Outcomes

• Tour Guide provides step-by-step AI navigation assistance inside Ramp's financial operations product.
• DOM simplification and fixed option labels reportedly improved output accuracy.
• The public writeup does not disclose adoption, accuracy, support-deflection, or revenue metrics.

The only concrete scale detail in the fetched source is the fixed A-to-Z option labeling; product-wide adoption and traffic are not disclosed.