Specialized Retrieval Subagent with Reinforcement Learning Post-Training for Spreadsheet Navigation

ramp · Finance · 2026 · source Agentic: yes · promotional bias: low · summary-only (no fetched source) · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Spreadsheet-agent retrieval and model specialization

Ramp needed to reduce retrieval inefficiency inside Ramp Sheets because the main spreadsheet agent spent a large share of tool calls navigating sheets, reading ranges, and retrying failed reads instead of solving user tasks.

Constraints:

• The main agent spent 17.8% of all tool calls on navigation and data retrieval.
• About 75% of retrieval calls were followed by another read call, implying failed first-pass retrieval.
• Spreadsheet retrieval had to be fast enough for synchronous user-facing interactions.
• Training required a verifiable reward signal because the target task was exact-value spreadsheet retrieval.

Architecture

Ramp carved spreadsheet retrieval out of the main Ramp Sheets agent and trained Fast Ask, a narrow Qwen 3.5-35B-A3B retrieval subagent, using RL on synthetic spreadsheet workbooks. The subagent uses a small tool interface to inspect workbook metadata, read ranges, run sandboxed Python, and return exact answers, letting frontier models spend tokens on judgment rather than navigation.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  ramp_sheets_main_agent["Ramp Sheets main agent"] --> fast_ask_retrieval_subagent["Fast Ask retrieval subagent"]
  fast_ask_retrieval_subagent --> workbook_metadata_tool["Workbook metadata tool"]
  fast_ask_retrieval_subagent --> range_reader_tool["Range reader tool"]
  fast_ask_retrieval_subagent --> sandboxed_python_tool["Sandboxed Python tool"]
  synthetic_workbook_environment["Synthetic workbook environment"] --> fast_ask_retrieval_subagent
  synthetic_workbook_environment --> deterministic_reward_verifier["Deterministic reward verifier"]
  deterministic_reward_verifier --> prime_intellect_training_stack["Prime Intellect training stack"]

Components

Data flow

A spreadsheet question is routed to Fast Ask. The retrieval subagent inspects workbook metadata, reads bounded ranges, optionally runs sandboxed Python, and emits a parseable final answer. During training, synthetic workbooks with ground-truth answers score complete tool-use trajectories with deterministic correctness, efficiency, and concision rewards.

Why this architecture

Ramp specialized retrieval because the task was narrow, verifiable, latency-sensitive, and responsible for a measurable production bottleneck. Exact-answer rewards let them train a smaller subagent while preserving frontier-model capacity for judgment-heavy work.

Models used: Qwen 3.5-35B-A3B · Claude Opus 4.6 · Claude Haiku 4.5

Scale

• The main spreadsheet agent spent 17.8% of all tool calls on navigation and retrieval.
  “they discovered that the main agent spent 17.8% of all tool calls on navigation and data retrieval activities”summary, stated
• About 75% of retrieval calls were immediately followed by another read call.
  “about 75% of these retrieval calls were immediately followed by another read call, indicating that the agent frequently failed to retrieve the correct information on the first attempt”summary, stated
• The training run used 100 steps, batch size 256, and eight rollouts per example.
  “The training run used 100 training steps with a batch size of 256 and 8 rollouts per example”summary, stated
• Fast Ask beat Claude Opus 4.6 by 4 percentage points while running at Haiku 4.5 latency.
  “The trained model beat Claude Opus 4.6 (62% accuracy) by 4 percentage points while running at Haiku 4.5 latency.”summary, stated

Alternatives considered (roads not taken)

Prompting the main general-purpose spreadsheet agent harder

Rejected because: Ramp chose a separate specialist because prompting would still leave a large, slower general-purpose model responsible for navigation.

Human demonstrations or real production data for training

Rejected because: Synthetic workbooks provided exact ground truth, controlled task variation, and scalable verifiable training examples.

Large frontier models for the retrieval bottleneck

Rejected because: The trained smaller specialist outperformed Claude Opus 4.6 on exact-match accuracy while running at Haiku-class latency.

Patterns

tool-use-orchestration (agent architecture)

Fast Ask learns spreadsheet retrieval through a constrained three-tool interface and bounded turn budget rather than unrestricted workbook exploration.

synthetic-data-generation (training data)

Ramp generated synthetic business workbooks with ground-truth answers to scale verifiable retrieval training without human demonstrations.

eval-driven-development (evaluation)

The retrieval subagent was trained and evaluated with exact-match rewards and held-out examples aligned to the production retrieval task.

latency-optimization (serving)

Ramp optimized a small specialist to preserve production latency while reducing costly, repeated spreadsheet reads by the main agent.

fine-tuning-adaptation (model adaptation)

The Qwen base model was post-trained with RL for a narrow spreadsheet-navigation task.

Tradeoffs

Train a narrow retrieval subagent.

Chose A specialized RL-trained Qwen model for Fast Ask over Leaving retrieval inside the main general-purpose agent.

Rationale: The specialist targets a measurable latency and cost bottleneck while freeing frontier models for judgment-heavy work.

Use deterministic exact-match reward.

Chose Correctness-dominant RL reward with small efficiency and concision terms over Subjective human or LLM-as-judge trajectory scoring.

Rationale: Retrieval answers are verifiable, so deterministic rewards avoid noisy subjective labels while still shaping efficient behavior.

Use synthetic workbook tasks for RL.

Chose Generated business workbooks with controlled task types and exact answers over Human demonstrations or real production-data collection.

Rationale: Synthetic data allowed scale, reliability, task variation, and exact-answer control, but leaves some uncertainty about real-world generalization.

Gotchas (what bit them)

General agents can burn tool calls on navigation instead of task progress.

Root cause: Spreadsheet retrieval caused repeated reads and inefficient exploration inside the main agent workflow.

Resolution: Split the bottleneck into a specialized retrieval subagent trained to return the relevant cells or computed values.

Synthetic retrieval tasks can be too clean unless they include production-like ambiguity.

Root cause: Clean workbooks do not force models to handle distractors, unreliable shortcuts, or ambiguous identifiers.

Resolution: Ramp added difficulty levels with decoy sheets, incomplete helper summaries, hard cases, and obfuscated invoice references.

Tool-use RL can be slowed by rollout generation.

Root cause: Agent rollouts involve multi-turn metadata reads, range reads, Python execution, and final answers, making rollout generation slower than supervised data loading.

Resolution: Use asynchronous off-policy training so rollout workers continue generating trajectories while the trainer updates the model.

Evaluation

Ramp evaluated the base model, trained Fast Ask model, and Claude-family models on a held-out exact-match retrieval task set with wall-clock rollout time.

Metrics: Base Qwen3.5-35B-A3B accuracy: 56%. · Trained Fast Ask accuracy: 66%. · Claude Opus 4.6 accuracy: 62%. · Training took approximately 26 hours.

Outcomes

• Fast Ask improved exact-match accuracy by 10 percentage points over the base Qwen model.
• The trained specialist beat Claude Opus 4.6 by 4 percentage points while running at Haiku-class latency.
• Training reduced average completion time rather than increasing it.

The summary reports a 26-hour RL run, 100 training steps, batch size 256, eight rollouts per example, and 128-example held-out evaluations every 20 steps.