Unified Pipeline for AI Token Spend Management and Attribution

ramp · Finance · 2026 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: AI token spend observability

Ramp needed real-time visibility and attribution for volatile AI token spend across teams, customers, models, use cases, and runaway production behaviors.

Constraints:

• AI usage events are bursty and can reach tens of thousands of events per minute.
• Costs need high-precision per-token accounting.
• Events must be tenant-isolated and idempotent.
• Customers need attribution by user, team, project, model, and use case.
• LLMOps issues such as phantom reasoning tokens, prompt bloat, missing caching, and agent loops affect both latency and cost.

Architecture

Ramp built a streaming token-spend pipeline with webhook ingestion from LiteLLM and OpenRouter, OAuth/RLS/IP/audit validation, Kafka buffering, ClickHouse ReplacingMergeTree storage, and dashboards for multi-dimensional attribution, forecasting, and anomaly investigation.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  ai_gateways["AI gateways"] --> webhook_ingestion_endpoints["Webhook ingestion endpoints"]
  validation_decorator_stack["Validation decorator stack"] --> webhook_ingestion_endpoints
  webhook_ingestion_endpoints --> kafka_event_stream["Kafka event stream"]
  kafka_event_stream --> clickhouse_sink_connector["ClickHouse sink connector"]
  clickhouse_sink_connector --> clickhouse_usage_table["ClickHouse usage table"]
  clickhouse_usage_table --> attribution_dashboards["Attribution dashboards"]
  attribution_dashboards --> forecasting_and_anomaly_workflows["Forecasting and anomaly workflows"]

Components

Data flow

LiteLLM and OpenRouter gateways send usage events to Ramp webhooks. Decorators validate OAuth2, establish tenant-scoped RLS, check IP whitelists, and write audits. Events stream through Kafka for durability and burst absorption, sink into ClickHouse with deduplication, and dashboards aggregate cost by dimensions such as model, team, user, project, and use case.

Why this architecture

Ramp used a streaming analytics pipeline because AI costs are high-volume, bursty, tenant-sensitive, and operationally coupled to latency; invoices are too late to diagnose token-level regressions.

Models used: LiteLLM · OpenRouter · Gemini

Scale

• Nearly half of businesses on Ramp's platform pay for at least one AI model provider.
  “nearly half of businesses on their platform now pay for at least one AI model provider”summary, stated
• Average monthly AI token spend across customers increased 13x.
  “average monthly AI token spend across customers has increased 13x.”summary, stated
• A single customer can generate tens of thousands of events per minute.
  “A single customer might generate tens of thousands of events per minute during batch jobs or when autonomous agents enter processing loops.”summary, stated
• Cost fields use Decimal(20,10) precision.
  “The schema uses Decimal(20,10) precision for cost fields”summary, stated

Alternatives considered (roads not taken)

Wait for invoices or aggregate billing reports

Rejected because: Token-level latency and cost regressions remain invisible until too late.

Write webhooks directly to ClickHouse

Rejected because: Kafka absorbs bursty event spikes and queues during ClickHouse outages.

Use lower-precision cost fields

Rejected because: Small per-token rounding errors compound over millions of events.

Patterns

observability-tracing (cost telemetry)

Ramp collects token-level usage events and attributes spend by model, team, user, project, and use case.

cost-aware-routing (AI spend management)

The system identifies oversized models, missing caching, prompt bloat, and runaway loops that inflate spend.

async-message-orchestration (streaming pipeline)

Kafka decouples bursty webhook ingestion from storage and provides replayability.

privacy-preserving-processing (tenant isolation)

OAuth, RLS, IP whitelists, and audit logs enforce multi-tenant access controls.

Tradeoffs

Use webhook ingestion from gateways.

Chose LiteLLM/OpenRouter webhooks over Heavy application-code instrumentation.

Rationale: Customers can configure gateways to send events with minimal code changes.

Use ClickHouse with ReplacingMergeTree.

Chose Columnar analytics with storage-level deduplication over Application-managed deduplication and slower aggregate queries.

Rationale: This supports fast multi-dimensional cost analytics and exactly-once convergence after replay.

Gotchas (what bit them)

Model package upgrades can silently change token behavior.

Root cause: A LiteLLM upgrade caused Gemini to emit phantom reasoning tokens by default.

Resolution: Track token-level usage and latency so package-induced regressions surface immediately.

AI usage is bursty and can overwhelm direct ingestion.

Root cause: Batch jobs or runaway agents can emit tens of thousands of events per minute.

Resolution: Buffer through Kafka before ClickHouse storage.

Token-cost rounding errors compound at scale.

Root cause: Per-token prices are tiny but applied across millions of events.

Resolution: Use Decimal(20,10) precision for cost fields.

Evaluation

Ramp evaluates the pipeline by anomaly detection, latency/cost incident diagnosis, forecasting, multi-dimensional attribution, and fast aggregate query performance.

Metrics: 13x increase in average monthly AI token spend. · Nearly half of businesses paying at least one AI model provider. · Tens of thousands of events per minute burst capacity. · Month of usage by model/team computed in milliseconds. · Decimal(20,10) cost precision.

Outcomes

• Ramp diagnosed phantom reasoning tokens causing latency and cost spikes.
• The system provides real-time attribution and forecasting for AI token spend.
• Customers can detect oversized models, missing caching, runaway loops, and prompt bloat.

The pipeline is designed for bursty usage up to tens of thousands of events per minute per customer.