AI-Powered GTM Data Platform with Agentic Operations

rippling · HR · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.0 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Go-to-market data infrastructure for agentic operations

Rippling needed a trustworthy GTM data foundation that could unify messy third-party identities, support AI/ML workloads, and serve low-latency answers to both humans and agents.

Constraints:

Legacy data models were unreliable for agentic workloads
Entity resolution across hundreds of millions of heterogeneous records is computationally expensive
Sales-conversation retrieval needs both semantic concepts and exact business terms

Architecture

Rippling rebuilt GTM data on a Databricks lakehouse so governed data, hosted model inference, entity resolution, real-time natural-language access, and semantic retrieval sit on one source of truth. Incremental ingestion lands raw data in Bronze, ML entity resolution runs in Silver with candidate blocking, supervised pairwise classification, confidence scores, and audit trails, and business-ready Gold tables feed AI/ML pipelines plus a Genie API interface for people and agents. A separate sales-conversation retrieval pipeline incrementally processes new/changed calls, labels speaker sides, chunks conversational turns, enriches buyer evidence, embeds enriched text, combines dense and keyword retrieval, and joins lakehouse business context after retrieval.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  databricks-lakehouse[Databricks lakehouse] --> bronze-ingestion[Bronze ingestion]
  bronze-ingestion --> silver-entity-resolution[Silver entity resolution]
  silver-entity-resolution --> blocking-tree[Blocking tree]
  blocking-tree --> pairwise-classifier[Pairwise classifier]
  silver-entity-resolution --> entity-audit-trail[Entity audit trail]
  silver-entity-resolution --> gold-business-tables[Gold business tables]
  databricks-lakehouse --> hosted-foundation-models[Hosted foundation models]
  gold-business-tables --> genie-api[Genie API]
  genie-api --> growthos-agent-harness[GrowthOS agent harness]
  conversation-delta-worklist[Conversation delta worklist] --> conversation-retrieval-agent[Conversation retrieval agent]
  conversation-retrieval-agent --> speaker-labeler[Speaker labeler]
  speaker-labeler --> chunk-enricher[Chunk enricher]
  chunk-enricher --> vector-search-index[Vector search index]
  vector-search-index --> conversation-retrieval-agent
  gold-business-tables --> conversation-retrieval-agent
  conversation-retrieval-agent --> growthos-agent-harness

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`databricks-lakehouse`	Unified GTM data foundation co-locating governed data, AI/ML pipelines, hosted model access, and serving layers.	`bronze-ingestion`, `silver-entity-resolution`, `gold-business-tables`, `hosted-foundation-models`
`bronze-ingestion`	Incremental raw ingestion layer for third-party SaaS exports and internal GTM data.	`databricks-lakehouse`, `silver-entity-resolution`
`silver-entity-resolution`	ML-based entity resolution layer that reconciles company/person records across heterogeneous sources.	`bronze-ingestion`, `blocking-tree`, `pairwise-classifier`, `entity-audit-trail`, `gold-business-tables`
`blocking-tree`	Candidate-reduction structure trained to avoid n-squared brute-force comparison at hundreds-of-millions scale.	`silver-entity-resolution`, `pairwise-classifier`
`pairwise-classifier`	Supervised model scoring whether candidate records represent the same entity and supporting match-type classification.	`blocking-tree`, `silver-entity-resolution`
`entity-audit-trail`	Confidence score and ranked source provenance for every resolved canonical record, including conflicts.	`silver-entity-resolution`, `gold-business-tables`
`gold-business-tables`	Conformed business-ready layer consumed by teams, AI/ML pipelines, Genie, analytics joins, and agents.	`silver-entity-resolution`, `genie-api`, `conversation-retrieval-agent`
`hosted-foundation-models`	Managed model access inside the data platform for summarization, classification, and AI/ML pipeline work.	`databricks-lakehouse`, `gold-business-tables`
`genie-api`	Real-time natural-language interface over governed GTM data for humans and AI agents.	`gold-business-tables`, `growthos-agent-harness`
`growthos-agent-harness`	Multi-model agent harness for growth use cases that queries GTM data and acts on findings.	`genie-api`, `conversation-retrieval-agent`
`conversation-delta-worklist`	Delta change tracking worklist of new or changed sales conversations for incremental refresh.	`conversation-retrieval-agent`
`conversation-retrieval-agent`	Semantic-search agent over sales conversations for reps, marketers, and agentic GTM workflows.	`conversation-delta-worklist`, `speaker-labeler`, `chunk-enricher`, `vector-search-index`, `gold-business-tables`
`speaker-labeler`	Uses participant metadata and deterministic scoring to label seller/customer turns.	`conversation-retrieval-agent`, `chunk-enricher`
`chunk-enricher`	Preserves conversational context and extracts pain points, objections, triggers, competitors, timing, and sentiment before embedding.	`speaker-labeler`, `vector-search-index`
`vector-search-index`	Storage-optimized vector search index over enriched text with high-selectivity filters and hybrid dense/keyword retrieval.	`chunk-enricher`, `conversation-retrieval-agent`

Data flow

Third-party SaaS exports and internal GTM data are ingested incrementally into the lakehouse. The Silver layer resolves company/person identities by blocking candidate pairs and classifying matches; the resulting canonical entities carry confidence and source audits. Gold tables serve AI/ML pipelines and the Genie API, which humans and agents query in real time. For conversation search, Delta change tracking selects new or changed calls; the pipeline labels seller/customer speakers, chunks complete conversational exchanges, enriches buyer evidence with objections/triggers/sentiment, embeds the enriched text into storage-optimized vector search, retrieves with hybrid dense/keyword methods, and joins heavier business context from the lakehouse after retrieval.

5 supporting quotes

“The whole system is one platform with a clean separation of concerns — incremental ingestion into a medallion lakehouse, ML entity resolution in the Silver layer, AI/ML pipelines and a real-time API on top, serving both people and agents.”article, stated

“The system applies candidate blocking and then uses a supervised pairwise classifier to score whether two records represent the same entity.”article, stated

“Agents query it in real time to analyze a range of first- and third-party signals across the customer lifecycle, then act on what they find.”article, stated

“First, the pipeline identifies only new or changed conversations using Delta change tracking.”article, stated

“Before embedding, we run enrichment on each chunk: pain points, objections, buying triggers, competitor mentions, contract timing, sentiment.”article, stated

Why this architecture

Rippling’s central design principle is that agents need trustworthy data more than more context. Databricks was chosen because AI/ML workloads could run close to governed data with hosted model access. The medallion lakehouse separates ingestion, resolution, and serving, while entity-resolution audit trails establish what is real before agents act. Conversation search precomputes enriched evidence once and serves compact retrieval results repeatedly, avoiding the cost, latency, and auditability problems of dumping raw warehouse or transcript records into LLM context windows.

4 supporting quotes

“We didn’t want three separate systems for those three needs. We wanted one source of truth.”article, stated

“Keeping model inference next to governed data — instead of exporting it to yet another service — removed a whole category of plumbing, latency, and governance headaches.”article, stated

“Every resolved entity carries a confidence score, a ranked audit of every source that contributed to the final canonical record — winners, losers, where they conflicted.”article, stated

“We do the hard reasoning once, at ingestion. At query time the agent gets a compact, grounded evidence set — not a wall of text.”article, stated

Models used: managed foundation models · entity-resolution classifier · embedding models · multi-model GrowthOS harness

Scale

The GTM organization included 1,000+ people and later the unified interface served more than 1,500 people across the company.
“With 1,000+ people in go-to-market and Rippling growing fast”article, stated
“used by more than 1,500 people across the company using a unified source of truth for analytics and market intelligence data.”article, stated
Entity resolution ran across hundreds of millions of heterogeneous vendor records and completed as a Spark prediction job across thousands of partitions.
“We deployed the resolution pipeline as a Spark job across hundreds of millions of records pulled from many heterogeneous vendor sources”article, stated
“The production prediction job completed across thousands of Spark partitions. It took just under 40 hours.”article, stated
The semantic search corpus contains millions of sales conversations and supports grounded answers in seconds.
“semantic search over our library of sales-data — millions of conversations — so a rep, marketer or an agent acting on their behalf, can ask a plain-language question”article, stated
Storage-optimized vector search kept multi-million-record query latency in the tens of milliseconds.
“query latency settled into the tens of milliseconds even at multi-million-record scale.”article, stated

Alternatives considered (roads not taken)

The incumbent Snowflake warehouse for AI/ML summarization and classification workloads

Rejected because: The Databricks POC won on cost, speed, and in-platform hosted model access.

“We evaluated Databricks against Snowflake, our incumbent cloud data warehouse.”article, stated

“Databricks won on every metric that mattered to us:”article, implied

Three separate systems for AI/ML workloads, identity unification, and low-latency serving

Rejected because: Rippling wanted one governed source of truth instead of splitting data, inference, and serving across disconnected stacks.

“We didn’t want three separate systems for those three needs. We wanted one source of truth.”article, stated

Rule-based entity matching only

Rejected because: Rule-based matching collapsed under heterogeneous vendor edge cases at GTM scale.

“Rule-based matching gets you partway and then collapses under edge cases.”article, stated

Directly feeding warehouse records into LLM context for every request

Rejected because: That repeats extraction/reasoning, burns tokens, creates variable latency, and weakens auditability.

“every question re-pulls full records, re-runs extraction, and burns tokens doing reasoning that should have happened once. Cost compounds, latency varies, and nothing is auditable.”article, stated

Pure vector retrieval over sales conversations

Rejected because: GTM questions need exact product names, competitors, account stages, acronyms, and fuzzy semantic concerns.

“Hybrid retrieval beats pure vectors. Sales conversations mix fuzzy concepts (“pricing concerns”) with exact terms: product names, competitors, account stages, acronyms.”article, stated

Patterns

`data-quality-loop` (entity-resolution-provenance)

Rippling resolves messy third-party records into canonical entities with confidence scores and source audit trails before agents consume GTM data.

“to unify these records at scale — reconciling entities across heterogeneous third-party sources into a single view.”article, stated

`tool-use-orchestration` (agent-ready-data-interface)

Agents query Genie in real time to analyze first- and third-party lifecycle signals, then act on the findings.

“Agents query it in real time to analyze a range of first- and third-party signals across the customer lifecycle, then act on what they find.”article, stated

`hybrid-search` (sales-conversation-retrieval)

Sales conversation retrieval combines dense semantic retrieval with keyword retrieval for fuzzy concepts and exact terms.

“Hybrid retrieval beats pure vectors. Sales conversations mix fuzzy concepts (“pricing concerns”) with exact terms: product names, competitors, account stages, acronyms.”article, stated

`batch-offline-processing` (incremental-embedding-pipeline)

The conversation pipeline processes only new or changed calls so daily refreshes scale with change volume rather than corpus size.

“the pipeline identifies only new or changed conversations using Delta change tracking. Those calls become the worklist for the rest of the job, so daily refreshes scale with change volume rather than total corpus size.”article, stated

`context-compaction` (retrieval-ready-conversation-enrichment)

Rippling enriches chunks with objections, buying triggers, sentiment, and other signal types before embedding, so agents receive compact evidence instead of raw transcripts.

“Before embedding, we run enrichment on each chunk: pain points, objections, buying triggers, competitor mentions, contract timing, sentiment.”article, stated

`cost-aware-routing` (precomputed-ai-ready-representations)

Rippling precomputes AI-ready representations once and serves them repeatedly through governed retrieval rather than rebuilding context for every request.

“Directly feeding warehouse records into LLM context for every MCP request wastes tokens and produces unpredictable results. The better pattern is to precompute the AI-ready representation once”article, stated

Tradeoffs

Where to run AI/ML workloads for GTM data.

Chose Databricks co-located with governed data over continuing on Snowflake plus separate inference services.

Rationale: Databricks won the POC on cost, speed, and hosted-model access, while avoiding data movement and governance headaches.

“Keeping model inference next to governed data — instead of exporting it to yet another service — removed a whole category of plumbing, latency, and governance headaches.”article, stated

How to structure conversation retrieval.

Chose enriched chunks and lean vector indexes with business context joined after retrieval over putting all transcript and business context into the vector store.

Rationale: Separating retrieval from analytics keeps indexes smaller and answers richer.

“The index should retrieve candidates quickly. The lakehouse should provide the full business context after retrieval. Keeping those jobs separate gave us a smaller index, faster queries, and richer final answers.”article, stated

How to keep the data layer usable beyond one query path.

Chose Delta tables supporting Iceberg reads and interoperability by default over locking GTM data into a single engine or access pattern.

Rationale: Open table formats preserve flexibility for multiple engines, teams, and future agent use cases.

“We standardized on open table formats — Delta tables supporting Iceberg reads — so the data stays open and accessible to other engines and tools instead of being locked into one query path.”article, stated

How to scale large semantic indexes.

Chose storage-optimized vector search for the transcript index over in-memory vector endpoints past multi-million-record scale.

Rationale: Storage-optimized search was less brittle and kept latency acceptable as the index grew.

“Once a semantic index grows past a few million records, in-memory endpoints get expensive and brittle.”article, stated

“We moved the transcript index onto storage-optimized Vector Search”article, stated

How to refresh conversation retrieval pipelines.

Chose incremental synchronization around new/changed data over rebuilding or re-embedding the world on each refresh.

Rationale: Processing only changes keeps daily refresh cost tied to change volume rather than total corpus size.

“It’s the difference between a pipeline that scales with your changes and one that scales with your total data size — only one of those stays affordable.”article, stated

Gotchas (what bit them)

Agentic workloads amplify unreliable data models.

Root cause: Agents need a single source of truth, and duplicate or stale lead records produce bad actions at scale.

Resolution: Rippling rebuilt the GTM foundation around ML entity resolution, canonical records, confidence scores, and audit trails.

“Inability to implement agentic, high leverage workloads due to unreliable data models”article, implied

Naively dumping full transcripts into context windows does not scale.

Root cause: It is expensive, slow, and forces the model to rediscover the same structure on every query.

Resolution: Do reasoning once at ingestion and give query-time agents compact grounded evidence sets.

“The alternative is dumping full transcripts into a context window every time. That works in a demo. In production it’s expensive, slow, and the model rediscovers the same structure on every call.”article, stated

Large vector indexes become expensive and brittle if sized like small prototypes.

Root cause: In-memory endpoints struggle once semantic indexes grow past a few million records.

Resolution: Move large transcript indexes to storage-optimized vector search and design for future scale.

“Once a semantic index grows past a few million records, in-memory endpoints get expensive and brittle.”article, stated

Evaluation

Rippling evaluated platform migration with a Databricks-vs-Snowflake POC, measured vector search latency and GTM A/B outcomes, and used provenance/confidence for entity-resolution auditability.

Metrics: pipeline cost · processing and inference speed · query latency · demos booked · new opportunities · addressable TAM · sales-play generation cost

“put two platforms directly against each other on the workloads we actually cared about: AI/ML pipelines doing summarization and classification at scale.”article, stated

“query latency settled into the tens of milliseconds even at multi-million-record scale.”article, stated

Outcomes

The platform became core production infrastructure in less than three months.
Staged A/B tests showed demos booked up about one third and new opportunities up about one fifth.
The platform increased addressable TAM by over 10 million potential users and made personalized sales plays roughly an order of magnitude cheaper.

The platform serves more than 1,500 people according to the article body, and the Genie interface section describes 2,000+ users.

3 supporting quotes

“essentially zero to a core piece of production infrastructure used by multiple teams including operations, data science, growth marketing and growth engineering in less than 3 months”article, stated

“helped lift demos booked by about a third and new opportunities by roughly a fifth in a staged A/B test vs templated emails.”article, stated

“This also allowed the launch of GrowthOS — our multi-model agent harness for growth.”article, stated