Building Unified API Infrastructure for AI Integration at Scale

merge · Tech · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: AI data integration infrastructure

Merge had to support AI products that need secure access to customer work data through both comprehensive synced datasets for RAG and live API calls for agentic actions, while handling authentication, tool explosion, DLP, pagination, and enterprise reliability.

Constraints:

Architecture

Merge keeps its classic unified API product for full normalized data sync and vector-store/RAG use cases, and adds Agent Handler for live MCP calls. Agent Handler exposes list_tools and call_tool over Merge connectors, handles per-user authentication, offers tool search, scans responses through DLP, and routes calls through existing enterprise-grade integration infrastructure.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  third-party-platform-apis["third-party platform APIs"] --> merge-connector-layer["Merge connector layer"]
  merge-connector-layer --> synced-data-pipeline["synced data pipeline"]
  synced-data-pipeline --> vector-store["vector store"]
  vector-store --> ai-product-rag-layer["AI product RAG layer"]
  ai-product-rag-layer --> llm-agent-runtime["LLM agent runtime"]
  merge-connector-layer --> agent-handler-mcp-server["Agent Handler MCP server"]
  agent-handler-mcp-server --> list-tools-endpoint["list_tools endpoint"]
  agent-handler-mcp-server --> call-tool-endpoint["call_tool endpoint"]
  agent-handler-mcp-server --> tool-search-endpoint["tool search endpoint"]
  agent-handler-mcp-server --> authentication-flow["authentication flow"]
  authentication-flow --> agent-handler-mcp-server
  call-tool-endpoint --> merge-connector-layer
  call-tool-endpoint --> dlp-scanner["DLP scanner"]
  merge-connector-layer --> dlp-scanner
  dlp-scanner --> security-approval-workflow["security approval workflow"]
  llm-agent-runtime --> agent-handler-mcp-server
  list-tools-endpoint --> llm-agent-runtime
  tool-search-endpoint --> llm-agent-runtime
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
third-party platform APIs Source systems such as HR, accounting, CRM, ticketing, file storage, Slack, Linear, and chat platforms. Merge connector layer
Merge connector layer Handles native integrations, credentials, normalization, and production API access across platforms. synced data pipeline, Agent Handler MCP server, authentication flow, DLP scanner
synced data pipeline Copies and normalizes full customer datasets for downstream RAG and semantic search. vector store, AI product RAG layer
vector store Embeds synced data to support semantic search and retrieval. AI product RAG layer
AI product RAG layer Uses synced/vectorized data to answer retrieval-heavy questions. LLM agent runtime
Agent Handler MCP server Provides live MCP tool discovery and tool invocation backed by Merge connectors. list_tools endpoint, call_tool endpoint, tool search endpoint, authentication flow, DLP scanner, LLM agent runtime
list_tools endpoint Returns standard tool/function descriptions available for a third-party platform. LLM agent runtime
call_tool endpoint Invokes selected tools with arguments against underlying APIs. Merge connector layer, DLP scanner
tool search endpoint Accepts an intent string and returns a smaller set of relevant tools. LLM agent runtime
authentication flow Checks credentials for a user identity and exposes only an authentication tool until connected. Agent Handler MCP server
DLP scanner Detects secrets or sensitive data in API responses before agent calls proceed. call_tool endpoint, security approval workflow
security approval workflow Lets enterprise security teams review new agent workflows before deployment. DLP scanner
LLM agent runtime Chooses tools, performs RAG, or calls live APIs depending on the workflow. AI product RAG layer, Agent Handler MCP server

Data flow

For RAG, Merge syncs third-party customer data, normalizes it, and lets AI products vectorize/embed it into local stores. For agent actions, an LLM or agent framework calls Agent Handler; list_tools exposes relevant functions, call_tool invokes third-party APIs, authentication is required per user, DLP scans responses, and tool search narrows context before execution.

3 supporting quotes
“Our classic product, our unified product is a synced product where we're bringing full data sets across and normalizing to a single format.”transcript, stated
“our new product is that MCP live calling and when you combine that with our classic synced data set, you end up being able to really do anything with third party data and AI.”transcript, stated
“There's one called list tools which the agent knows how to call and it returns to the agent a standard format of all the functionalities or tools”transcript, stated

Why this architecture

Merge keeps both synced data and live MCP calls because they solve different problems: vectorized synced data wins for semantic retrieval quality, while live API calls are cheaper and flexible for agent actions. Agent Handler wraps live calls in authentication, security, and tool-discovery infrastructure so agents can safely act on third-party data.

3 supporting quotes
“There's pros and cons. We can get into if you want, but pros and cons of both approaches. Our classic product, our unified product is a synced product where we're bringing full data sets across and normalizing to a single format. That's essential for”transcript, stated
“That's essential for semantic search for a lot of AI based use cases, but also it's expensive to run. You have a full copy of the data set. You run into rate limits. You run into partnership limits. There are there are some inherent disadvantages there”transcript, stated
“live API requests is really cheap. It's very flexible, but it also doesn't give you the same abilities to do retrieval in good ways”transcript, stated

Models used: Mistral · Perplexity · LLMs

Scale

Alternatives considered (roads not taken)

Use only live MCP/API calls for AI integrations.

Rejected because: Live calls are flexible and cheap but poor for retrieval-heavy semantic search compared with local vector stores.

“live API requests is really cheap. It's very flexible, but it also doesn't give you the same abilities to do retrieval in good ways”transcript, stated
“a vector store outperforms every single time hands. It doesn't even come close." ## MCP Implementation and Production Challenges The Agent Handler product implements MCP through a standardized interface. When integrated into production environments, the”summary, stated

Use only synced datasets/vector stores.

Rejected because: Synced data is expensive, hits rate and partnership limits, and does not replace live action-oriented endpoints.

“it's expensive to run. You have a full copy of the data set. You run into rate limits. You run into partnership limits.”transcript, stated
“when you combine that with our classic synced data set, you end up being able to really do anything with third party data and AI.”transcript, stated

Expose all tools directly into the LLM context.

Rejected because: As integrations grow, tool volume can overwhelm context, so Merge uses intent-based tool search.

“As customers add more integrations, they face tool explosion—where thousands of tools overwhelm the agent's context window.”summary, stated
“a tool search endpoint that accepts an intent string and returns the appropriate tools to accomplish the task, reducing context window consumption. Pagination and Context Management: Gil identifies a critical limitation of MCP that many developers overlook:”summary, stated

Let agents freely compose large workflows without deterministic boundaries.

Rejected because: Merge favors small composable components when reliability matters, because intermediate validation is easier.

“if you want something to be deterministic you want the outcomes to be mostly reliable you want to break it down as much as possible”transcript, stated
“it's better to separate that out make it composable and combine them into sort of one workflow. And that's why so agent handler our product provides those smallest level components and then and then agent builders or just direct LLM calls can decide how they”transcript, stated

Patterns

rag-retrieval (retrieval)

Merge's synced unified API product enables customers to copy, normalize, vectorize, and retrieve customer data for AI search.

“do I want to create a copy of that data set and bring it over and likely vectorize and embed it in a in a vector store”transcript, stated
“That's essential for semantic search for a lot of AI based use cases”transcript, stated

tool-use-orchestration (agent architecture)

Agent Handler exposes third-party API actions as MCP tools that agents can select and call.

“There's one called list tools which the agent knows how to call”transcript, stated
“a call tool endpoint that lets you actually invoke the tool with arguments.”transcript, stated

guardrails-io-filtering (security)

Agent Handler includes DLP scanning to block secrets and sensitive data from agent calls.

“Agent Handler includes a built-in DLP layer that scans for sensitive data and blocks agent calls if it detects secrets, API keys, or other confidential information.”summary, stated

human-in-the-loop-gate (governance)

Enterprise workflows can require security-team approval before new agent workflows are deployed.

“Merge has implemented approval workflows where new agent workflows must be reviewed by security teams before deployment.”summary, stated

context-compaction (context management)

Tool search narrows tool lists so thousands of available tools do not consume the agent's context window.

“As customers add more integrations, they face tool explosion—where thousands of tools overwhelm the agent's context window. Merge addresses this with a tool search endpoint that accepts an intent string and returns the appropriate tools to accomplish the task, reducing context window consumption.”summary, stated
“returns the appropriate tools to accomplish the task, reducing context window consumption.”summary, stated

Tradeoffs

Support synced datasets and live MCP calls.

Chose Two access paths over A single universal integration mode.

Rationale: Synced/vectorized data is better for retrieval, while live calls are cheaper and better for agentic actions.

“when you combine that with our classic synced data set, you end up being able to really do anything with third party data and AI.”transcript, stated
“There are there are some inherent disadvantages there versus live syncing is really I'm sorry, live API requests is really cheap. It's very flexible, but it also doesn't give you the same abilities to do retrieval in good ways because you're limited by the access patterns of the underlying APIs.”transcript, stated

Break tools into small composable components.

Chose Small deterministic MCP operations over One giant prompt or free-form agent workflow.

Rationale: Smaller pieces make outcomes more reliable and easier to validate between steps.

“if you want something to be deterministic you want the outcomes to be mostly reliable you want to break it down as much as possible”transcript, stated
“it's better to separate that out make it composable and combine them into sort of one workflow.”transcript, stated

Drive internal AI adoption through champions.

Chose Department-level transformative representation over Forcing every employee to become transformative immediately.

Rationale: Merge found energized champions better at adopting tools, teaching teammates, and building department-specific workflows.

“So, we identified the people who are just really excited about AI. That was one of our RevOps leaders. That was one of our engineering leaders.”transcript, stated
“every department must have transformative representation who is doing that for their team”transcript, stated

Gotchas (what bit them)

Live MCP calls are weak for large-scale retrieval.

Root cause: Underlying APIs constrain access patterns, and result sets pass through the LLM context rather than program variables.

Resolution: Use synced/vectorized data for retrieval-heavy workloads and live API calls for action-oriented workflows.

“it also doesn't give you the same abilities to do retrieval in good ways because you're limited by the access patterns of the underlying APIs.”transcript, stated
“the entire result set passes through the LLM's context window, creating scalability problems.”summary, stated

Naive agent calls can leak secrets.

Root cause: Third-party APIs may return sensitive data that the agent can then send elsewhere.

Resolution: Add DLP scanning and security review workflows around agent actions.

“know a secret key for one of the repos from an endpoint because you hit a key endpoint or whatever it is and your agent has the ability to just send that data anywhere." Agent Handler includes a built-in DLP layer that scans for sensitive data and blocks agent calls if it detects secrets, API”summary, stated
“blocks agent calls if it detects secrets, API keys, or other confidential information.”summary, stated

Legacy enterprise auth does not fit clean OAuth assumptions.

Root cause: Many third-party enterprise systems do not use OAuth, so a universal MCP auth flow has to handle varied schemes.

Resolution: Expose only an authentication tool until credentials exist for the user identity.

“many legacy enterprise systems don't use OAuth, creating ongoing integration challenges.”summary, stated
“If not, it returns a single "authenticate this user" tool. Merge provides a component that handles the authentication flow, including OAuth and other authentication methods.”summary, stated

Tool granularity can make workflows either unreliable or too low-level.

Root cause: Large prompts are less deterministic, but tiny tools force orchestration complexity onto agent builders.

Resolution: Expose small reliable components today while expecting higher-level API abstractions to emerge.

“agent handler our product provides those smallest level components and then and then agent builders or just direct LLM calls can decide how they want to use them.”transcript, stated

Evaluation

The source discusses production validation through customer adoption, LLM-provider usage, internal AI output gains, security/compliance requirements, and observed retrieval tradeoffs rather than a single quantitative benchmark.

Metrics: Largest LLM providers use Merge, according to the speaker · Perplexity uses Agent Handler for Linear, CRM, and chat connectors · Initial customer discovery: 100 conversations, about 90 positive, 12 onboarded · Qualitative internal output increases · No formal retrieval-quality benchmark disclosed in transcript

3 supporting quotes
“You know um Perplexity for example uses our agent handler product. Um they built the linear connector with it. They've done CRM connectors.”transcript, stated
“the KPIs that they already have, they are not improving that much, but then they are doing like many more things.”transcript, stated
“for it." And so we had a stream of 90 people who had said yes we will use this platform of those I think when we first opened the platform like 12 on boarded but that was a huge number for us to get started and have 12 people actually not just onboarded but like we're a deep technical platform”transcript, stated

Outcomes

Merge describes five years of company growth, large LLM-provider customers, offices in SF/New York/Berlin, and initial validation/onboarding numbers, but does not disclose exact revenue or throughput.

3 supporting quotes
“our core product is still heavily in use by AI companies. In fact, it's actively being adopted by the top AI companies”transcript, stated
“You know um Perplexity for example uses our agent handler product. Um they built the linear connector with it. They've done CRM connectors.”transcript, stated
“there hasn't been any change in need for integrations and for data because now everyone has the same LLMs.”transcript, stated