Building a Search Engine for AI Agents: Infrastructure, Product Development, and Production Deployment

exa.ai · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Search infrastructure for AI agents

Exa needed to build search for AI agents rather than humans: agents need semantic search, raw data, low latency, privacy, control over retrieval, and chained search workflows, while traditional search engines are optimized for consumer clicks and short keyword queries.

Constraints:

Google-style search is optimized for consumer clicks and short keyword searches
Agents need raw information they can ingest, not listicle-style summaries
Long semantic queries do not work well on traditional search
AI agent workflows may require many chained searches where latency compounds
Enterprise customers care about privacy and data retention
Search over the web requires owning index, model, product, and infrastructure decisions

Architecture

Exa built a search engine for AI agents by training its own models, building its own web index, embedding web documents, ranking through semantic/vector matching plus keywords, and exposing both a low-latency search API and Websites, an agentic search product that can find and enrich difficult web targets. Owning the model and index supports privacy, low latency, custom retrieval behavior, re-ranking, and research-first improvements.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  ai-agent[AI Agent] --> search-api[Search API]
  ai-agent[AI Agent] --> websites[Websites]
  websites[Websites] --> agentic-workflows[Agentic Workflows]
  agentic-workflows[Agentic Workflows] --> search-api[Search API]
  search-api[Search API] --> exa-index[Exa Index]
  web-crawler[Web Crawler] --> embedding-models[Embedding Models]
  web-crawler[Web Crawler] --> exa-index[Exa Index]
  embedding-models[Embedding Models] --> exa-index[Exa Index]
  exa-index[Exa Index] --> re-ranker[Re Ranker]
  re-ranker[Re Ranker] --> search-api[Search API]
  gpu-cluster[GPU Cluster] --> embedding-models[Embedding Models]
  gpu-cluster[GPU Cluster] --> re-ranker[Re Ranker]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`AI Agent`	Downstream agent or AI application that needs web retrieval.	`Search API`, `Websites`
`Search API`	Low-latency API for agent and developer search use cases.	`AI Agent`, `Exa Index`, `Re Ranker`
`Websites`	Agentic search product that finds and enriches hard-to-find web entities.	`AI Agent`, `Search API`, `Agentic Workflows`
`Agentic Workflows`	Multi-step search and enrichment workflows that assemble validated lists.	`Websites`, `Search API`
`Web Crawler`	Ingestion layer that collects documents from the web.	`Embedding Models`, `Exa Index`
`Embedding Models`	Exa-trained models that embed web documents and support semantic matching.	`Web Crawler`, `Exa Index`
`Exa Index`	Owned web index for semantic and keyword search.	`Search API`, `Web Crawler`, `Embedding Models`, `Re Ranker`
`Re Ranker`	Exa-trained ranking layer optimized for faster search results.	`Search API`, `Exa Index`
`GPU Cluster`	Owned compute cluster used for research and model/index development.	`Embedding Models`, `Re Ranker`

Data flow

Exa ingests web documents, turns them into embeddings, and stores them in its own index. A developer, agent, or Websites workflow sends a natural-language or structured search request; Exa combines semantic/vector matching with keyword matching, reranks results with its own re-ranker, returns URLs or full content, and can chain searches through Websites to assemble validated lists from the web.

4 supporting quotes

“basically what we do is we ingest all of the documents in the web and turn them into embeddings. Um so we're able to basically capture all of these semantic understanding of those websites. And then whenever we're doing the search, yeah, we're we can, you know, find”transcript, stated

“works. So basically, we train our own model. We build our own index and basically what we do is we ingest all of the documents in the web and turn them into embeddings. Um so we're able to basically capture all of these semantic”transcript, stated

“yeah, we're we can, you know, find results based on keywords, but also on that full kind of vector matching and cosign similarity there. So that's a little bit of how we're we're going about it. And you're right that like long queries don't work well at all on Google. So I'll give you a random”transcript, stated

“yes let's use our search in the back end and combine it with very intelligent agentic workflows on top of it to return a perfect list that is fully validated of exactly what you want. So I had given the example of of software developers but let's say you want to find”transcript, stated

Why this architecture

Exa built the stack in house because search for agents needs different incentives from consumer search: raw primary information, semantic understanding, customization, low latency, and privacy. Owning the index, models, re-ranker, and cluster lets Exa tune retrieval for AI agents instead of wrapping Google-like platforms.

3 supporting quotes

“versus for agents uh the reality is they don't want like one listical that summarizes the answer of what they're looking for. They want raw information that can ingest a lot of it and they want a lot more control and precision over what information to find. So it”transcript, stated

“looking for. They want raw information that can ingest a lot of it and they want a lot more control and precision over what information to find. So it just requires a very different logic behind search that's a lot more meaning based but also that has all these”transcript, stated

“house. So because we own the model, we own the index, we can guarantee this privacy um which for a lot of customers matters way more than I ever imagined I would say. So >> for many people that uh worked B2C before, they are surprised how much”transcript, stated

Models used: Exa search models · embedding models · Exa re-ranker · LLMs

Scale

Exa spent millions of dollars on a cluster for R&D and search breakthroughs.
“start so like we spend millions ous of dollars on a cluster because we wanted to be able to have access to a lot of compute to be able to do a lot of R&D and truly like discover breakthroughs in search right like a lot of the things we're doing have never been done before”transcript, stated
The company grew from around 7-8 people to 28, with a goal of 55 by the next quarter.
“I think I was probably we were seven maybe seven or eight and now we're 28 and by the end of this next quarter we want to be 55 so growing very quickly hiring a lot. We we've been spending a lot of time even as a team on recruiting. I mean when I joined I actually spent pretty much two months I”transcript, stated
Exa reports roughly 95% inbound growth/distribution.
“cognizant of. And so far we've grown I would say 95% inbound which is uh which is a huge benefit as a company. So I think nailing that distribution and how do you combine like productled growth this like strong developer community and”transcript, stated
Agent workflows may require 20 searches or 30 searches, making latency compound.
“If you think about it, if you need to do 20 searches, every little bit of latency that's higher compounds. So you really really need to unlock low latency for a lot of use cases to even make sense. So that's one. And then this like privacy and data retention has been actually one”transcript, stated
“that needs to do a lot of calls. So if you have 30 different searches to run again it like accumulates. So it matters a lot a lot for that type of use case and we basically invested in like bringing and training our own re-ranker to make it faster. Uh we parallelized a”transcript, stated
Exa has run hundreds of millions of queries.
“before. We have hundreds of millions of queries, right, that we've run. And that data is so interesting if you can look at like frequency of topics or how they're being used in terms of use cases and test for these really specific real”transcript, stated
The Websites example asks for US healthcare startups with doctor founders, more than $50 million raised, and a new product in the last year.
“healthcare startups in the US with founders that are doctors that have raised more than $50 million and released a new product in the last year”transcript, stated

Alternatives considered (roads not taken)

Use Google-style consumer search

Rejected because: Consumer search optimizes for clicks and short keyword queries, not raw data ingestion and precise semantic retrieval for agents.

“in in a lot of ways. And Google is really built for consumers and especially the way that the company evolved and the business model evolved is they're optimizing for clicks, right?”transcript, stated

“is they're optimizing for clicks, right? They're optimizing for you as a person to search for something in very few keywords also because humans tend to be lazy when we type and getting links and kind of clicking on those links and making Google ad revenue, right?”transcript, stated

Wrap existing search platforms

Rejected because: Wrapping platforms inherits latency, privacy, and zero-data-retention limitations.

“are building their own models the majority of them even like providers within search wrap platforms like Google so and then you end up having all the limitations that Google has but you also end up having a bunch of other limitations like you can't do true like”transcript, stated

“limitations like you can't do true like zero data retention and you can't serve certain like low latencies because you're constrained by the systems that you wrapped. So we invested very heavily on research and I would say in the beginning it was almost like not not”transcript, stated

Offer only APIs

Rejected because: Some customers needed a ready-made agentic search product instead of engineering a custom solution.

“all these people that don't necessarily want to build a custom solution or dedicated engineering resources but they also need to find very hard things and uh how do we serve them? So the idea is yes let's use our search in the back end and combine it with very intelligent”transcript, stated

“it's a readym made agent that's out of the box that people can use or rather it's like this really fast low latency API”transcript, stated

Patterns

`rag-retrieval` (retrieval)

Exa provides web retrieval tailored for AI agents that need raw information for downstream LLM workflows.

“They want raw information that can ingest a lot of it”transcript, stated

`retrieval-ranking` (search)

Exa embeds web documents and reranks results for semantic agent search.

“turn them into embeddings. Um so we're able to basically capture all of these semantic understanding of those websites.”transcript, stated

“training our own re-ranker to make it faster.”transcript, stated

`latency-optimization` (performance)

The search API is optimized for agent workflows where many search calls compound latency.

“if you need to do 20 searches, every little bit of latency that's higher compounds.”transcript, stated

`privacy-preserving-processing` (security)

Owning the model and index lets Exa offer privacy and data-retention guarantees.

“because we own the model, we own the index, we can guarantee this privacy”transcript, stated

`multi-agent-orchestration` (agentic-search)

Websites layers agentic workflows on top of search to find and enrich complex web targets.

“combine it with very intelligent agentic workflows on top of it to return a perfect list that is fully validated”transcript, stated

Tradeoffs

Own the search stack

Chose Own index, models, cluster, and re-ranker over Wrapping Google-like search platforms.

Rationale: Ownership enables privacy guarantees, low-latency targets, and research-driven improvements for agent search.

“there are very few companies out there that have their own index and are building their own models”transcript, stated

“you can't do true like zero data retention and you can't serve certain like low latencies”transcript, stated

Offer both API and Websites product

Chose Low-latency API plus ready-made agentic search over API-only product surface.

Rationale: Some customers want to build custom agent integrations, while others need a ready-made product for hard web-finding tasks.

“how do we serve all of these needs rather it's a readym made agent that's out of the box that people can use or rather it's like this really fast low latency API”transcript, stated

Invest early in research infrastructure

Chose Research-first organization and owned GPU cluster over Waiting for demand to justify infrastructure.

Rationale: Exa expected agent search to require breakthroughs not possible with wrapped infrastructure.

“we've been big believers in being a research first organization from the start”transcript, stated

“we spend millions ous of dollars on a cluster because we wanted to be able to have access to a lot of compute”transcript, stated

Gotchas (what bit them)

Traditional search incentives do not match agent needs.

Root cause: Consumer search optimizes for clicks, short keywords, and summaries rather than precise raw data retrieval.

Resolution: Exa built a search engine specifically for AI use cases.

“They're optimizing for you as a person to search for something in very few keywords also because humans tend to be lazy when we type”transcript, stated

“it's the first search engine in the world that's built for AI use cases.”transcript, stated

Wrapping existing search platforms blocks some enterprise use cases.

Root cause: Wrapped platforms constrain privacy and latency properties.

Resolution: Exa owns its model, index, and stack.

“you end up having all the limitations that Google has”transcript, stated

“because we own the model, we own the index, we can guarantee this privacy”transcript, stated

Latency compounds in multi-search agent workflows.

Root cause: Agents may run many sequential or parallel search calls before answering.

Resolution: Exa invested in a faster re-ranker, parallelization, and a low-latency search API.

“if you have an agent that needs to do a lot of calls. So if you have 30 different searches to run again it like accumulates.”transcript, stated

“training our own re-ranker to make it faster. Uh we parallelized a bunch of stuff”transcript, stated

Evaluation

The transcript does not describe a formal benchmark harness, but Exa evaluates search through query volume, user research, latency needs, product usage, inbound growth, and research-led reranker/search improvements.

Metrics: hundreds of millions of queries · 20-search and 30-search latency compounding use cases · 95% inbound growth · team growth from 7-8 to 28 · privacy and zero data retention requirements

3 supporting quotes

“We have hundreds of millions of queries, right, that we've run.”transcript, stated

“if you need to do 20 searches, every little bit of latency that's higher compounds.”transcript, stated

“so far we've grown I would say 95% inbound”transcript, stated

Outcomes

Exa built an AI-agent-focused search engine with its own model and index
The company created both a search API and Websites agentic search product
Exa reports hundreds of millions of queries run
The team grew from roughly 7-8 people to 28 and targeted 55 by the next quarter

The source reports hundreds of millions of queries, millions spent on a cluster, 95% inbound growth, and agent workflows where 20-30 searches make low latency critical.

3 supporting quotes

“we train our own model. We build our own index”transcript, stated

“We have hundreds of millions of queries, right, that we've run.”transcript, stated

“we were seven maybe seven or eight and now we're 28 and by the end of this next quarter we want to be 55”transcript, stated

Building a Search Engine for AI Agents: Infrastructure, Product Development, and Production Deployment

Problem

Architecture

Components

Data flow

Why this architecture

Scale

Alternatives considered (roads not taken)

Use Google-style consumer search

Wrap existing search platforms

Offer only APIs

Patterns

rag-retrieval (retrieval)

retrieval-ranking (search)

latency-optimization (performance)

privacy-preserving-processing (security)

multi-agent-orchestration (agentic-search)

Tradeoffs

Own the search stack

Offer both API and Websites product

Invest early in research infrastructure

Gotchas (what bit them)

Traditional search incentives do not match agent needs.

Wrapping existing search platforms blocks some enterprise use cases.

Latency compounds in multi-search agent workflows.

Evaluation

Outcomes

`rag-retrieval` (retrieval)

`retrieval-ranking` (search)

`latency-optimization` (performance)

`privacy-preserving-processing` (security)

`multi-agent-orchestration` (agentic-search)