Enhancing AI Coding Agent Performance with Custom Semantic Search

cursor · Tech · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: AI coding-agent code retrieval

Cursor needed its coding agent to understand large codebases better than grep-only search allowed, especially when users asked natural-language questions about where behavior lived in code.

Constraints:

Coding agents need to read files and search for relevant information before answering or editing.
Regex search works well for literal matches but misses semantic intent.
Search tooling must improve agent quality across multiple frontier coding models.
Production value must show up in both offline retrieval benchmarks and online user outcomes.
Large codebases amplify retrieval difficulty.

Architecture

Cursor added semantic search as a first-class coding-agent tool alongside grep. It trained a custom embedding model from agent session traces, using an LLM to rank which code content would have been useful earlier in the agent's work, then built fast indexing pipelines so the agent can retrieve semantically relevant code segments from natural-language queries during a session.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  agent-coding-prompt["Agent coding prompt"] --> agent-search-tools["Agent search tools"]
  agent-search-tools --> semantic-search-retriever["Semantic search retriever"]
  agent-search-tools --> grep-search["Grep search"]
  semantic-search-retriever --> coding-agent-answer["Coding agent answer"]
  grep-search --> coding-agent-answer
  coding-agent-answer --> agent-session-traces["Agent session traces"]
  agent-session-traces --> llm-ranking-teacher["LLM ranking teacher"]
  llm-ranking-teacher --> custom-embedding-model["Custom embedding model"]
  indexing-pipelines["Indexing pipelines"] --> custom-embedding-model
  custom-embedding-model --> semantic-search-retriever

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Agent coding prompt`	User task that requires the agent to understand relevant codebase behavior.	`Agent search tools`
`Agent search tools`	Tool surface combining grep-style literal search and semantic search.	`Semantic search retriever`, `Grep search`
`Grep search`	Regex-based command-line search for literal matches.	`Coding agent answer`
`Semantic search retriever`	Retrieves code segments matching natural-language intent.	`Coding agent answer`
`Custom embedding model`	Embeds code and queries using training signals from prior agent sessions.	`Semantic search retriever`
`Indexing pipelines`	Build fast retrieval indexes for codebase segments.	`Custom embedding model`
`Agent session traces`	Training data showing searches, file opens, and ultimately useful code.	`LLM ranking teacher`
`LLM ranking teacher`	Ranks retrospectively useful content to train the embedding model.	`Custom embedding model`
`Coding agent answer`	Final answer or code change produced from retrieved context.	`Agent session traces`

Data flow

Agent session traces show what searches and file opens eventually led to the right code. An LLM ranks which content would have helped at each step, those rankings train a custom embedding model, and indexing pipelines make semantic code retrieval available to the agent alongside grep during future coding tasks.

“Our approach uses agent sessions as training data: when an agent works through a task, it performs multiple searches and opens files before finding the right code.”article, stated

“We provide these traces to an LLM, which ranks what content would have been most helpful at each step.”article, stated

Why this architecture

Cursor kept grep but added semantic retrieval because coding agents need both literal and meaning-based navigation. Training on agent traces aligns retrieval with how agents actually solve tasks instead of relying only on generic code similarity.

“One tool Cursor’s agent uses is semantic search, which retrieves segments of code matching natural language queries”article, stated

“This creates a feedback loop where the model can learn from how agents actually work through coding tasks, rather than relying on generic code similarity.”article, stated

Models used: custom embedding model · LLM ranking teacher · Composer · frontier coding models

Scale

Semantic search improved QA accuracy by an average of 12.5%.
“Achieving on average 12.5% higher accuracy in answering questions (6.5%–23.5% depending on the model).”article, stated
Online code retention increased by 0.3% overall.
“We see agent code retention increases by 0.3% when semantic search is available.”article, stated
Large codebases with 1,000 or more files saw a 2.6% code-retention increase.
“This effect increases to 2.6% on large codebases with 1,000 files or more.”article, stated
Without semantic search, dissatisfied follow-up requests increased by 2.2%.
“We observed a 2.2% increase in dissatisfied follow-up user requests when semantic search was not available.”article, stated

Alternatives considered (roads not taken)

Rely exclusively on grep and command-line search.

Rejected because: Semantic search significantly improved agent performance, especially on large codebases.

“While you could rely exclusively on grep and similar command-line tools for search, we've found that semantic search significantly improves agent performance”article, stated

Use generic code similarity as the retrieval objective.

Rejected because: Cursor trained from actual agent traces and LLM-generated helpfulness rankings.

“align its similarity scores with these LLM-generated rankings.”article, stated

Patterns

`rag-retrieval` (retrieval)

The coding agent retrieves relevant code segments from an indexed codebase before answering or editing.

“semantic search, which retrieves segments of code matching natural language queries, such as “where do we handle authentication?””article, stated

`hybrid-search` (retrieval)

Cursor uses semantic search alongside grep rather than replacing literal search.

“Our agent makes heavy use of grep as well as semantic search, and the combination of these two leads to the best outcomes.”article, stated

`retrieval-ranking` (retrieval)

An LLM ranks retrospectively helpful code content, and the embedding model learns from those rankings.

“We provide these traces to an LLM, which ranks what content would have been most helpful at each step.”article, stated

`eval-driven-development` (evaluation)

Cursor evaluates semantic search with Cursor Context Bench plus online A/B tests.

“The comparison looks at performance with two sets of available tools: one that includes semantic search and one that does not.”article, stated

Tradeoffs

Search tool mix

Chose Semantic search plus grep. over Grep alone..

Rationale: Semantic search improves outcomes, while grep remains heavily used for literal code navigation.

“Semantic search is currently necessary to achieve the best results, especially in large codebases.”article, stated

Training signal

Chose Agent session traces ranked by an LLM. over Generic code similarity training..

Rationale: Trace-based rankings teach the model what would have helped actual agents earlier in the task.

“This creates a feedback loop where the model can learn from how agents actually work through coding tasks”article, stated

Gotchas (what bit them)

Literal search alone misses natural-language intent.

Root cause: Users ask questions like where authentication is handled, which may not contain exact strings.

Resolution: Add semantic search that retrieves code segments by meaning.

“returning the right answer requires building an understanding of the codebase by reading files and searching for relevant information.”article, stated

Generic embedding training may not match agent workflows.

Root cause: The useful code for an agent is revealed by how the agent searches, opens files, and finally solves tasks.

Resolution: Use agent traces and an LLM ranking teacher to train the custom embedding model.

“By analyzing these traces, we can see in retrospect what should have been retrieved earlier in the conversation.”article, stated

Evaluation

Cursor evaluated semantic search through Cursor Context Bench offline across most-used models and through online A/B testing using code retention and dissatisfied follow-up requests.

Metrics: 12.5% average QA accuracy lift · 6.5%-23.5% accuracy lift depending on model · 0.3% code retention increase · 2.6% code retention increase on 1,000+ file codebases · 2.2% increase in dissatisfied requests without semantic search

“We maintain an evaluation dataset, Cursor Context Bench, focused on retrieving information in codebases with known correct answers.”article, stated

“We ran an A/B test where both groups used the same model, but one group's agent had access to semantic search while the other relied solely on traditional search tools like grep.”article, stated

Outcomes

Improved offline question-answering accuracy by 12.5% on average.
Improved code retention, especially on large codebases.
Reduced dissatisfied follow-up requests when semantic search was available.
Improved accuracy across all tested frontier coding models.

Reported effects include 12.5% average QA lift, 0.3% overall code-retention lift, 2.6% lift on codebases with 1,000+ files, and 2.2% more dissatisfied requests without semantic search.

“Increasing accuracy across all models we tested, including all frontier coding models.”article, stated