Building a Production Coding Agent Model with Speed and Intelligence

cursor · Tech · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production coding-agent model training for an AI IDE

Cursor needed a coding-agent model that was fast enough for synchronous developer flow while still smart enough to solve real software engineering tasks, use IDE tools, and improve from production-like reinforcement-learning rollouts.

Constraints:

• developers disliked the middle ground where coding agents took 10 to 20 minutes but were not autonomous enough to leave alone
• the model had to balance near-frontier coding ability with much faster token generation
• training needed to mimic Cursor's production IDE environment, including real tool formats and tool responses
• rollouts could consume hundreds of thousands to millions of tokens and hundreds of tool calls with highly variable durations
• large mixture-of-experts training had to run across thousands of GPUs without making training prohibitively slow
• environment infrastructure had to support bursty RL workloads, secure sandbox execution, and many cloud VMs

Architecture

Cursor built Composer as a specialized coding-agent model trained with reinforcement learning in a production-like IDE environment. The system combines a trainer for model updates, a Ray-backed inference server for rollouts and tool execution, and environment servers/cloud VMs that load user code, run tools, edit files, and mirror Cursor's production agent surface. Composer is optimized for fast token generation, parallel tool calls, semantic code search, and agent behavior that searches and reads before editing.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  cursor-ide[Cursor IDE] --> cursor-backend[Cursor backend]
  cursor-backend --> composer-model[Composer model]
  composer-model --> tool-interface[Tool interface]
  tool-interface --> semantic-search-tool[Semantic search tool]
  tool-interface --> environment-servers[Environment servers]
  environment-servers --> cloud-vm-fleet[Cloud VM fleet]
  composer-model --> inference-server[Inference server]
  inference-server --> environment-servers
  inference-server --> training-server[Training server]
  training-server --> custom-kernel-training-stack[Custom kernel training stack]
  custom-kernel-training-stack --> inference-server

Components

Data flow

In product use, a Cursor user submits a query, Composer decides which IDE tools to call, reads/searches/edits/runs commands, and returns a result quickly enough to keep the developer in the loop. In training, the inference server runs multiple rollouts in Cursor-like environments, scores outputs, sends advantages back to the trainer, receives updated weights, and load-balances uneven rollouts across threads, processes, and cloud VM environments.

Why this architecture

Cursor built a specialized model and production-matching RL stack because a generic frontier-only strategy left developers waiting, while a mocked training environment would not teach the agent to use Cursor's real tools. The co-designed IDE, cloud-agent infrastructure, semantic search, custom kernels, and rollout load balancing let the model learn fast, practical tool use in conditions close to actual product use.

Models used: Cursor Composer · Tab autocomplete model · custom embedding model · large mixture-of-experts model · GPT 5.1 Codex · Sonnet 4.5

Scale

• Composer targets models at similar intelligence while being about four times more efficient at token generation.
  “shines is it's about four times more efficient at token generation than models at a similar level of intelligence. So we're trying to mesh speed as well as intelligence. So why did we build this model? I mean obviously cursor has an IDE. Why are we getting into the model space? Why do we”transcript, stated
• The agent tool surface has about ten tools, with five core tools highlighted.
  “to make a series of tool calls. And our agent has about 10 tools give or take, but we're going to focus on five here. So reading files, editing files, searching your codebase, looking at lints, and then also running terminal or shell commands. And the agent then is”transcript, stated
• Composer training uses a large mixture-of-experts model parallelized across thousands of GPUs.
  “Um, in this case with composer, we're training a large mixture of experts model and it's being parallelized across thousands of GPUs and if we don't speed that up, it's going to take forever to train the thing. So, we want to make it really fast and match the training and”transcript, stated
• Real-world rollouts may involve hundreds of thousands to millions of tokens and hundreds of tool calls.
  “models are going to use hundreds of thousands to millions of tokens. They're going to make hundreds of different tool calls”transcript, stated
• Custom low-precision kernels made the mixture-of-experts layer about three and a half times faster on Nvidia Blackwell chips.
  “for the mixture of experts layer was about three and a half times faster uh a speed up on Nvidia Blackwell chips”transcript, stated
• Cursor described the training/cloud-agent infrastructure as having many clusters and hundreds of thousands of VMs.
  “we have many different clusters, hundreds of thousands of VMs here and you can see behind me one of the internal dashboards we built uh with composer actually to visualize uh all of the different VMs in the fleet.”transcript, stated

Alternatives considered (roads not taken)

Use only the most powerful frontier model for coding-agent work

Rejected because: The middle latency band broke flow; Cursor wanted either very fast synchronous work or long autonomous background execution, so Composer optimized the fast foreground lane.

Mock or simulate the Cursor environment loosely during training

Rejected because: Cursor wanted exact tool formats, tool responses, and environment behavior so the model would transfer to production use.

Run coding tools serially by default

Rejected because: Parallel tool calls made Composer feel faster and helped reach a checkpoint developers would use daily.

Wait for uneven RL rollouts to finish together

Rejected because: Rollouts finish at different times, so the inference server load-balances across threads and processes to reduce idle time.

Build a general-intelligence model

Rejected because: Cursor focused RL, data, and compute on the narrower objective of excellent coding models.

Patterns

fine-tuning-adaptation (model-training)

Cursor used reinforcement learning rollouts in a production-like coding environment to adapt Composer into a specialized coding-agent model.

tool-use-orchestration (agent-architecture)

Composer chooses among file reading, editing, search, lint, and shell tools, including deciding when to call them serially or in parallel.

eval-driven-development (evaluation)

Cursor judged Composer checkpoints against internal benchmarks based on its own repositories and whether its developers would use the checkpoint daily.

latency-optimization (production-performance)

Composer trades a small amount of absolute frontier intelligence for much faster generation and parallel tool use so coding agents can remain foreground and flow-preserving.

hybrid-search (agent-context)

Cursor gives Composer semantic codebase search from a custom embedding model so the agent can retrieve likely files before editing.

Tradeoffs

Optimize Composer for fast foreground coding rather than maximum frontier intelligence

Chose near-frontier speed-oriented model over the slowest most capable frontier model for every coding task.

Rationale: Cursor wanted to avoid the semi-async latency valley and keep developers in flow while preserving enough intelligence for real work.

Train in production-like environments

Chose cloud VM environments that mirror Cursor production over mocked or simplified tool simulations.

Rationale: Production fidelity gives the model real tool affordances and helps transfer learned behavior to the product.

Use custom low-precision kernels

Chose custom kernels for low-precision mixture-of-experts training over slower standard training implementation.

Rationale: Cursor needed to speed up training and make updated weights easier to ship back to inference.

Let Composer call tools in parallel

Chose parallel tool execution when useful over serial file-by-file tool use.

Rationale: Parallel reads and searches reduce end-to-end wait time and make Composer feel faster to developers.

Specialize on coding instead of broad AGI

Chose high-quality coding data and domain-specific RL over general-intelligence model training.

Rationale: Cursor's product goal was excellent coding agents, and RL worked well for the specific task.

Gotchas (what bit them)

A fast model was not useful until it was smart enough for daily coding.

Root cause: The early Cheetah/Slug-style model had appealing speed but did not yet meet users' intelligence threshold for regular work.

Resolution: Cursor built internal production-like benchmarks and RL training until developers would use the checkpoint every day.

RL training quality depended on matching production tool behavior.

Root cause: Training with different tool formats or responses would create a gap from how Composer behaves inside Cursor.

Resolution: Cursor built environment servers and cloud VMs that closely match the production Cursor environment.

Rollout workloads had uneven lengths and could waste compute.

Root cause: Different rollouts use different numbers of tokens, tool calls, package installs, and command durations.

Resolution: The inference server load-balances across threads and processes instead of waiting for the slowest rollout.

The model initially edited too aggressively.

Root cause: Early agent behavior made too many or unnecessary edits before gathering enough context.

Resolution: More RL training improved search/read-before-edit behavior.

Model research turned into an infrastructure problem.

Root cause: Fast large-model training, spiky rollouts, environment fidelity, VM orchestration, and tool execution all required systems work.

Resolution: Cursor co-designed the IDE, cloud-agent infrastructure, inference server, training stack, and custom kernels with the model work.

Evaluation

Cursor evaluated Composer with internal benchmarks based on its own repos and workflows, checkpoint usefulness for daily product development, rollout scoring during RL, benchmark comparisons to open-source/frontier coding models, user feedback from early releases, and behavioral observations such as parallel tool use and reduced unnecessary edits.

Metrics: internal benchmark performance on Cursor repositories · developer daily-use threshold for model checkpoints · token generation efficiency versus similar-intelligence models · rollout scores and advantages · performance against open-source and frontier coding models · parallel tool-use effectiveness · frequency of unnecessary edits

Outcomes

• Cursor released Composer in Cursor 2.0.
• Composer reached performance better than the best open-source models and close to recent frontier coding models while generating tokens much faster.
• RL improved the model from roughly best-open-model performance toward frontier coding-agent performance.
• Composer learned to use semantic search, read files before editing, and call tools in parallel.
• Cursor's own team used Cursor and Composer-style workflows to accelerate research and product development.

The training setup involved a large mixture-of-experts model parallelized across thousands of GPUs, rollouts with hundreds of thousands to millions of tokens and hundreds of tool calls, custom kernels with about a 3.5x MoE speedup on Blackwell, and many clusters with hundreds of thousands of VMs.