Variable Aggression Code Autocomplete with Fine-Tuned LLMs

windsurf · Tech · 2026 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: AI code autocomplete

Windsurf needed to improve code autocomplete by saving more real developer keystrokes rather than merely optimizing acceptance rate on safe, short suggestions.

Constraints:

Autocomplete runs on every keypress, so prompt length, latency, and cost matter.
Acceptance rate can reward trivial predictions that add little value.
Users have heterogeneous preferences for how much code the model should predict.
RL reward functions can be hacked by code models optimizing for runnable code rather than developer intent.
Incorrect aggressive suggestions may impose cognitive overhead.

Architecture

Windsurf rebuilt Tab v2 around context engineering, RL training, and user-selectable aggression. The team shortened the autocomplete prompt by 76%, removed irrelevant Cascade/tool context, added useful related-file and cursor context, rebuilt training data around production context, trained models with RL for different aggression levels, evaluated prediction frequency, acceptance, characters per prediction, and accepted characters, and exposed aggression as a configurable product parameter.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  editor-keypress-stream["Editor keypress stream"] --> context-engineering-layer["Context engineering layer"]
  context-engineering-layer --> tab-v2-model["Tab v2 model"]
  tab-v2-model --> aggression-controller["Aggression controller"]
  aggression-controller --> user-preference-settings["User preference settings"]
  tab-v2-model --> rl-training-pipeline["RL training pipeline"]
  rl-training-pipeline --> evaluation-and-ab-testing["Evaluation and AB testing"]
  evaluation-and-ab-testing --> user-preference-settings

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Editor keypress stream`	Triggers autocomplete predictions during coding.	`Context engineering layer`
`Context engineering layer`	Builds compact, task-specific model context from cursor, related files, and prompt instructions.	`Editor keypress stream`, `Tab v2 model`
`Tab v2 model`	Generates autocomplete, multi-line, navigation, and import predictions.	`Context engineering layer`, `Aggression controller`
`Aggression controller`	Adjusts prediction length/risk according to user-selected aggression level.	`Tab v2 model`, `User preference settings`
`User preference settings`	Let developers choose the autocomplete aggression level.	`Aggression controller`
`RL training pipeline`	Trains autocomplete behavior and aggression levels against reward/evaluation data.	`Tab v2 model`, `Evaluation and AB testing`
`Evaluation and AB testing`	Measures prediction frequency, acceptance rate, characters per prediction, accepted characters, and user interviews.	`RL training pipeline`, `User preference settings`

Data flow

On each editor keypress, Tab gathers optimized code context, related files, cursor information, and prompt instructions, sends them to a fine-tuned autocomplete model, and returns code predictions at a user-selected aggression level. Production metrics and interviews feed evaluation, RL reward design, A/B testing, and future model improvements.

3 supporting quotes

“The original Tab feature, launched one year prior to this update, provided four predictive capabilities triggered on every keypress”summary, stated

“Through systematic prompt engineering, they achieved a 76% reduction in system prompt prefix length.”summary, stated

“Windsurf made the strategic decision to parameterize aggression and let users choose their preferred level.”summary, stated

Why this architecture

Windsurf optimized for total accepted code rather than raw acceptance because safe, short suggestions can look good in metrics while saving little work. Aggression became configurable because the same model behavior was too bold for some developers and too timid for others.

“Windsurf reframed the optimization goal as maximizing total keystrokes saved for developers.”summary, stated

“the exact same model was described as "too aggressive" by some users and "not bold enough" by others.”summary, stated

Models used: Fine-tuned code autocomplete LLM · Reinforcement learning training pipeline

Scale

The team analyzed billions of production predictions.
“After analyzing billions of predictions in production”summary, stated
System prompt prefix length was reduced by 76%.
“they achieved a 76% reduction in system prompt prefix length.”summary, stated
Characters per prediction increased by 54% on average.
“The headline result was an average 54% increase in characters per prediction while maintaining comparable acceptance rates.”summary, stated
Accepted code improved by 25-75% depending on aggression level.
“The final results show the impact of this approach: 25-75% more accepted code depending on the chosen aggression level.”summary, stated

Alternatives considered (roads not taken)

Optimize raw acceptance rate

Rejected because: Acceptance rate can be gamed by making only obvious, low-value suggestions.

“acceptance rate can be artificially inflated by only making predictions when confidence is extremely high”summary, stated

Reuse Cascade's system prompt

Rejected because: Cascade context was unoptimized and poisoned the autocomplete task.

“The team inherited a system prompt that had been copied from their Cascade product, which was completely unoptimized for the autocomplete use case.”summary, stated

“The team removed unused tool call prompts and examples that were poisoning the context”summary, stated

Choose one aggression level for all users

Rejected because: User interviews showed the same behavior felt too aggressive to some and not bold enough to others.

“optimizing for the median user would leave significant value on the table.”summary, stated

Patterns

`context-engineering` (prompting)

Windsurf removed irrelevant prompt/tool context, shortened the prompt by 76%, and added task-specific cursor/related-file context.

“At production scale, the team had to make every token count. They brought Cognition's internal tooling to analyze their data pipeline and identify smaller optimizations.”summary, stated

“Specific improvements included fixing how they expressed the cursor position to the model and adding related file context”summary, stated

`fine-tuning-adaptation` (model training)

Tab v2 uses RL-trained autocomplete behavior and aggression levels informed by production context data.

“The team then applied reinforcement learning (RL) to train the new Tab model. Interestingly, this represented the first real production usage of the RL infrastructure they would later use for SWE 1.5, another product.”summary, stated

“This bidirectional flow—using production insights to improve training, and training improvements to enhance production performance—is a hallmark of mature LLMOps practices.”summary, stated

`eval-driven-development` (product metrics)

The team redefined success around keystrokes saved, A/B tested Pareto improvements, and added qualitative interviews to catch preference variance.

“They explicitly held prediction frequency constant during optimization to avoid gaming the acceptance rate metric.”summary, stated

“The evaluation strategy combined automated metrics with extensive A/B testing on real users. The team tracked multiple dimensions of performance: prediction frequency, acceptance rate, characters per prediction, and ultimately total accepted characters (the product of the previous metrics).”summary, stated

`data-quality-loop` (training feedback)

Production data, reward-hack discoveries, and interviews feed model and reward-function improvements.

“Addressing these reward hacks required iterating on both the reward function and training loops. The team had to "better capture the concept of high quality aggression" by inventing new evaluations across their training data.”summary, stated

Tradeoffs

User-selectable aggression

Chose Let developers choose prediction boldness over Optimize one model setting for the median user.

Rationale: The same autocomplete behavior creates different value and annoyance for different users.

“Rather than choosing a single aggression level that optimizes for the average user, Windsurf made the strategic decision to parameterize aggression”summary, stated

Shorter task-specific context

Chose Remove irrelevant prompt/tool instructions over Reuse a broad product prompt from Cascade.

Rationale: Shorter context reduced cost/latency and improved autocomplete-specific performance.

“This seemingly simple change had cascading benefits: reduced API costs, faster time-to-first-token, and improved model performance.”summary, stated

Gotchas (what bit them)

Acceptance rate rewards trivial suggestions.

Root cause: A model can improve acceptance by predicting only obvious, short completions.

Resolution: Windsurf optimized accepted characters/keystrokes saved while holding prediction frequency constant.

“The insight here is subtle but important for LLMOps practitioners: the true business value comes from helping developers write more code faster”summary, stated

Code RL reward functions can learn the wrong behavior.

Root cause: Standard code-model post-training assumes code should be runnable, but autocomplete often happens in incomplete intermediate states.

Resolution: Windsurf invented new evaluations to capture high-quality aggression instead of runnable-code restoration.

“the model learning to predict "def" when a user types "def" in Python, simply to restore the file to a runnable state.”summary, stated

“developers are constantly in intermediate states where code doesn't run, and restoring to a runnable state may not reflect their intent.”summary, stated

Aggregate A/B metrics can hide user preference heterogeneity.

Root cause: Different developers want different autocomplete boldness.

Resolution: Windsurf exposes aggression levels for users to choose.

“aggregate metrics can mask heterogeneous preferences within your user base.”summary, stated

Evaluation

Windsurf evaluates autocomplete with prediction frequency, acceptance rate, characters per prediction, total accepted characters, A/B tests, and qualitative interviews.

Metrics: 76% prompt prefix reduction · 54% average increase in characters per prediction · 25-75% more accepted code · Prediction frequency · Acceptance rate

“The team tracked multiple dimensions of performance: prediction frequency, acceptance rate, characters per prediction, and ultimately total accepted characters”summary, stated

“Through A/B testing, they validated that Tab v2 represented a Pareto improvement over v1—meaning it was better across all meaningful dimensions, not just trading off one metric for another.”summary, stated

Outcomes

System prompt prefix length fell by 76%.
Characters per prediction increased by 54% on average.
Accepted code increased by 25-75% depending on aggression level.
User preference heterogeneity became an explicit product control.

The system was rebuilt after analyzing billions of production predictions, with autocomplete triggered on every keypress.

“Windsurf developed Tab v2, an AI-powered code autocomplete system that addresses the challenge of balancing prediction frequency, accuracy, and code length”summary, stated

“The final results show the impact of this approach: 25-75% more accepted code depending on the chosen aggression level.”summary, stated