Context Engineering for Background Coding Agents at Scale

spotify · Media & Entertainment · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Background coding agents for enterprise-scale repository migrations

Spotify needed its background coding agent to automate software maintenance across many repositories while still producing mergeable PRs. The team found that the difficult part was not simply connecting an agent to code, but giving it enough context, prompts, and constrained tools to behave predictably at migration scale.

Constraints:

Migrations must work across thousands of repositories with heterogeneous build systems.
Context selection can overwhelm or starve the agent if users choose file scopes manually.
Multi-file changes can exceed simple loop turn limits and fill the context window.
Tooling must enable verification without introducing unpredictable behavior.
Evaluation of prompts and models was still mostly intuition-driven.

Architecture

Spotify moved from open-source agents and a custom loop to Claude Code with Anthropic Agent SDK, wrapped by disciplined context engineering: static migration prompts, tests as desired end state, constrained tools, a verify MCP tool for internal build systems, standardized Git access, and a Bash allowlist.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  workflow-context-providers["Workflow context providers"] --> migration-prompt["Migration prompt"]
  migration-prompt --> claude-code-agent["Claude Code agent"]
  claude-code-agent --> verify-tool["Verify tool"]
  verify-tool --> repository-build-systems["Repository build systems"]
  claude-code-agent --> git-tool["Git tool"]
  git-tool --> pull-request-workflow["Pull request workflow"]
  claude-code-agent --> bash-allowlist["Bash allowlist"]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Migration prompt`	Large, static prompt artifact encoding task goals, preconditions, examples, and desired end state.	`Claude Code agent`, `Workflow context providers`
`Claude Code agent`	Background coding agent that interprets higher-level migration goals and manages code edits.	`Migration prompt`, `Verify tool`, `Git tool`, `Bash allowlist`, `Pull request workflow`
`Verify tool`	MCP interface for running Spotify build systems, formatters, linters, and tests with summarized output.	`Claude Code agent`, `Repository build systems`
`Git tool`	Limited, standardized Git surface that exposes safe subcommands and standard commit behavior.	`Claude Code agent`, `Pull request workflow`
`Bash allowlist`	Small set of allowed shell utilities, including ripgrep, available to the agent.	`Claude Code agent`
`Repository build systems`	Existing in-house build/test systems invoked through the verify tool.	`Verify tool`
`Pull request workflow`	Production path for background-agent changes after verification.	`Claude Code agent`, `Git tool`
`Workflow context providers`	Humans or separate workflow agents that condense relevant internal/external context into prompts.	`Migration prompt`

Data flow

A migration request is turned into a prompt with relevant context and preconditions. Claude Code edits code, receives build/test feedback through the verify tool, uses limited Git/Bash capabilities, and opens PRs when the migration reaches its test-defined end state.

“The agent takes a few turns editing files based on the prompt, incorporating feedback from the build system with each turn.”article, stated

“because our agent needs to operate on thousands of existing repositories with very different build configurations. This also allows us to reduce noise in the tool output by summarizing logs into something more digestible for the agent.”article, stated

Why this architecture

The architecture optimizes for predictability. Claude Code improved task management, while static prompts and a limited tool surface let Spotify reason about, test, and constrain behavior across many repositories.

“we prefer to have larger static prompts, which are easier to reason about. You can version-control the prompts, write tests, and evaluate their performance.”article, stated

“We keep our background coding agent very limited in terms of tools and hooks so it can focus on generating the right code change from a prompt.”article, stated

Models used: Claude Code · Anthropic Agent SDK · Goose · Aider

Scale

Spotify applied Claude Code to about 50 migrations.
“Claude Code is our top-performing agent, which we’ve applied for about 50 migrations and the majority of the background agent PRs merged into production.”article, stated
The migrations target thousands of repositories.
“writing good prompts — and verifying that the agent did the right thing — becomes significantly more difficult if you want to apply a change over thousands of repos.”article, stated
The cited system merged thousands of PRs across hundreds of repositories.
“Spotify built a large scale system that used Claude Agent SDK to merge thousands of PRs across hundreds of repositories.”article, stated

Alternatives considered (roads not taken)

Open-source agents like Goose and Aider

Rejected because: They could explore code from a prompt, but Spotify could not make them reliably produce mergeable PRs for migrations.

“But we quickly ran into issues when we tried to scale these agents to our migration use case. It was difficult to get them to reliably produce mergeable PRs.”article, stated

Homegrown LLM loop

Rejected because: It handled small edits but struggled with complex multi-file changes, context-window limits, and rigid instructions.

“Our homegrown agentic loop had key challenges — it required overly rigid instructions and struggled with complex, multistep edits.”article, stated

Expose more dynamic MCP context tools

Rejected because: Dynamic tools help with ambiguity but make behavior less testable and predictable.

“The more tools you have, the more dimensions of unpredictability you introduce.”article, stated

Patterns

`prompt-versioning` (prompt-operations)

Spotify uses larger static migration prompts because they can be version-controlled, tested, and evaluated.

“we prefer to have larger static prompts, which are easier to reason about. You can version-control the prompts, write tests, and evaluate their performance.”article, stated

`eval-driven-development` (evaluation)

Prompts define verifiable end states and the agent iterates using feedback from formatters, linters, tests, and build systems.

““Make this code better” is not a good prompt. The agent needs a verifiable goal so it can iterate on a solution as it goes.”article, stated

`tool-use-orchestration` (orchestration)

The production agent gets a small, intentionally designed tool surface rather than arbitrary code-search or documentation tools.

“Notably, we don’t currently have code search or documentation tools exposed to our agent. We instead ask users to condense relevant context into the prompt up front”article, stated

`guardrails-io-filtering` (governance)

Git and Bash are available only through limited, standardized interfaces to reduce unsafe or unpredictable operations.

“This allows us to selectively expose git subcommands (e.g., never push or change origin) and standardize others (e.g., setting the committer and using standardized commit message formats).”article, stated

Tradeoffs

Agent selection

Chose Claude Code over open-source agents and a custom loop.

Rationale: Claude Code let users describe outcomes, manage tasks dynamically, and avoid some context-window limitations.

“To solve this, we turned to Claude Code, which allowed us to use more-natural task-oriented prompts, and we benefited from its built-in ability to manage Todo lists and spawn subagents efficiently.”article, stated

Tooling strategy

Chose limited tools and hooks over a broader MCP-enabled context-fetching toolset.

Rationale: Tool limitation made the agent more predictable and focused on the intended code change.

“We keep our background coding agent very limited in terms of tools and hooks so it can focus on generating the right code change from a prompt.”article, stated

Migration task granularity

Chose one change at a time over combining related changes into one elaborate prompt.

Rationale: Combined changes are more likely to exhaust context or produce partial results.

“It can be convenient for users to combine several related changes into one elaborate prompt, but it’s more likely to get the agent in trouble because it will exhaust its context window or it will deliver a partial result.”article, stated

Gotchas (what bit them)

Manual file-scope selection failed at migration scale.

Root cause: Broad git-grep scopes overwhelmed the context window, while narrow scopes hid needed files.

Resolution: Use better agent task/context management and provide condensed relevant context up front.

“Picking a too-broad search pattern — like git grep -l '*.java' — could easily overwhelm the context window of the LLM. Conversely, picking a too-limited search pattern would not allow the agent to solve the problem”article, stated

Simple loops lost the task on complex multi-file edits.

Root cause: The custom agent filled its context and ran out of turns on cascading changes.

Resolution: Adopt Claude Code for dynamic task management and longer operations.

“For this use case, our loop often ran out of turns. It also tended to get lost when it filled up its context window, forgetting the original task after a few turns.”article, stated

Prompt/model quality was not yet systematically measurable.

Root cause: The team was still evolving prompts by trial and error and lacked structured evaluation of prompt/model choices.

Resolution: The post points to feedback loops as the next step for more predictable results.

“we are still flying mostly by intuition. Our prompts evolve by trial and error. We don’t yet have structured ways to evaluate which prompts or models perform best.”article, stated

Evaluation

The agent is evaluated operationally through tests, linters, formatters, build feedback, mergeability, and PR production outcomes, while the team acknowledges the need for better structured prompt/model evaluation.

Metrics: formatters, linters, and tests · PR mergeability · about 50 migrations · thousands of PRs across hundreds of repositories

“* A “verify” tool that runs formatters, linters, and tests. We found it easier to encode how to invoke our in-house build systems in an MCP”article, stated

“And even if we make it to a merged PR, how do we know if it actually solved the original problem?”article, stated

Outcomes

Claude Code became Spotify's top-performing agent for this background migration workflow.
The approach was applied to about 50 migrations, with most background-agent PRs merged into production.
The system is described as merging thousands of PRs across hundreds of repositories.

This is a large-scale repository migration system: thousands of repos are the target setting, and the cited outcome is thousands of PRs across hundreds of repositories.

“Claude Code is our top-performing agent, which we’ve applied for about 50 migrations and the majority of the background agent PRs merged into production.”article, stated

“Spotify built a large scale system that used Claude Agent SDK to merge thousands of PRs across hundreds of repositories.”article, stated