AI-Powered Background Coding Agents for Large-Scale Software Maintenance

spotify · Media & Entertainment · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Media & Entertainment

Spotify needed to scale complex code migrations and maintenance across many repositories after deterministic Fleet Management scripts no longer covered moderately complex changes.

Constraints:

Architecture

Spotify added background coding agents to Fleet Management: developers provide natural-language migration instructions, agents edit repositories, MCP tooling runs formatting and linting, LLM judges evaluate diffs, and traces/logs support debugging.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  fleet-management-platform["Fleet Management platform"]
  interactive-prompt-gathering-agent["Interactive prompt-gathering agent"]
  agent-instruction-runner["Agent instruction runner"]
  coding-agents["Coding agents"]
  mcp-tooling["MCP tooling"]
  diff-evaluation["Diff evaluation"]
  mlflow-traces["MLflow traces"]
  pull-request-creation["Pull request creation"]
  fleet-management-platform --> agent-instruction-runner
  interactive-prompt-gathering-agent --> agent-instruction-runner
  agent-instruction-runner --> coding-agents
  agent-instruction-runner --> mcp-tooling
  mcp-tooling --> coding-agents
  coding-agents --> diff-evaluation
  mcp-tooling --> diff-evaluation
  diff-evaluation --> mlflow-traces
  diff-evaluation --> pull-request-creation
  pull-request-creation --> fleet-management-platform
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Fleet Management platform Existing migration platform for deterministic and large-scale repository maintenance. Agent instruction runner
Interactive prompt-gathering agent Helps developers specify the maintenance task before background execution. Agent instruction runner
Agent instruction runner Delegates natural-language migration prompts to coding agents. Coding agents, MCP tooling
Coding agents Make repository changes for complex migrations and maintenance tasks. Diff evaluation, Pull request creation
MCP tooling Runs local formatting, linting, and project-specific checks. Coding agents, Diff evaluation
Diff evaluation Uses LLM judges and traces to assess generated changes. Pull request creation, MLflow traces
MLflow traces Captures agent traces and logs for debugging and observability. Diff evaluation
Pull request creation Submits generated maintenance changes to teams. Fleet Management platform

Data flow

The CLI delegates prompts to agents, runs formatting and linting with MCP, evaluates diffs with LLM judges, uploads logs, and captures MLflow traces.

“This CLI can delegate executing a prompt to an agent, run custom formatting and linting tasks using local Model Context Protocol (MCP), evaluate a diff using LLMs as a judge, upload logs to Google Cloud Platform (GCP), and capture traces in MLflow.”article, stated
“This CLI can delegate executing a prompt to an agent, run custom formatting and linting tasks using local Model Context Protocol (MCP), evaluate a diff using LLMs as a judge, upload logs to Google Cloud Platform (GCP), and capture traces in MLflow.”article, stated

Why this architecture

The design supplements deterministic migration scripts because moderately complex migrations need instruction-following agents, despite slower and less predictable output.

“We replaced deterministic migration scripts with an agent that takes instructions from a prompt.”article, stated
“Performance is a key consideration, as agents can take a long time to produce a result, and their output can be unpredictable.”article, stated

Scale

Alternatives considered (roads not taken)

Deterministic migration scripts for every maintenance task

Rejected because: Spotify replaced scripts with prompt-driven agents where changes became too complex for declarative migrations alone.

“We replaced deterministic migration scripts with an agent that takes instructions from a prompt.”article, stated

Manual prompt collection for every migration

Rejected because: Spotify uses an interactive agent to gather task information before background execution.

“They first talk to an interactive agent that helps to gather information about the task at hand.”article, stated

Trust generated diffs without a judge

Rejected because: The workflow evaluates diffs using LLM judges before teams merge generated PRs.

“This CLI can delegate executing a prompt to an agent, run custom formatting and linting tasks using local Model Context Protocol (MCP), evaluate a diff using LLMs as a judge, upload logs to Google Cloud Platform (GCP), and capture traces in MLflow.”article, stated

Patterns

tool-use-orchestration (orchestration)

Spotify wraps coding agents in a CLI that coordinates prompt execution, deterministic tooling, logs, and traces.

“This CLI can delegate executing a prompt to an agent, run custom formatting and linting tasks using local Model Context Protocol (MCP), evaluate a diff using LLMs as a judge, upload logs to Google Cloud Platform (GCP), and capture traces in MLflow.”article, stated

llm-as-judge (evaluation)

The migration workflow uses LLM-based judging to evaluate generated diffs before they become pull requests.

“This CLI can delegate executing a prompt to an agent, run custom formatting and linting tasks using local Model Context Protocol (MCP), evaluate a diff using LLMs as a judge, upload logs to Google Cloud Platform (GCP), and capture traces in MLflow.”article, stated

multi-agent-orchestration (orchestration)

For ad hoc work, an interactive agent gathers task context and hands a refined prompt to the background coding agent.

“They first talk to an interactive agent that helps to gather information about the task at hand.”article, stated

Tradeoffs

Replace deterministic migration-script declaration with natural-language agent instructions for complex migrations.

Chose an agent that takes instructions from a prompt over deterministic migration scripts for every transformation.

Rationale: The article says deterministic automation handled toil, but moderately complex migrations still required many human hands.

“We replaced deterministic migration scripts with an agent that takes instructions from a prompt.”article, stated

Gotchas (what bit them)

Background coding agents introduce unpredictable output, long runtimes, and high compute cost.

Root cause: Code agents operate autonomously over nontrivial repositories and can take substantial time and compute to produce a result.

Resolution: Spotify calls out guardrails, sandboxing, and cost management as necessary controls.

“Performance is a key consideration, as agents can take a long time to produce a result, and their output can be unpredictable.”article, stated

Evaluation

Generated diffs are evaluated with LLMs as judges and traced through MLflow.

“This CLI can delegate executing a prompt to an agent, run custom formatting and linting tasks using local Model Context Protocol (MCP), evaluate a diff using LLMs as a judge, upload logs to Google Cloud Platform (GCP), and capture traces in MLflow.”article, stated

Outcomes

The workflow has expanded from fleet migrations to ad hoc background coding tasks.

“To date, our agents have generated more than 1,500 pull requests that teams across Spotify have merged into our production codebase.”article, stated