Long-Running Agent Harness for Multi-Context Software Development

anthropic · Tech · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Long-running autonomous software development agents

Anthropic needed coding agents to make reliable progress on complex software projects across multiple context windows without losing memory, duplicating work, or declaring unfinished applications complete.

Constraints:

• Each new agent session begins with no memory of previous sessions once the context window is exhausted.
• Compaction alone did not preserve sufficiently clear instructions for the next agent instance.
• Agents tended to one-shot too much work, leave half-finished features, or mark features complete without end-to-end validation.
• The harness needed durable handoff artifacts and a clean code state suitable for subsequent sessions.

Architecture

Anthropic built a two-phase harness for long-running coding agents. An initializer agent creates durable environment artifacts such as init.sh, a progress log, git history, and a JSON feature list with testable requirements. Subsequent coding-agent sessions orient themselves by reading those artifacts, choose one incomplete feature, run the app and a smoke test, implement incrementally, validate with Puppeteer browser automation, commit progress, and update the handoff log.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  Initializer["Initializer agent"] --> Feature_JSON["Feature list JSON"]
  Initializer --> Progress_log["Progress log"]
  Initializer --> Git_history["Git history"]
  Initializer --> Init_script["init.sh"]
  Feature_JSON --> Coding_agent["Coding agent"]
  Progress_log --> Coding_agent
  Git_history --> Coding_agent
  Init_script --> Coding_agent
  Init_script --> Dev_server["Development server"]
  Coding_agent --> Codebase["Codebase"]
  Codebase --> Git_history
  Codebase --> Dev_server
  Coding_agent --> Puppeteer["Puppeteer MCP"]
  Puppeteer --> Dev_server
  Coding_agent --> Progress_log

Components

Data flow

The user prompt enters the initializer session, which expands it into a structured feature backlog and environment scripts. Later coding sessions read git logs, progress notes, and the feature list, start the development server through init.sh, run basic end-to-end checks, implement one feature, use browser automation to validate behavior, and persist a commit plus progress update for the next fresh-context session.

[Assistant] I'll start by getting my bearings and understanding the current state of the project."

— article, stated

Why this architecture

The harness moves memory out of transient model context and into durable software-engineering artifacts. JSON feature lists prevent one-shotting and premature completion, progress logs and git history orient fresh sessions, and browser automation closes the gap between code edits and user-visible behavior.

Models used: Claude Agent SDK · Opus 4.5 · Claude

Scale

• The target tasks can span hours or days.
  “developers are increasingly asking them to take on complex tasks requiring work that spans hours, or even days.”article, stated
• The claude.ai clone initializer created more than 200 testable features.
  “For a "clone of claude.ai" project, the initializer created over 200 features with detailed test steps, all initially marked with a passes: false status.”summary, stated
• Each coding session begins by reading progress, feature, and git artifacts before implementation.
  “Read the git logs and progress files to get up to speed on what was recently worked on.”article, stated

  “Read the features list file and choose the highest-priority feature that’s not yet done to work on.”article, stated

Alternatives considered (roads not taken)

Rely on context compaction alone

Rejected because: Compaction did not reliably preserve enough clear instructions for the next fresh agent session.

Let agents attempt the whole application in one pass

Rejected because: One-shotting exhausted context and left half-implemented undocumented features for later sessions.

Use Markdown for the feature list

Rejected because: The model was more likely to inappropriately modify or overwrite Markdown, while JSON proved more robust.

Test only with code inspection, unit tests, or curl

Rejected because: Those checks missed whether web features actually worked from a user perspective.

Patterns

context-engineering (agent-architecture)

Anthropic engineered what survives across context windows by externalizing session memory into JSON feature lists, progress logs, git history, startup scripts, and onboarding steps.

agent-memory (state)

Progress files and git history act as persistent memory that fresh agent sessions read before selecting new work.

structured-output-contract (coordination)

The initializer creates a JSON feature list with descriptions, test steps, and pass/fail status, and coding agents are constrained to update only status.

tool-use-orchestration (agent-architecture)

The coding agent uses shell commands, file reads, git, startup scripts, and Puppeteer browser automation as part of its session routine.

eval-driven-development (evaluation)

Features are only marked passing after browser-based validation and a basic end-to-end smoke test at the start of each session.

Tradeoffs

Use a two-prompt harness

Chose initializer and coding-agent prompts in the same underlying harness over one generic prompt for all sessions.

Rationale: Different session phases needed different behavior: environment setup first, incremental feature work thereafter.

Persist feature state in JSON

Chose JSON feature list with pass/fail fields over Markdown feature tracking.

Rationale: JSON was less likely to be inappropriately changed or overwritten by the model.

Require session startup overhead

Chose read artifacts, start server, and smoke-test before new work over immediately coding the next feature.

Rationale: The overhead prevents agents from building on a broken state and saves time otherwise spent guessing prior work.

Use a general-purpose coding agent with phase-specific prompts

Chose same system and tools with different initial prompts over fully separate specialized agents.

Rationale: The current demo used prompt specialization, while the source leaves true multi-agent specialization as an open research question.

Gotchas (what bit them)

Compaction does not automatically make long-running agents reliable.

Root cause: Fresh sessions can still lack clear prior-work instructions and have to reconstruct partially implemented state.

Resolution: Use durable artifacts such as progress logs, feature JSON, git history, and startup scripts as the cross-session memory layer.

Agents overreach when the work is not decomposed into explicit feature units.

Root cause: A high-level prompt encourages the model to attempt too much in one context window.

Resolution: Have the initializer expand the prompt into a comprehensive feature list and require coding agents to work one feature at a time.

Agents may mark features complete without user-visible validation.

Root cause: Code-level tests and curl checks can miss broken end-to-end browser behavior.

Resolution: Prompt the agent to use Puppeteer MCP and test web features as a human user would before marking completion.

Browser automation still leaves blind spots.

Root cause: Tool and vision limitations prevent Claude from seeing some browser-native UI states such as alert modals.

Resolution: Keep testing limitations explicit and avoid assuming browser automation catches every class of bug.

Evaluation

Anthropic evaluated the harness qualitatively through web-app building experiments, feature pass/fail tracking, git/progress handoffs, startup smoke tests, and browser automation. The article does not provide quantitative success rates, cost, or bug metrics.

Metrics: feature pass/fail status in JSON · successful basic chat smoke test at session start · browser automation screenshots and user-flow validation · clean mergeable code state after each session · no quantitative benchmark metrics reported

Outcomes

• The harness enabled Claude Agent SDK sessions to make incremental progress across many context windows.
• Feature JSON, progress logs, git history, and init.sh reduced the need for fresh agents to infer prior state.
• Puppeteer MCP browser testing improved the agent's ability to find user-visible bugs.
• The article leaves open questions about quantitative metrics, security, cost, and generalization beyond full-stack web apps.

The reported scale is long-horizon software work across hours or days and over 200 feature requirements in a claude.ai clone example, not a public production throughput metric.