Autonomous Multi-Agent System for Complex Software Development

factory · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Complex software development with agents

Factory needed a way for agentic software development to handle broad projects that exceed a single context window and become unreliable when one agent accumulates irrelevant or self-confirming context.

Constraints:

Architecture

Factory's Missions system decomposes large software goals into focused roles and artifacts. An orchestrator clarifies requirements, writes a validation contract, decomposes the mission into features and milestones, and delegates implementation to fresh workers. Workers receive narrow feature specs, write tests first, implement, and hand off. Fresh validators review worker output and black-box behavior against the contract. Findings return to the orchestrator as targeted fix features until milestone validation passes or the mission halts for the user.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  user-goal[User goal] --> orchestrator[Orchestrator]
  orchestrator --> validation-contract[Validation contract]
  orchestrator --> feature-list[Feature list]
  feature-list --> programmatic-runner[Programmatic runner]
  programmatic-runner --> workers[Workers]
  workers --> shared-artifacts[Shared artifacts]
  programmatic-runner --> validators[Validators]
  validators --> validation-contract
  validators --> shared-artifacts
  validators --> orchestrator
  orchestrator --> shared-artifacts
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
User goal Initial natural-language project objective. Orchestrator
Orchestrator Clarifies requirements, writes the validation contract, decomposes features, steers milestones, and creates fix features. User goal, Validation contract, Feature list, Programmatic runner, Shared artifacts, Validators
Validation contract Finite checklist of behavioral assertions defining completion and correctness. Orchestrator, Validators
Feature list Ordered set of well-scoped features and milestones generated from requirements. Orchestrator, Programmatic runner
Programmatic runner Spawns workers for features and triggers milestone validation. Feature list, Workers, Validators
Workers Fresh agents that write tests first, implement well-specified features, and hand off work. Programmatic runner, Shared artifacts, Validators
Validators Fresh agents that evaluate correctness, completeness, worker trajectories, and black-box behavior. Programmatic runner, Workers, Validation contract, Orchestrator, Shared artifacts
Shared artifacts Distributed mission state: validation contract, features, research notes, guidelines, and knowledge base. Orchestrator, Workers, Validators

Data flow

A user describes the desired software. The orchestrator asks clarifying questions, writes a validation contract, and generates a feature list. A runner spawns workers for features in order; each worker starts fresh, writes tests, implements, and hands off. At milestones, fresh scrutiny and user-testing validators evaluate the result, update shared state, and surface issues. The orchestrator converts actionable gaps into fix features and repeats until validation passes.

3 supporting quotes
“A user describes what they want built. The orchestrator investigates and asks clarifying questions until the requirements are unambiguous.”article, stated
“A programmatic runner takes the feature list and spawns a worker for each feature in order. Each worker starts with a fresh context, receives its feature spec, writes tests first, then implements.”article, stated
“After validation, the orchestrator reviews what workers and validators flagged. It creates fix features targeted at actionable gaps”article, stated

Why this architecture

Missions separates planning, implementation, and validation because agents are highly reactive to their current context. Narrow worker contexts reduce distraction, fresh validators avoid implementation bias, and shared artifacts carry the full mission state without stuffing every detail into one model context. Defining correctness before features prevents the validation contract from being contaminated by implementation choices.

3 supporting quotes
“Most of the architecture follows from one core observation: agents are highly reactive to their context. 1.”article, stated
“An agent that implemented something is worse at objectively evaluating its own work than a fresh, unbiased reviewer.”article, stated
“* At the mission level, the orchestrator defines correctness first - creating a validation contract, a set of behavioral assertions that define success, before defining any features.”article, stated

Models used: orchestrator model · worker models · validator models

Scale

Alternatives considered (roads not taken)

One broad agent session for the whole project

Rejected because: Single agents accumulate irrelevant context and become less focused and reliable.

“Agent sessions work well for focused tasks, but most real projects are too broad and complex for a single context window to hold.”article, stated
“A single agent eventually runs into a problem: the more it sees, the less focused and reliable it becomes. Missions is our system for solving that.”article, stated

Let workers judge their own correctness

Rejected because: Workers are biased toward confirming their own implementation, so independent validators decide correctness.

“They iterate until they believe the work is correct, then hand it off. But the final judgment on correctness is not their call.”article, stated

Create features before correctness criteria

Rejected because: The validation contract would be biased by already planned implementation details.

“If it had created the features first, the contract would be influenced by the implementation it had already planned.”article, stated

Validators inspect only source code

Rejected because: User-testing validators exercise the system as a black box against the behavior contract.

“fresh agents that exercise the system as a black box - using it the way a real user would - rather than inspecting the code”article, stated

Patterns

multi-agent-orchestration (agentic)

Missions coordinates orchestrators, workers, and validators with separated goals and handoffs.

“Each role has a single goal, and the system is structured so that nothing in an agent's trajectory pulls it away from that goal.”article, stated

context-compaction (memory)

Large mission state is held in shared artifacts rather than one model context.

“No single agent needs to hold the complete picture in its context at once.”article, stated
“The full state is distributed across shared artifacts”article, stated

eval-driven-development (quality)

Correctness is defined first as a validation contract and then checked by fresh validators after milestones.

“the orchestrator defines correctness first - creating a validation contract”article, stated
“Once all features within a milestone are complete, the runner triggers validation using fresh agents.”article, stated

human-in-the-loop-gate (control)

If implementation or validation blocks, the mission halts and returns control to the user.

“If implementation or validation is blocked, the orchestrator halts the mission and hands control back to the user.”article, stated

model-routing (modeling)

Different roles can use models optimized for planning, execution, cost, or skeptical validation.

“model choice becomes local to each role: broad planning and judgment for the orchestrator, reliable execution and cost efficiency for workers, thoroughness and skepticism for validators.”article, stated

Tradeoffs

Distribute mission state through artifacts

Chose validation contract, feature list, research notes, guidelines, and knowledge base over one agent holding the entire project context.

Rationale: Artifacts let each agent read only what is relevant to its role and avoid context bloat.

“Each agent reads what's relevant to its current job.”article, stated
“Even the orchestrator delegates deep investigation to subagents to avoid consuming every detail itself.”article, stated

Use fresh validators after implementation

Chose independent scrutiny and user-testing validators over worker self-certification.

Rationale: Fresh validators avoid implementer bias and can test the system like a real user.

“An agent that implemented something is worse at objectively evaluating its own work than a fresh, unbiased reviewer.”article, stated
“User-testing validators exercise the system as a black box - using it the way a real user would”article, stated

Have workers write tests before code

Chose tests reflecting intended behavior over tests shaped by the worker's implementation.

Rationale: Test-first worker execution reduces the chance that tests merely encode implementation details.

“Each worker writes tests before code, so the tests reflect intended behavior rather than implementation details.”article, stated

Gotchas (what bit them)

Irrelevant context degrades agent focus.

Root cause: Broad tasks make the trajectory accumulate information not relevant to the current step.

Resolution: Missions splits work into focused units with fresh agents and narrowly scoped goals.

“Irrelevant context accumulates.”article, stated
“It breaks large work into focused units handled by fresh agents with narrowly scoped goals”article, stated

Self-review is biased after implementation.

Root cause: An implementer agent's prior reasoning creates a bias toward confirming its own work.

Resolution: Fresh validators review worker implementation and black-box behavior independently.

“An agent that implemented something is worse at objectively evaluating its own work than a fresh, unbiased reviewer.”article, stated
“evaluate completed work for correctness and completeness, surfacing bugs and gaps.”article, stated

Feature planning can contaminate correctness criteria.

Root cause: If the system plans features before defining validation, the contract can reflect implementation bias.

Resolution: The orchestrator writes the validation contract before defining features.

“This ordering matters.”article, stated
“the orchestrator draws from its understanding of requirements. If it had created the features first, the contract would be influenced”article, stated

Evaluation

Factory evaluated Missions with milestone validation rounds, black-box user-testing validators, issue/fix-feature counts, generated code/test metrics, and statement coverage.

Metrics: 38.8k lines of code generated · 52.5% of generated lines were tests · 89.25% statement coverage · 2-4 validation rounds per milestone · 81 issues surfaced by validators · 21 targeted fix features generated by the orchestrator

3 supporting quotes
“It generated 38.8k lines of code (52.5% of those lines tests) with 89.25% statement coverage.”article, stated
“validators surfaced 81 issues, and the orchestrator generated 21 targeted fix features (34.4% of implementation work) to close them.”article, stated
“Every milestone converged in 2-4 validation rounds.”article, stated

Outcomes

The source reports a sample mission with 38.8k lines, 52.5% tests, 89.25% statement coverage, and every milestone converging in 2-4 validation rounds.

3 supporting quotes
“Missions is our first version of a system that closes the software development loop.”article, stated
“better planners produce tighter specs, better workers make fewer mistakes, and better validators can judge correctness more reliably”article, stated
“More ambitious missions become viable for more teams and codebases.”article, stated