Zero Human-Written Code: Harness Engineering for Autonomous AI Agents at Scale

openai · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Autonomous AI software engineering and harness engineering

OpenAI's internal application team wanted autonomous coding agents to produce production software at scale while humans stopped writing code and stopped reviewing implementation diffs directly.

Constraints:

The operating model deliberately used zero human-written code and zero human code reviews.
Agents only have two primary levers: context and tools.
Quality had to be maintained through tests, guardrails, feedback loops, and specification artifacts rather than human implementation review.
Parallel agent work requires systems-thinking coordination rather than hands-on coding.
The case is a self-reported OpenAI talk about OpenAI tooling and should be read with promotional context.

Architecture

The team built a harness-engineering operating system around Codex: repositories, runbooks, tool outputs, principles documents, CI anti-slop jobs, asynchronous reviewer loops, and specification pipelines are structured so agents receive useful context and tools. Humans focus on interfaces, plans, architecture, customer priorities, and high-leverage feedback, while agents implement, test, refactor, and learn from generated artifacts.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  human-harness-engineers["Human harness engineers"] --> codex-agents["Codex agents"]
  repository-context["Repository context"] --> codex-agents
  tooling-layer["Tooling layer"] --> codex-agents
  human-harness-engineers --> golden-principles-document["Golden principles document"]
  golden-principles-document --> ci-anti-slop-jobs["CI anti-slop jobs"]
  codex-agents --> ci-anti-slop-jobs
  ci-anti-slop-jobs --> feedback-artifacts["Feedback artifacts"]
  feedback-artifacts --> codex-agents
  codex-agents --> specification-pipeline["Specification pipeline"]
  human-harness-engineers --> specification-pipeline

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Human harness engineers`	Define interfaces, architecture, product priorities, plans, principles, and high-leverage feedback.	`Codex agents`, `Golden principles document`, `Specification pipeline`
`Codex agents`	Produce implementation code, tests, refactors, artifacts, and reviewer suggestions.	`Human harness engineers`, `Repository context`, `Tooling layer`, `CI anti-slop jobs`, `Specification pipeline`
`Repository context`	Provides grep-able source, docs, runbooks, issues, and accumulated organizational knowledge.	`Codex agents`, `Golden principles document`
`Tooling layer`	Exposes CLI, app, browser, test, lint, and other tool calls with agent-readable outputs.	`Codex agents`
`Golden principles document`	Stores engineering rules and feedback-derived principles used by agents and anti-slop systems.	`Human harness engineers`, `Repository context`, `CI anti-slop jobs`
`CI anti-slop jobs`	Scan code, detect principle violations, propose corrective PRs, and create feedback artifacts.	`Codex agents`, `Golden principles document`, `Feedback artifacts`
`Feedback artifacts`	Markdown lessons and run artifacts generated from human feedback and prior agent mistakes.	`CI anti-slop jobs`, `Codex agents`
`Specification pipeline`	Distills specs from working implementations and uses separate implementation and judge agents to refine them.	`Human harness engineers`, `Codex agents`

Data flow

Humans specify intent, architecture, or feedback; Codex agents use repository context, tools, and runbooks to produce code and tests; CI and anti-slop jobs detect violations; human thumbs-up/thumbs-down feedback and review comments are converted into markdown lessons; subsequent agent runs ingest those lessons and refine future behavior; and high-level specs can be distilled from accepted implementations and judged by separate agents.

3 supporting quotes

“The core principle recognizes that AI coding agents have only two fundamental levers: the context provided and the tools available.”summary, stated

“They implemented automated CI jobs that continuously scanned for "slop" (low-quality code), compiled lists of golden principles, identified divergences, and proposed corrective pull requests.”summary, stated

“The agent would generate markdown artifacts documenting lessons learned from human feedback, which would be saved to GitHub run artifacts and ingested in subsequent runs.”summary, stated

Why this architecture

The architecture treats agentic development as an operating system rather than a prompt trick: tools and context are deliberately shaped so autonomous agents can make progress, detect mistakes, absorb feedback, and parallelize work while humans supervise the highest-leverage artifacts.

“Rather than optimizing individual prompts or context windows, harness engineering considers how to structure the entire repository, documentation, testing infrastructure, and tool ecosystem so that agents can operate with minimal human intervention.”summary, stated

“This level of parallelization requires stepping back from hands-on coding to adopt a systems-thinking mentality similar to that of a group tech lead or engineering manager.”summary, implied

Models used: Codex · Codex Mini · GPT-5.2 · GPT-5.3 · GPT-5.4 · GPT-5.5 · Reviewer agents · Judge agents

Scale

PR throughput rose from about 3.5 to 70 PRs per engineer per week across GPT-5.2 to GPT-5.5.
“the team scaled their PR throughput per engineer per week from approximately 3.5 with GPT-5.2 to 70 with GPT-5.5”summary, stated
The operating model aims at billion-token-per-day agent consumption.
“To reach billion-token-per-day consumption, developers must think beyond pair programming toward massively parallel operations with 15 or more workstreams open simultaneously.”summary, stated
The principles issue grew to 100-150 comments.
“This issue grew to 100-150 comments and became the seed data for automated anti-slop systems that would spider through the codebase,”summary, stated
New hires took approximately two weeks to ramp into the agentic workflow.
“New hires took approximately two weeks to ramp up to this way of operating.”summary, stated

Alternatives considered (roads not taken)

Traditional human code production and human code review

Rejected because: The experiment deliberately moved humans to specifications, architecture, feedback, and high-level planning instead of implementation diffs.

“The team adopted two core constraints that distinguish their approach: zero human-written code and zero human code reviews.”summary, stated

Pre-write detailed specifications before implementation

Rejected because: The team found it more effective to build a strawman first, refine it, and then distill the specification from the accepted artifact.

“Rather than starting with specifications and refining them during implementation, they found it more effective to produce code first as a strawman,”summary, stated

Shift every quality check left into deterministic tests

Rejected because: Small prompt or principles changes were often cheaper and propagated through future agent runs.

“they found it often cheaper to make small prompt adjustments to their golden principles document and let those changes propagate through the next agent runs.”summary, stated

Patterns

`context-engineering` (agent harness)

The team curates repository context, runbooks, principles, artifacts, and semantically meaningful tool outputs so agents have the right working context.

“This includes making nonfunctional requirements explicit, surfacing golden principles at the right time, and creating feedback loops that allow agents to learn from their mistakes.”summary, stated

`tool-use-orchestration` (developer tooling)

Manual interventions were progressively converted into proper tool calls and CLI/app/browser capabilities agents could use directly.

“Over time, these manual interventions were systematized into proper tool calls, reducing human involvement. This iterative automation of manual steps became a core pattern: observe where human time is spent, identify repetitive patterns, and build tools to eliminate that friction.”summary, stated

`eval-driven-development` (quality)

Tests, CI, reviewer agents, anti-slop scans, and feedback artifacts replace pre-merge human implementation review.

“Testing infrastructure was designed specifically for agent consumption rather than human readability. Test output was compressed yet semantically meaningful, providing context about what went wrong and how to fix it rather than exhaustive technical logs.”summary, stated

“The team also employed reviewer agents as part of their quality assurance process.”summary, stated

`multi-agent-orchestration` (software engineering)

Multiple agents implement, review, judge specs, and operate in parallel workstreams under human orchestration.

“To reach billion-token-per-day consumption, developers must think beyond pair programming toward massively parallel operations with 15 or more workstreams open simultaneously.”summary, stated

`observability-tracing` (feedback loops)

Session logs, run artifacts, test failures, and review comments are captured and fed back into later agent runs.

“have humans provide thumbs-up/thumbs-down feedback with review comments, then in the next run ingest that feedback and compare it with session logs to identify mistakes and misaligned priorities.”summary, stated

Tradeoffs

Move humans out of implementation diffs

Chose Human focus on context, tools, specs, architecture, and feedback over Human-written code and human code review.

Rationale: The team could scale agent throughput while preserving quality through guardrails and asynchronous review loops.

“Ryan's time allocation shifted from 50-70% code production before this approach to roughly 30% on the hardest refactors and zero-to-one product ideation,”summary, stated

Use prompt/principles changes as quality interventions

Chose Small changes to golden principles and prompt context over Always adding deterministic tests or documentation updates.

Rationale: Agent runs could absorb principles changes and propagate them into future work more cheaply.

“This represented an inversion of the traditional DevOps "shift left" mentality, where the cheapest intervention was actually a prompt change rather than deterministic tests or documentation updates.”summary, stated

Distill specs from accepted implementations

Chose Code-first strawmen followed by spec extraction and judge refinement over Fully specified design documents before implementation.

Rationale: Accepted artifacts contain dense information about the team's real decisions and desired behavior.

“This approach recognizes that produced artifacts, whether code, spreadsheets, or documents, are information-dense records of the decisions made to create something the team deemed good.”summary, stated

Gotchas (what bit them)

Agents need agent-readable failure context, not raw logs.

Root cause: Exhaustive technical logs are less useful to agents than semantically meaningful instructions and runbooks.

Resolution: Format test and lint failures as concise prose that names the mistake, file, runbook, and trusted repair path.

“test failures and lint errors are formatted not as exhaustive technical logs but as semantically meaningful prose messages that guide the agent toward solutions.”summary, stated

Repeated human corrections must become system memory.

Root cause: Without feedback loops, humans keep giving the same review comments and slop recurs.

Resolution: Convert comments into golden principles, anti-slop scans, markdown artifacts, and subsequent agent inputs.

“The focus was on eliminating slop systematically and finding ways to prevent the same feedback from being given twice.”summary, stated

High-level plans and interfaces remain high-leverage human review points.

Root cause: Under-specified or over-specified plans become prompts that agents execute poorly over long spans of work.

Resolution: Keep selective human oversight for complex plans, phase milestones, and interface definitions.

“These items required human review because they functioned as prompts for agents to execute, and under-specification or over-specification would lead to poor results.”summary, stated

Evaluation

The team evaluates agent work through tests designed for agent consumption, CI anti-slop jobs, reviewer agents, human thumbs-up/down feedback, session-log comparison, and judge-agent comparison of specs against implementations.

Metrics: PRs per engineer per week · Feedback-derived slop violations · Golden-principle divergence · Spec/implementation alignment · Onboarding ramp time · Parallel workstreams and token throughput

“These systems employed a three-phase learning loop: produce PRs to fix violations, have humans provide thumbs-up/thumbs-down feedback with review comments, then in the next run ingest that feedback”summary, stated

“Third, a judge agent compared the original implementation with the derived artifact and the spec, identifying misalignments and proposing spec changes to improve the next iteration.”summary, stated

Outcomes

The team reports scaling from approximately 3.5 to 70 PRs per engineer per week across GPT-5.2 to GPT-5.5.
New hires reportedly ramped into the autonomous development environment in about two weeks.
Humans shifted attention toward interfaces, system architecture, product prioritization, and quality systems rather than routine code production.

The operating model targets massively parallel work with 15+ workstreams and very high token consumption rather than pair-programming-style single-agent use.

“The team's throughput of 70 PRs per engineer per week with GPT-5.5 would be impossible without this parallel, asynchronous operating model.”summary, stated

“The case study ultimately demonstrates that LLMOps at scale requires rethinking fundamental assumptions about software development.”summary, stated