Building an AI-Powered Software Factory with Autonomous Code Generation and Review

twin_sun · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Autonomous software development factory

Twin Sun wanted to scale software delivery across many concurrent client projects by automating planning, implementation, review, testing, and iteration without sacrificing code quality or human accountability.

Constraints:

Agency teams work across many projects with limited context about each other's applications.
Generated code must pass tests, security checks, accessibility checks, and review.
The factory needs cycles and escalation when automation cannot converge.
Humans still specify priorities and inspect work at defined points.

Architecture

Twin Sun built Scarif, a software factory that starts from task specifications in Jira, delegates work to runners running Claude Code agents, plans and implements changes, runs automated tests and checks, performs code review, evaluates review findings against the plan and acceptance criteria, and cycles back to revise until the evaluator signs off or a human needs to inspect the work. The factory grew out of an incremental PR review bot, Twin Sun Dev, which eventually approved a large share of pull requests.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  human-planner[Human Planner] --> jira-queue[Jira Queue]
  jira-queue --> scarif-factory[Scarif Factory]
  human-planner --> scarif-factory[Scarif Factory]
  scarif-factory --> claude-code-runners[Claude Code Runners]
  claude-code-runners --> ci-and-checks[CI and Checks]
  ci-and-checks --> evaluator[Evaluator]
  scarif-factory --> review-agent[Review Agent]
  review-agent --> evaluator[Evaluator]
  evaluator --> scarif-factory[Scarif Factory]
  evaluator --> human-planner[Human Planner]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Human Planner`	Sets priorities, discusses plans, and creates stories.	`Jira Queue`, `Scarif Factory`
`Jira Queue`	Task-specification source for the factory.	`Scarif Factory`
`Scarif Factory`	Coordinates planning, implementation, review, evaluation, and cycles.	`Jira Queue`, `Claude Code Runners`, `Review Agent`, `CI and Checks`, `Evaluator`
`Claude Code Runners`	Developer agents that plan and implement tasks.	`Scarif Factory`, `CI and Checks`
`Review Agent`	Automated code-review agent derived from Twin Sun Dev.	`Scarif Factory`, `Evaluator`
`CI and Checks`	Automated tests, security checks, and accessibility checks.	`Claude Code Runners`, `Evaluator`
`Evaluator`	Decides whether findings require revision, signoff, or human escalation.	`Scarif Factory`, `Human Planner`

Data flow

Humans define priorities and stories in Jira. Scarif pulls available work, creates a plan, delegates implementation to Claude Code runners, runs review and CI feedback, returns failures to the developer agent, and loops through implementation/review/evaluation until signoff or human escalation.

“our factory as it stands today starts with a task specification. We enter tasks like you would enter them into Exten Jira.”transcript, stated

“have work delegated to runners that are running Claude code agents. And they review the plan, implement the task and then it goes through our workflow of, you know, PR review.”transcript, stated

Why this architecture

Twin Sun started where the workflow was already disciplined: pull-request review. Once the bot had more context than many human reviewers, the same workflow step became a factory component. Scarif then wrapped task planning, implementation, tests, review, and evaluation into a cyclic factory instead of a one-shot code-generation tool.

“project it often had better context than most of the team about what a what a pull request was for. So over time it started feeling like the human review was just a distraction and taking time away from folks when Twin Sun Dev could do a lot of the work.”transcript, stated

“things about this is not an acyclic graph. Uh so there are some, you know, there is like a cycle counter. So it's like, "All right, once you get to like five cycles, probably need a human to look at this." But so it will go back and do it again and go through, you know, all those”transcript, stated

Models used: Claude Code

Scale

Twin Sun Dev approved roughly 70% of pull requests after autonomous approval was enabled.
“something like 70% I think to me of pull requests were approved by Twin Sun Dev and now most projects that is the PR process. And so that fell right into place in a workflow step in our factory. But it's that openness”transcript, stated
Twin Sun works across many concurrent projects and small one- or two-person teams.
“We work on a lot of projects concurrently. We have a lot of small teams, you know, one or two person dev teams.”transcript, stated
Scarif escalates after about five cycles.
“right, once you get to like five cycles, probably need a human to look at this." But so it will go back and do it again and go through, you know, all those stages, implement, review, and then evaluate again. And”transcript, stated

Alternatives considered (roads not taken)

Humans read every line and approve every PR.

Rejected because: The automated reviewer often had more task context and reduced distraction for humans.

“human review was just a distraction and taking time away from folks when Twin Sun Dev could do a lot of the work.”transcript, stated

One-shot autonomous implementation.

Rejected because: The factory needs review, evaluator feedback, and revision cycles.

“So the agent gets that context builds a new plan for finishing the implementation, goes back through review and testing”transcript, stated

General-purpose dark factory first.

Rejected because: Twin Sun incrementally automated a known workflow step before expanding.

“got. It's the first tool that humans have made that we can speak to and it speaks back. Can't do that with a hammer. Um but as a consultant, I've been kind”transcript, implied

Patterns

`tool-use-orchestration` (software-factory)

Scarif coordinates Jira, Claude Code runners, automated checks, review, and evaluator loops.

“take the ticket, break it into its own plan”transcript, stated

`human-in-the-loop-gate` (quality-control)

Scarif escalates after repeated cycles or where human review is required.

“probably need a human to look at this.”transcript, stated

`eval-driven-development` (review)

The factory separately evaluates review findings against the plan and acceptance criteria before signoff.

“separately evaluate the review. Go, "Okay, based on this these findings, here was our initial plan with acceptance criteria.”transcript, stated

Tradeoffs

PR process

Chose Autonomous approval for many pull requests over Mandatory human PR approval.

Rationale: Twin Sun Dev had enough task context to handle many reviews and reduce human distraction.

“most projects that is the PR process.”transcript, stated

Factory control flow

Chose Cyclic review/evaluation loop with human escalation over Acyclic one-pass workflow.

Rationale: Implementation needs revision until acceptance criteria and review findings are resolved.

“it will go back and do it again and go through, you know, all those stages, implement, review, and then evaluate again.”transcript, stated

Gotchas (what bit them)

Review bots need project context, not generic style comments.

Root cause: Agency reviewers often lack context about other teams' apps.

Resolution: Use task specification and project information in automated PR review.

“we also want to ensure we're implementing the thing that was specified.”transcript, stated

Automation must handle failed checks as feedback, not terminal failure.

Root cause: Tests, security, accessibility, or review may fail on first implementation.

Resolution: Return feedback to the developer agent and rebuild the plan.

“all that feedback gets returned to the the developer agent if things don't pass muster.”transcript, stated

Evaluation

Twin Sun evaluates through PR approval rates, review feedback, tests, security/accessibility checks, evaluator signoff, and human escalation after repeated cycles.

Metrics: Autonomous PR approval rate · Cycle count · Test results · Security checks · Accessibility checks · Human escalation

“automated tests, a practice again we've we've had for years. Security checks, accessibility checks”transcript, stated

Outcomes

Twin Sun Dev approved about 70% of pull requests.
Scarif became the factory coordinating planning, implementation, review, and test loops.
Human review shifted toward escalation and oversight rather than every-line inspection.

The source gives a 70% PR approval figure, many concurrent projects, small one- or two-person teams, and a five-cycle human escalation threshold.

“that fell right into place in a workflow step in our factory.”transcript, stated