Building Fully Autonomous Coding Agents for Non-Technical Users

replit · Tech · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Autonomous coding agents for non-technical software creation

Replit wanted coding agents that non-technical knowledge workers could use without making technical decisions, debugging code, or manually testing every UI path. The team argues that autonomy should mean the agent can make technical decisions during an irreducible runtime, not merely run for a long time. The practical blockers were painted-door features, lack of reliable user feedback, compounding errors, context pollution during long runs, and parallel-agent workflows that assume users can decompose tasks and resolve merge conflicts.

Constraints:

Architecture

Replit's autonomous agent architecture combines frontier model capability, autonomous verification, context management, and sub-agent orchestration. The main agent loop plans, implements, and tests while taking technical decisions on behalf of non-technical users. Verification uses autonomous testing: a testing sub-agent writes Playwright code, interacts with the web app through DOM/programmatic browser APIs, database reads, logs, API calls, literal clicks, and screenshots as fallback, then turns those tests into a reusable regression suite. Context management persists plans, task lists, docs, and memories to the codebase/file system. Sub-agents run from fresh contexts with minimal input, return only final observations, and discard polluted browser/test contexts. Replit is extending the same core loop into a parallel orchestrator that decomposes tasks and decides parallelism on behalf of the user.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  non-technical-user[Non-technical user] --> replit-autonomous-coding-agent[Replit autonomous coding agent]
  replit-autonomous-coding-agent --> main-agent-loop[Main agent loop]
  replit-autonomous-coding-agent --> frontier-model[Frontier model]
  replit-autonomous-coding-agent --> codebase-state-store[Codebase state store]
  replit-autonomous-coding-agent --> sub-agent-orchestrator[Sub-agent orchestrator]
  main-agent-loop --> codebase-state-store
  codebase-state-store --> memory-file-store[Memory file store]
  codebase-state-store --> generated-application[Generated application]
  main-agent-loop --> sub-agent-orchestrator
  sub-agent-orchestrator --> testing-sub-agent[Testing sub-agent]
  testing-sub-agent --> playwright-test-writer[Playwright test writer]
  playwright-test-writer --> browser-automation-layer[Browser automation layer]
  playwright-test-writer --> regression-test-suite[Regression test suite]
  browser-automation-layer --> generated-application
  generated-application --> database-log-api-feedback[Database/log/API feedback]
  database-log-api-feedback --> testing-sub-agent
  testing-sub-agent --> final-observation[Final observation]
  final-observation --> main-agent-loop
  main-agent-loop --> parallel-core-loop-orchestrator[Parallel core-loop orchestrator]
  parallel-core-loop-orchestrator --> parallel-sub-agents[Parallel sub-agents]
  parallel-sub-agents --> merge-conflict-mitigation[Merge conflict mitigation]
  merge-conflict-mitigation --> main-agent-loop
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Non-technical user Knowledge worker who specifies product goals but cannot be expected to make technical decisions or resolve implementation issues. Replit autonomous coding agent
Replit autonomous coding agent Agent B3-style coding agent that takes technical decisions and runs planning, implementation, and testing loops. Main agent loop, Frontier model, Codebase state store, Sub-agent orchestrator
Main agent loop Core loop that plans, writes code, decides when to verify, receives observations, and continues the build. Codebase state store, Sub-agent orchestrator, Parallel core-loop orchestrator
Frontier model Baseline model capability injected into the main agentic loop. Main agent loop
Codebase state store Codebase and file-system state used for documentation, plans, task lists, and memories outside the active context window. Memory file store, Generated application
Memory file store Persisted memories that the agent can write and later reload when relevant. Main agent loop
Sub-agent orchestrator Creates fresh-context sub-agents with minimal relevant context and returns only their outputs to the main loop. Testing sub-agent, Main agent loop
Testing sub-agent Isolated sub-agent responsible for browser/app verification so browser observations do not pollute the main coding context. Playwright test writer, Generated application, Final observation
Playwright test writer LLM-generated Playwright code used for expressive browser testing and reusable regression tests. Browser automation layer, Regression test suite
Browser automation layer Programmatic browser and DOM interaction layer, with screenshots/computer use as fallback. Generated application, Database/log/API feedback
Generated application The web application being built, clicked, tested, and repaired by the agent. Database/log/API feedback
Database/log/API feedback Feedback channels for database state, logs, API calls, and app interactions used to detect painted doors. Testing sub-agent
Regression test suite Reusable Playwright scripts generated during testing and rerunnable in the future. Testing sub-agent
Final observation Condensed verification result returned from the testing sub-agent to the main loop. Main agent loop
Parallel core-loop orchestrator Planned next-generation orchestration where the core loop decomposes and parallelizes work without user-managed threads. Parallel sub-agents, Merge conflict mitigation
Parallel sub-agents Concurrent workers launched by the core loop to trade compute for elapsed time. Merge conflict mitigation
Merge conflict mitigation Software-engineering task structuring intended to reduce sub-agents stepping on one another. Main agent loop

Data flow

A non-technical user states what they want to build. The main agent loop plans, implements, and decides when verification is required. It writes code and persistent state into the codebase/file system, then spawns a testing sub-agent with a fresh, minimal context. The testing sub-agent writes Playwright code, interacts with the generated app, queries databases/logs/APIs, uses screenshots when necessary, and returns only the final observation. The main loop uses that observation to fix painted doors and continue. Future parallelism moves task decomposition into the core loop so the agent can dispatch subtasks without the user manually orchestrating threads or resolving merge conflicts.

4 supporting quotes
“What agents do is that they always go through this loop of planning, implementing and testing. And of course to make this happen and to make it work correctly, you want this work to be happening over a long quering trajectory. So our goal is to maximize the”transcript, stated
“we directly write playrite code and playrite code is first of all very manageable for LLMs. LLMs are kind of amazing at writing playright. You know this is the experience that we had uh since we started to work on this project is also very powerful and”transcript, stated
“We make this happen all within a sub agent. Then we scratch the context window of that sub agent. We just return back the last observation to the agent loop”transcript, stated
“the subtask that we're going to be working on are not determined by the user but they're determined by the corion loop”transcript, stated

Why this architecture

Replit separates coding, verification, memory, and parallel orchestration because non-technical users cannot supply the missing technical supervision. Autonomous browser testing closes the feedback loop without waiting for users. Playwright code is more expressive and reusable than generic browser tools. Sub-agents isolate noisy browser/test contexts from the main coding loop. File-system and codebase state reduce pressure on context windows. Core-loop orchestration keeps parallelism from becoming a user-facing coordination burden.

4 supporting quotes
“I can't expect knowledge workers to know what kind of technical decisions an agent should be making. We should offload completely the level of complexity away from them. Of course, it took a while to get here. So I'm I'm sure what I'm showing you here is”transcript, stated
“Now we don't have to wait anymore for human feedback. We have a way to elicit as much information as possible from the app autonomously.”transcript, stated
“playright code because you can reuse those tests. The moment you write a test in script then you can rerun it as many times as you want. So in a sense the moment you created a test you're also creating a regression test suite that you can keep running in the future. And all these kind of uh tricks”transcript, stated
“there is a separation of concern here. Got the main agent loop running. We decide at a certain point that it's time to verify”transcript, stated

Models used: Frontier models · LLMs · ReAct agents · Native tool calling · Playwright · Browser automation · Computer use · Claude Sonnet 4.7 · OpenAI models

Scale

Alternatives considered (roads not taken)

Treat autonomy as simply long runtime.

Rejected because: Replit argues autonomy should mean taking technical decisions for the user; narrow tasks can be autonomous and fast.

“about how something has been built. They just want to see their goals to be accomplished. So autonomy should not be basically conflated with long run times. And similarly, it shouldn't become a vanity metric. You know, a lot of us are talking about it as a as a badge of honor.”transcript, stated
“you can have an agent that is really autonomous and is still fast as long as you give it a very narrow scope for the task, you know, at hand. So what we can accomplish in this way is that the user still maintains control on the aspects that they care about”transcript, stated

Ask non-technical users for technical feedback and manual QA.

Rejected because: Users cannot provide the feedback agents need, do not want to test every button, and cannot make technical decisions.

“nontechnical users not only cannot make technical decisions but also they cannot provide the technical feedback”transcript, stated
“though we relied on that with our first release of the agent last year quickly we undo that users don't want to spend time doing testing. So we had to find a complete you know orthogonal solution to that which is autonomous testing and it solves several different”transcript, stated

Use pure computer-use/screenshot testing for browser verification.

Rejected because: Computer-use testing is expensive and slow; Playwright/browser-use programmatic testing is cheaper, faster, and more expressive, with screenshots as fallback.

“computer use it's a onetoone mapping with user interface so the model is directly interacting with the application it requires screenshots it tends to be fairly expensive and fairly slow”transcript, stated
“nothing has worked. So we have a full back to computer use. But the vast majority of times what we do is that we have programmatic interactions with the applications. So we interact with the database, we read the logs, we do API calls, we literally click on the app and get back all the”transcript, stated

Expose only generic browser tools.

Rejected because: Generic click/fill tools cannot cover the long tail of idiosyncratic web interactions; writing Playwright code is more expressive and reusable.

“it's difficult to enumerate all the different type of interactions you could be having with a browser. The problem of testing is very similar to the Tesla analogy I was making before. Maybe this cardality of tools available is enough for 99% of the”transcript, stated
“we directly write playrite code and playrite code is first of all very manageable for LLMs. LLMs are kind of amazing at writing playright. You know this is the experience that we had uh since we started to work on this project is also very powerful and”transcript, stated

Put browser actions and observations directly into the main loop.

Rejected because: It polluted and confused the main coding loop, so testing had to move into isolated sub-agents.

“to do browser opt browser actions to put back the observation of your browser actions into the main loop you tend to confuse the the hedging loop very much”transcript, stated
“not only we have to build all the playright framework that I was showing to you before but we also have to move our entire architecture into sub aents. So at this point you can see very clearly why there is a separation of concern here. Got the main agent loop running. We”transcript, stated

Make users orchestrate parallel agents manually.

Rejected because: Non-technical users cannot decompose tasks, manage threads, or resolve merge conflicts; the core loop must orchestrate parallelism.

“which I go basically call user has an orchestrator is the fact that tasks the par task that you want to run are determined by you by the user”transcript, stated
“my users don't even know what's the concept of merge conflicts. It's something that I have to figure out on our own.”transcript, stated

Patterns

multi-agent-orchestration (orchestration)

Replit uses sub-agents for isolated tasks such as testing, and is moving toward a core loop that orchestrates parallel sub-agents without user decomposition.

“the key ingredient to make this happen has been how good models hand as agent builders have become in doing sub agent orchestration.”transcript, stated
“the subtask that we're going to be working on are not determined by the user but they're determined by the corion loop”transcript, stated

tool-use-orchestration (orchestration)

The testing agent writes and runs Playwright/browser automation, calls APIs, reads logs, inspects databases, clicks the app, and falls back to screenshots/computer use when needed.

“we interact with the database, we read the logs, we do API calls, we literally click on the app and get back all the information that we need.”transcript, stated
“So what we do uh in our case at rapid is we directly write playrite code and playrite code is first of all very manageable for LLMs.”transcript, stated

eval-driven-development (evaluation)

Autonomous testing is part of the agent loop: the agent verifies local correctness, detects painted doors, and turns generated Playwright tests into a regression suite.

“It's very important that we test for local correctness of our agent at every step that it takes”transcript, stated
“the moment you created a test you're also creating a regression test suite that you can keep running in the future.”transcript, stated

context-compaction (context)

Sub-agent isolation reduces context pollution and improves memory-compression cadence; testing contexts are discarded after returning only final observations.

“we went from roughly 35 to 4550 recently. So big improvement in terms of how often we are recompressing our context”transcript, stated
“We just return back the last observation to the agent loop and then we keep running in that way.”transcript, stated

agent-memory (context)

The agent uses the codebase and file system as durable state for documentation, plans, task lists, and memories that can be reloaded when relevant.

“using the codebase itself to maintain state. So you can write documentation while the agent is creating new code.”transcript, stated
“you can even dump directly your memories in the file system and then making sure that your agent decides when to write them back”transcript, stated

latency-optimization (operations)

Replit chooses browser-use/Playwright testing and parallelism to lower verification latency and keep users engaged during autonomous runs.

“roughly a order of magnitude cheaper and faster compared to computer use.”transcript, stated
“you want to see as much work as done as possible in the shortest span of time.”transcript, stated

cost-aware-routing (operations)

The architecture trades extra compute for elapsed time in parallel agents and avoids overusing expensive screenshot/computer-use testing when Playwright can do the job.

“it it applies to computing in general and for parallel agents what you do is that you you trade off basically extra compute in exchange for time. Why there is this trade-off?”transcript, stated
“it requires screenshots it tends to be fairly expensive and fairly slow”transcript, stated

Tradeoffs

Definition of autonomy

Chose technical-decision autonomy for scoped tasks over long runtime as a vanity metric.

Rationale: The important span is time where the user does not make technical decisions, not merely how many hours the agent runs.

“So autonomy should not be basically conflated with long run times. And similarly, it shouldn't become a vanity metric.”transcript, stated
“having a span of time where the user doesn't have to make any technical decisions”transcript, stated

Verification mechanism

Chose Playwright/browser-use with programmatic feedback over pure screenshot-based computer use.

Rationale: Browser-use/Playwright is cheaper, faster, more expressive, and can become a regression suite, while screenshots remain a fallback.

“A good way in in the middle is browser use where we simulate the user interface.”transcript, stated
“they helped us to create something that is roughly a order of magnitude cheaper and faster compared to computer use.”transcript, stated

Testing representation

Chose LLM-written Playwright code over predefined generic browser tools.

Rationale: Playwright is manageable for LLMs, more expressive than a generic tool set, and reusable as regression tests.

“we directly write playrite code and playrite code is first of all very manageable for LLMs.”transcript, stated
“The moment you write a test in script then you can rerun it as many times as you want.”transcript, stated

Context management

Chose file-system/codebase state plus sub-agent isolation over stuffing all state and browser observations into the main context window.

Rationale: The approach keeps the main loop globally coherent while offloading memories and noisy testing contexts.

“there are several different ways to maintain state which don't imply chucking all the state into your context window.”transcript, stated
“you only get the output the results. You inject them back into the main loop and you keep running in this way.”transcript, stated

Parallelism orchestration

Chose core loop as orchestrator over user as orchestrator.

Rationale: The core loop can decompose tasks and choose parallelism without imposing cognitive burden or merge-conflict handling on non-technical users.

“what we're working on today for our next version of the agent is having the core loop as the orchestrator.”transcript, stated
“there's no cognitive burden to for the user to understand how they should be decomposing the task.”transcript, stated

Gotchas (what bit them)

Agents create painted doors without testing.

Root cause: Generated UI or backend features may look present while handlers, database wiring, or full component behavior are missing.

Resolution: Run autonomous browser/application testing and feed observations back into the agent so it can repair broken features.

“So without testing agents build a lot of painted doors. In our case the painted doors are very visible because we create a lot of web applications.”transcript, stated
“we collect enough feedback that allows our agent both to make progress and also to fix all the painted doors that it encounters.”transcript, stated

Non-technical users cannot close the technical feedback loop.

Root cause: They cannot inspect code, debug, or provide detailed implementation feedback; manual QA is tedious.

Resolution: Autonomous testing elicits feedback from the app/environment without waiting for users.

“nontechnical users not only cannot make technical decisions but also they cannot provide the technical feedback”transcript, stated
“Now we don't have to wait anymore for human feedback. We have a way to elicit as much information as possible from the app autonomously.”transcript, stated

Generic browser tools miss the long tail of web interactions.

Root cause: A small set of generic tools cannot enumerate every idiosyncratic user interaction a web app may require.

Resolution: Have the LLM write Playwright code directly, which is more expressive and reusable.

“there is always a long tale of idiosyncratic interactions that a user makes with the with a web application that are hard to map into this tool”transcript, stated
“So what we do uh in our case at rapid is we directly write playrite code and playrite code is first of all very manageable for LLMs. LLMs are kind of amazing at writing playright.”transcript, stated

Browser-testing observations polluted the main coding loop.

Root cause: Putting browser actions and observations into the same context as code generation confused the agent loop.

Resolution: Move testing into sub-agents, discard the sub-agent context, and return only the final observation.

“when you allow the main loop not only to create code but also to do browser opt browser actions to put back the observation of your browser actions into the main loop you tend to confuse”transcript, stated
“We make this happen all within a sub agent. Then we scratch the context window of that sub agent.”transcript, stated

Parallel agents can create user-facing coordination burdens.

Root cause: Manual task decomposition and merge-conflict resolution assume expert software developers, not Replit's target users.

Resolution: Make the core loop orchestrate decomposition and parallelism, and structure tasks to reduce agents stepping on one another.

“So as a reminder, my users don't even know what's the concept of merge conflicts. It's something that I have to figure out on our own.”transcript, stated
“The agent does the task de composition on behalf of the user and this comes with a couple of advantages. First of all again there's no cognitive burden to for the user to understand how they should be decomposing the task.”transcript, stated

Long-running autonomy can make the user experience dull.

Root cause: Users may write a long prompt, leave, and hope the agent finishes, which pulls them out of the creative flow.

Resolution: Use parallelism and faster verification so more work completes in less time while the user stays engaged.

“You are not in the zone anymore. What you do is that you write a very long prompt.”transcript, stated
“it comes with the price of making the user experience less thrilling.”transcript, stated

Evaluation

Replit evaluates agent autonomy through local correctness checks, autonomous browser/app verification, internal feature-breakage evaluations, regression tests produced from Playwright scripts, and operational context metrics such as memories per compression. The talk does not present a public benchmark table, but it gives internal quality and efficiency signals.

Metrics: More than 30% of individual features broken in internal first-generation evaluations. · Almost every generated app having at least one painted door implied by that breakage rate. · Order-of-magnitude cheaper/faster testing compared with computer use. · Most tasks fitting within about 200,000 tokens. · Memory compression improving from roughly 35 to 45-50 memories per compression. · Parallel agents duplicating about 80% shared context.

5 supporting quotes
“We found out that more than 30% of the individual features happen to be broken.”transcript, stated
“almost every applications at least one broken feature or painted door.”transcript, stated
“roughly a order of magnitude cheaper and faster compared to computer use.”transcript, stated
“most of the tasks, even the more ambitious one, can be accomplished within the 200,000 tokens.”transcript, stated
“So we went from roughly 35 to 4550 recently. So big improvement in terms of how often we are recompressing our context just because we can offload a lot of the context pollution by means of using sub agents.”transcript, stated

Outcomes

The talk is product/architecture oriented and does not give public adoption numbers. It does give operational signals: first agent launch in September, B3 a couple months before the talk, several-hour agent runs, >30% broken feature rate in internal evals, 200k-token task context, 35 to 45-50 memories per compression, and 80% shared context duplication in parallel agents.

5 supporting quotes
“with rapid agent 3 what we accomplish is we we make sure that our agent takes holy technical decisions.”transcript, stated
“Now we don't have to wait anymore for human feedback. We have a way to elicit as much information as possible from the app autonomously.”transcript, stated
“the moment you created a test you're also creating a regression test suite that you can keep running in the future.”transcript, stated
“So we went from roughly 35 to 4550 recently. So big improvement in terms of how often we are recompressing our context just because we can offload a lot of the context pollution by means of using sub agents.”transcript, stated
“the core loop as an orchestrator is going to be the our main bet for the next few months.”transcript, stated