Building Reliable Agentic Systems in Production

factory.ai · Tech · year n/a · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Reliable production agents for software engineering

Factory.ai needed agentic systems that could handle real-world software-engineering workflows involving planning, decisions, external environment interaction, tools, feedback, and human intervention without drifting, looping, or losing reliability.

Constraints:

• Agentic systems for software engineering must operate across multi-step complex workflows, not only one-off IDE completions.
• Long-term plans can drift away from the original goal as intermediate errors propagate.
• Decision quality is affected by next-token conditioning, sampling variance, reasoning strategy, and model distribution fit.
• Agents must interact with external environments through carefully designed interfaces and tool abstractions.
• Coding environments produce noisy feedback such as logs, stdout, and debug output that must be filtered.
• Even 10-20% failure rates become a large operational burden at scale, so human interaction and UX remain necessary.

Architecture

Factory.ai frames reliable agents around three capabilities: planning, decision-making, and environmental grounding. Planning reliability uses intermediate context passing, subtask decomposition, model-predictive-control-style replanning, and explicit plan criteria. Decision reliability uses consensus mechanisms, explicit reasoning strategies, model-weight modification only for known distributions, and simulation-based decision search when tests or reward criteria can be executed. Environmental grounding relies on dedicated tool interfaces, explicit feedback processing, bounded exploration, and human guidance designed into the product UX.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  software-engineer[Software engineer] --> factory-agent-platform[Factory agent platform]
  factory-agent-platform --> planning-module[Planning module]
  planning-module --> intermediate-context-passer[Intermediate context passer]
  planning-module --> subtask-decomposition[Subtask decomposition]
  planning-module --> mpc-replanning-loop[MPC replanning loop]
  planning-module --> plan-criteria-checker[Plan criteria checker]
  factory-agent-platform --> decision-module[Decision module]
  subtask-decomposition --> decision-module
  decision-module --> consensus-sampler[Consensus sampler]
  decision-module --> reasoning-strategy[Reasoning strategy]
  decision-module --> fine-tuned-model-option[Fine-tuned model option]
  decision-module --> simulation-engine[Simulation engine]
  factory-agent-platform --> grounding-module[Grounding module]
  grounding-module --> dedicated-tool-interfaces[Dedicated tool interfaces]
  grounding-module --> feedback-processor[Feedback processor]
  feedback-processor --> mpc-replanning-loop
  grounding-module --> bounded-exploration[Bounded exploration]
  grounding-module --> external-environment[External environment]
  dedicated-tool-interfaces --> external-environment
  simulation-engine --> external-environment
  bounded-exploration --> external-environment
  external-environment --> feedback-processor
  factory-agent-platform --> human-guidance-ux[Human guidance UX]
  human-guidance-ux --> software-engineer

Components

Data flow

A software engineer gives the Factory platform a goal. The planning module creates or updates a plan using criteria, subtask decomposition, context passing, and feedback-driven replanning. The decision module samples or reasons over possible actions, optionally using consensus or simulation when the domain has executable tests or reward criteria. The grounding module executes through dedicated tools, reads logs and other environment feedback, filters noisy outputs, gathers bounded context, and routes human intervention through designed UX points when autonomous reliability is insufficient.

Why this architecture

The design treats reliability as a product of the whole agent system, not only the model. Plans need convergence criteria and adaptation; decisions need sampling or reasoning support; and environment interaction depends on tool interfaces and feedback processing. Factory.ai specifically argues that AI-computer interfaces are the differentiator because advanced models already provide much planning and reasoning capability, but they do not come with domain-specific tool integrations.

Models used: LLMs · Base models · Open-source models for possible fine-tuning · o1-style planning/reasoning models · LLM-based Monte Carlo tree search style simulations

Scale

• The speaker frames Factory's platform as serving organizations both small and large.
  “organizations both small and large uh collaborate with AI systems so that they can work on all of the tasks that a software developer might need to work on”transcript, stated
• Coding-agent feedback can include hundreds of thousands of lines of logs and stdout/debug output.
  “this could be truly hundreds of thousands of lines of standard out outputs debug logs”transcript, stated
• The speaker calls out 10-20% task failure rates as operationally large at scale.
  “let's say 10% of the time 20% of the time uh at scale that's actually a ton”transcript, stated
• The talk explicitly positions software-engineering autonomy as the company's highest-leverage problem.
  “bringing autonomy to software engineering is a compelling problem we really do think this is the highest leverage problem in the world”transcript, stated

Alternatives considered (roads not taken)

Use static up-front plans

Rejected because: Complex tasks require reevaluation as execution produces feedback and issues.

Fine-tune model weights for every reliability problem

Rejected because: Fine-tuning is expensive, locks in quality, and base models often perform better across broader distributions.

Give agents broad arbitrary tools by default

Rejected because: Broader tools increase capability but decrease accuracy and make abstraction-layer design more important.

Let the agent consume raw environmental feedback

Rejected because: Agents lack human-like sensory filtering and need explicit signal processing over noisy logs and outputs.

Remove human interaction once reliability improves

Rejected because: At scale, residual failure rates still produce too many failed tasks, so interaction design remains necessary.

Patterns

tool-use-orchestration (agent-architecture)

Factory.ai emphasizes custom AI-computer interfaces and dedicated tools as the main differentiator for production agents.

eval-driven-development (quality)

Simulation-based decisions use executable tests or compile checks as reward criteria for choosing among paths.

human-in-the-loop-gate (governance)

The product should include designed human intervention points instead of assuming autonomy eliminates interaction.

context-compaction (context-engineering)

Noisy environmental feedback is filtered so the agent receives critical data rather than raw logs and debug output.

multi-agent-orchestration (agent-architecture)

The talk allows for multi-state, multi-task plans coordinating multiple actors across software-development workflows.

confidence-thresholding (control)

Consensus mechanisms sample multiple outputs and select the most consistent or aggregated answer to improve decision quality.

Tradeoffs

Use consensus mechanisms

Chose Prompt ensembles, cluster sampling, or self-consistency over Trusting a single sampled model output.

Rationale: Multiple samples improve decision accuracy, but they increase inference cost.

Use model predictive control style replanning

Chose Dynamic plans that adapt to real-time feedback over A fixed initial plan.

Rationale: Adaptation helps agents respond to problems but raises the risk of straying from the intended trajectory.

Simulate decision paths

Chose Forward simulation with reward criteria where the domain supports it over Pure reasoning without executable path checks.

Rationale: Simulation can improve quality in software domains with tests, but it is slow, expensive, and hard to model.

Invest in AI-computer interfaces

Chose Dedicated integrations and tools around strong base models over Competing primarily through model-level planning/reasoning.

Rationale: Model providers already supply strong reasoning; domain tools and integrations are where product builders differentiate.

Use human guidance as product design

Chose Designed intervention and interaction points over Assuming reliability improvements remove the human.

Rationale: Residual failure rates are still operationally significant at scale.

Gotchas (what bit them)

Plans drift away from their original responsibility.

Root cause: Long multi-step plans can propagate incorrect predictions or misleading reasoning from earlier steps.

Resolution: Pass intermediate context, decompose tasks, use dynamic replanning, and define explicit plan criteria.

Explicit reasoning strategies can improve consistency but reduce adaptability.

Root cause: Forcing a reasoning strategy constrains the model's natural problem-solving path and can be wrong for a given task.

Resolution: Use reasoning patterns when they match the problem, but avoid assuming one strategy is universally correct.

Simulation-based decision systems are hard to debug.

Root cause: Simulations encode assumptions about the environment, reward, and scope that may be hard to validate.

Resolution: Use simulation selectively where executable reward criteria exist and treat modeling assumptions as a first-class risk.

Tool abstraction level creates a capability-versus-accuracy tradeoff.

Root cause: More general tools let agents do more, but the broad action space lowers accuracy and increases maintenance needs.

Resolution: Design dedicated tools and choose the abstraction layer where the agent can be most effective.

Bounded exploration can overload context or pull the agent off path.

Root cause: Retrieval and information gathering are difficult in open environments.

Resolution: Allow exploration, but keep it bounded and focused on the current problem space.

Evaluation

The talk does not report formal quantitative results for Factory.ai. Evaluation concepts appear as design guidance: convergence to plan criteria, self-consistency/consensus, executable software tests or compile checks as simulation rewards, and feedback processing over logs and environment outputs.

Metrics: Plan convergence · Decision consistency · Executable test pass/fail · Compile pass/fail · Inference cost · Simulation runtime and expense · Failure rate requiring human intervention · Log/feedback signal quality

Outcomes

• Factory.ai presents a reliability framework organized around planning, decision-making, and environmental grounding.
• The talk identifies AI-computer interfaces and tool design as a likely differentiator for agent builders.
• The talk does not provide specific production before/after metrics, customer deployment counts, or measured reliability gains.

The source is sparse on measured outcomes. Concrete scale references include organizations small and large, hundreds of thousands of lines of log/stdout/debug feedback, and the warning that 10-20% failures are large at scale.