Building Production-Ready AI Agents: OpenAI Codex CLI Architecture and Agent Loop Design

openai · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Local software-agent harness and coding-agent loop design

OpenAI needed Codex CLI to orchestrate reliable local code changes by coordinating user input, model inference, tool execution, prompt construction, streaming events, security boundaries, prompt caching, stateless API calls, and context compaction across long-running coding conversations.

Constraints:

• The agent can modify the local environment, so its primary output may be code changes rather than a text response.
• A single conversation turn can include many model/tool iterations and potentially hundreds of tool calls.
• Conversation history grows with each turn and can exhaust the model context window.
• Every request includes conversation state, creating apparent quadratic JSON growth over long conversations.
• Zero Data Retention configurations require stateless requests rather than server-side conversation state.
• Prompt caching only works on exact prefix matches, so feature changes can accidentally cause expensive cache misses.
• Codex-provided shell tools and user-provided MCP tools have different sandboxing responsibilities.

Architecture

Codex uses a harness centered on an agent loop. The CLI collects user input, layered instructions, tool definitions, environment context, and conversation history into a Responses API request. The model either emits an assistant message or requests a tool call. Codex executes tool calls, appends the resulting output items to the next request, and repeats until an assistant message ends the turn. The Responses API streams events through SSE; Codex transforms streaming text into UI updates and output items into future input state. To make this production-ready, Codex keeps requests stateless for ZDR, preserves prompt prefixes for caching, appends configuration changes instead of rewriting earlier context, and automatically compacts the conversation through the /responses/compact endpoint when token limits are approached.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  user[User] --> codex-cli[Codex CLI]
  codex-cli --> codex-harness[Codex harness]
  codex-cli --> local-environment[Local environment]
  codex-harness --> prompt-builder[Prompt builder]
  codex-harness --> responses-api-client[Responses API client]
  codex-harness --> tool-executor[Tool executor]
  codex-harness --> conversation-state-manager[Conversation state manager]
  prompt-builder --> instruction-sources[Instruction sources]
  prompt-builder --> tool-definitions[Tool definitions]
  prompt-builder --> conversation-state-manager
  tool-definitions --> codex-shell-tool[Codex shell tool]
  tool-definitions --> mcp-servers[MCP servers]
  responses-api-client --> responses-api-server[Responses API server]
  responses-api-server --> model[Model]
  responses-api-server --> sse-event-stream[SSE event stream]
  sse-event-stream --> internal-event-objects[Internal event objects]
  internal-event-objects --> conversation-state-manager
  internal-event-objects --> codex-cli
  tool-executor --> codex-shell-tool
  tool-executor --> mcp-servers
  tool-executor --> local-environment
  codex-shell-tool --> sandbox-and-approval-context[Sandbox and approval context]
  sandbox-and-approval-context --> conversation-state-manager
  conversation-state-manager --> prompt-cache[Prompt cache]
  prompt-cache --> responses-api-server
  conversation-state-manager --> compaction-endpoint[Compaction endpoint]
  compaction-endpoint --> responses-api-server

Components

Data flow

A user sends a message in Codex CLI. Codex builds a JSON payload with instructions, tools, and input, including developer messages for sandbox and configuration, user instructions from AGENTS files or skills, and the current user message. The request goes to a Responses API endpoint. The SSE stream returns output text deltas for UI streaming and output items such as reasoning/function-call records that Codex appends to the next input. If the model requests a tool call, Codex executes the relevant tool, appends the tool result, and re-queries the model. If the model returns an assistant message, the turn ends. Across turns, conversation history is resent statelessly, optimized by exact-prefix prompt caching and managed by automatic compaction once the context grows too large.

Why this architecture

Codex's architecture prioritizes safe local action, transparent state management, ZDR-compatible statelessness, prompt-cache efficiency, and context-window survival. Keeping the harness responsible for loop orchestration lets the model focus on deciding when to answer or request tools, while Codex handles execution, state replay, caching, compaction, and user-turn boundaries.

Models used: Responses API · OpenAI hosted models · gpt-oss · ollama · LM Studio · Azure-hosted Responses API · Codex CLI · MCP

Scale

• Codex CLI launched in April before the article's retrospective on lessons learned.
  “We’ve learned a tremendous amount about how to build a world-class software agent [since we first launched the CLI in April”article, stated
• Codex includes CLI, Cloud, and VS Code extension offerings sharing the harness.
  ““Codex” encompasses a suite of software agent offerings, including Codex CLI, Codex Cloud, and the Codex VS Code extension.”article, stated
• The CLI can target ChatGPT, OpenAI API, local localhost:11434, or Azure Responses API endpoints.
  4 supporting quotes

  “it uses https://chatgpt.com/backend-api/codex/responses as the endpoint”article, stated

  “it uses https://api.openai.com/v1/responses as the endpoint”article, stated

  “it defaults to http://localhost:11434/v1/responses running locally on your computer”article, stated

  “Codex CLI can be used with the Responses API hosted by a cloud provider such as Azure”article, stated
• User instruction aggregation is subject to a 32 KiB default limit.
  “Subject to a limit (32 KiB, by default), look in each folder from the Git/project root of the cwd”article, stated
• A single turn may include hundreds of tool calls and exhaust the context window.
  “an agent could decide to make hundreds of tool calls in a single turn, potentially exhausting the context window.”article, stated
• Prompt caching can make sampling linear rather than quadratic when cache hits occur.
  “When we get cache hits, sampling the model is linear rather than quadratic.”article, stated
• MCP tool support had a cache-miss bug from inconsistent tool enumeration order.
  “our initial support for MCP tools introduced a [bug where we failed to enumerate the tools in a consistent order”article, stated

Alternatives considered (roads not taken)

Use previous_response_id to avoid resending full history

Rejected because: Codex chooses stateless requests to simplify provider implementation and support Zero Data Retention customers.

Modify earlier prompt items when configuration changes

Rejected because: Changing early prompt content breaks exact-prefix cache hits, so Codex appends new messages for sandbox, approval, and working-directory changes.

Manual /compact only

Rejected because: Codex moved from manual compaction to automatic use of /responses/compact when auto_compact_limit is exceeded.

Assume all tools share Codex sandboxing

Rejected because: Only the Codex-provided shell tool is covered by Codex sandbox instructions; MCP tools must enforce their own guardrails.

Treat the assistant message as the agent's only output

Rejected because: For a software agent, code written or edited on the local machine is often the primary output.

Patterns

tool-use-orchestration (agent-runtime)

The Codex harness repeatedly asks the model whether to answer or request a tool call, executes requested tools, appends outputs, and re-queries until the model emits an assistant message.

context-compaction (context-management)

Codex replaces long conversation input with compacted representative items through /responses/compact when the automatic threshold is exceeded.

semantic-caching (performance)

Prompt caching reuses inference computation when subsequent prompts keep the old prompt as an exact prefix, making sampling linear instead of quadratic under cache hits.

privacy-preserving-processing (data-governance)

Codex avoids previous_response_id so every request is stateless and compatible with Zero Data Retention, while encrypted_content preserves reasoning continuity without storing customer data.

safety-policy-enforcement (tool-security)

Codex scopes sandbox instructions to its shell tool and makes MCP servers responsible for their own guardrails, preserving clear security boundaries by tool source.

prompt-versioning (configuration-management)

Codex layers system, developer, user, assistant, config, AGENTS, project-doc, and skill instructions with explicit precedence rather than a single prompt blob.

fallback-degradation (provider-flexibility)

Codex CLI can use multiple Responses API endpoints, including ChatGPT, OpenAI API keys, local gpt-oss through ollama/LM Studio, and Azure-hosted endpoints.

Tradeoffs

Keep Responses API requests stateless

Chose Full-history stateless requests over previous_response_id statefulness.

Rationale: Stateless requests preserve ZDR compatibility and simplify provider implementation, even though the JSON payload grows over conversation lifetime.

Optimize by preserving prompt prefixes

Chose Append-only prompt evolution where possible over Rewriting earlier prompt items when configuration changes.

Rationale: Exact prefix preservation enables prompt caching and avoids expensive cache misses during long agent sessions.

Automatically compact conversations

Chose /responses/compact with a compaction item over Manual /compact as the normal context-management path.

Rationale: Automatic compaction frees context window space while preserving the model's latent understanding through encrypted content.

Treat assistant message as turn termination, not sole output

Chose Code edits as primary work product plus assistant summary over Text-only chat answer model.

Rationale: A software agent's meaningful output is often file changes on the user's machine, while the assistant message returns control to the user.

Expose multiple Responses API endpoints

Chose ChatGPT, OpenAI API key, local gpt-oss, LM Studio/Ollama, and Azure-compatible endpoints over One fixed hosted inference endpoint.

Rationale: Endpoint flexibility supports different authentication modes, local/open-weight usage, and cloud-provider deployments.

Gotchas (what bit them)

Conversation-history replay makes the agent loop look quadratic.

Root cause: Each new request includes prior conversation messages and tool calls, so JSON sent to the API grows over time.

Resolution: Keep stateless requests for ZDR, but rely on exact-prefix prompt caching so model sampling stays linear under cache hits.

Changing tools mid-conversation can destroy prompt-cache efficiency.

Root cause: Prompt caching requires exact prefix matches, and tool definitions are part of the prompt prefix.

Resolution: Maintain stable tool ordering and be cautious with dynamic MCP tool-list changes.

MCP servers can change available tools while a conversation is in progress.

Root cause: MCP servers can send tool-list-changed notifications, but honoring them changes prompt tool definitions and can trigger expensive cache misses.

Resolution: Treat dynamic MCP tool lists as a performance-sensitive design decision rather than blindly updating prompt state.

Tool security boundaries differ by tool source.

Root cause: Codex sandbox instructions only govern the Codex-provided shell tool; MCP tools are external and must implement their own safeguards.

Resolution: Document and enforce tool-specific guardrail responsibility instead of assuming one sandbox covers every tool.

Long coding conversations can exhaust the context window.

Root cause: Every turn appends conversation history, messages, and tool calls, and a single turn may have hundreds of tool calls.

Resolution: Compact the conversation into a smaller representative item list using /responses/compact when auto_compact_limit is exceeded.

Evaluation

The article is architectural rather than benchmark-driven. It evaluates production readiness through design invariants and operational properties: stateless ZDR compatibility, exact-prefix preservation for prompt caching, automatic compaction before context exhaustion, correct SSE event handling, clean turn termination, and explicit sandbox boundaries for tool sources.

Metrics: Exact prefix preservation for cache hits · Linear rather than quadratic sampling under prompt cache hits · ZDR-compatible stateless requests · Context-window utilization and auto_compact_limit triggering · Consistent MCP tool enumeration order · SSE event transformation into UI deltas and future input items · Assistant-message termination of each turn

Outcomes

• Codex CLI provides a cross-platform local software agent for reliable code changes on a user's machine.
• The Codex harness underlies Codex CLI, Codex Cloud, and the VS Code extension.
• The loop supports many model/tool iterations within a turn while ending with an assistant message that returns control to the user.
• Stateless Responses API requests preserve ZDR compatibility without previous_response_id.
• Prompt prefix preservation and prompt caching mitigate the apparent quadratic cost of resending conversation history.
• Automatic compaction lets Codex continue long conversations without exhausting the context window.
• The source identifies future posts on CLI architecture, tool use, and sandboxing, so this extraction covers the loop and context-management layer only.

The article gives architecture scale rather than usage volume: Codex spans CLI, Cloud, and VS Code extension offerings; user instructions can include up to 32 KiB of project documentation by default; a single turn can involve hundreds of tool calls; and the CLI can target ChatGPT, OpenAI API, local localhost:11434, and Azure Responses API endpoints.