Context Engineering Strategies for Production AI Agents

manus · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production general-purpose AI agents

Manus had to build an agent architecture that could improve quickly without training a custom end-to-end model, while keeping long tool-use loops fast, cheap, and stable as context grows and the action space expands.

Constraints:

Architecture

Manus chose context engineering on top of frontier models. Its production loop keeps prompt prefixes stable and append-only for KV-cache reuse, masks tools instead of dynamically removing them, uses the file system as restorable external memory, keeps task goals recent via todo.md recitation, preserves failed actions for recovery, and adds controlled context variation to avoid repetitive-action ruts.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  user-request["User request"] --> agent-loop["Agent loop"]
  agent-loop --> virtual-machine-sandbox["Virtual machine sandbox"]
  virtual-machine-sandbox --> file-system-memory["File-system memory"]
  agent-loop --> action-space-state-machine["Action-space state machine"]
  action-space-state-machine --> append-only-context["Append-only context"]
  agent-loop --> append-only-context
  append-only-context --> kv-cache-layer["KV-cache layer"]
  agent-loop --> todo-memory["Todo memory"]
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
User request Initial task input that starts the Manus agent loop. Agent loop
Agent loop Model-driven loop that selects actions, receives observations, and continues until task completion. User request, Virtual machine sandbox, Action-space state machine, Append-only context, Todo memory
Virtual machine sandbox Execution environment for actions and externalized file-system state. Agent loop, File-system memory
Action-space state machine Context-aware tool-availability controller that masks token logits rather than removing tools mid-iteration. Agent loop, Append-only context
Append-only context Deterministic action/observation history optimized for KV-cache stability. Agent loop, KV-cache layer
File-system memory Externalized, restorable context store for files, documents, web pages, and long-lived state. Virtual machine sandbox, Agent loop
Todo memory Natural-language task recitation mechanism that keeps objectives in recent attention. Agent loop
KV-cache layer Inference cache whose hit rate determines much of the cost and latency profile for long-input agent loops. Append-only context

Data flow

A user request enters the Manus agent loop. The model selects actions from a predefined action space; the environment executes them in a virtual machine sandbox; observations append to context. To keep the loop stable, Manus preserves deterministic append-only context, offloads restorable details to files, masks unavailable tools by state, and keeps task objectives near the end of context.

“That action is then executed in the environment (e.g., Manus's virtual machine sandbox) to produce an observation. The action and observation are appended to the context, forming the input for the next iteration.”article, stated
“By constantly rewriting the todo list, Manus is reciting its objectives into the end of the context. This pushes the global plan into the model's recent attention span”article, stated

Why this architecture

Manus chose context engineering because it allowed faster iteration than fine-tuning and kept the product orthogonal to fast-moving model progress. The architecture focuses on KV-cache stability because agent inputs dwarf outputs and cached prefixes materially change cost and latency.

“That hard-earned lesson made the choice clear: Manus would bet on context engineering. This allows us to ship improvements in hours instead of weeks”article, stated
“If I had to choose just one metric, I'd argue that the KV-cache hit rate is the single most important metric for a production-stage AI agent. It directly affects both latency and cost.”article, stated

Models used: Frontier models · Claude Sonnet · GPT-3 · Flan-T5

Scale

Alternatives considered (roads not taken)

Train an end-to-end agentic model on open-source foundations

Rejected because: Past fine-tuning cycles took weeks and were vulnerable to frontier model progress making the custom model obsolete.

“should we train an end-to-end agentic model using open-source foundations, or build an agent on top of the in-context learning abilities of frontier models?”article, stated
“That hard-earned lesson made the choice clear: Manus would bet on context engineering.”article, stated

Dynamically add or remove tools mid-iteration

Rejected because: Tool-definition changes invalidate KV-cache prefixes and confuse the model when old actions reference removed tools.

“unless absolutely necessary, avoid dynamically adding or removing tools mid-iteration.”article, stated
“any change will invalidate the KV-cache for all subsequent actions and observations.”article, stated

Overly aggressive compression of context

Rejected because: Irreversible compression can remove observations needed for future action prediction.

“overly aggressive compression inevitably leads to information loss. The problem is fundamental: an agent, by nature, must predict the next action based on all prior state”article, stated

Patterns

semantic-caching (model-operations)

Manus keeps prompt prefixes stable, context append-only, and cache breakpoints explicit because KV-cache hit rate drives agent latency and cost.

“If I had to choose just one metric, I'd argue that the KV-cache hit rate is the single most important metric for a production-stage AI agent.”article, stated
“Make your context append-only. Avoid modifying previous actions or observations. Ensure your serialization is deterministic.”article, stated

tool-use-orchestration (orchestration)

Tool availability is controlled by a context-aware state machine and logit masking instead of changing the serialized tool set.

“Rather than removing tools, it masks the token logits during decoding to prevent (or enforce) the selection of certain actions based on the current context.”article, stated

agent-memory (context)

The file system is used as externalized structured memory that the model can write to and read from on demand.

“we treat the file system as the ultimate context in Manus: unlimited in size, persistent by nature, and directly operable by the agent itself.”article, stated

context-compaction (context)

Manus shrinks context by dropping restorable content when URLs or sandbox paths remain available.

“the content of a web page can be dropped from the context as long as the URL is preserved, and a document's contents can be omitted if its path remains available in the sandbox.”article, stated

fallback-degradation (reliability)

Failed actions and stack traces are intentionally preserved so the model can update its beliefs and avoid repeating mistakes.

“leave the wrong turns in the context. When the model sees a failed action—and the resulting observation or stack trace—it implicitly updates its internal beliefs.”article, stated

Tradeoffs

Product adaptation strategy

Chose context engineering on frontier models over training an end-to-end open-source agentic model.

Rationale: Context engineering let the team ship improvements in hours instead of weeks and stay aligned with model progress.

“This allows us to ship improvements in hours instead of weeks, and kept our product orthogonal to the underlying models”article, stated

Tool availability control

Chose masking token logits based on context state over removing tools from the serialized action space.

Rationale: Masking avoids KV-cache invalidation and stale references while still enforcing state-specific tool choices.

“Rather than removing tools, it masks the token logits during decoding to prevent (or enforce) the selection of certain actions based on the current context.”article, stated

Context compression

Chose restorable file-based externalization over irreversible compression/truncation.

Rationale: Agents may need any prior observation later, so Manus keeps a path or URL that can restore omitted content.

“This allows Manus to shrink context length without permanently losing information.”article, stated

Gotchas (what bit them)

KV-cache can be silently destroyed by unstable prompt prefixes.

Root cause: Even a single-token difference near the beginning of the prompt invalidates the cache from that point onward.

Resolution: Keep prompt prefixes stable, avoid precise timestamps at the start, and serialize context deterministically.

“even a single-token difference can invalidate the cache from that token onward. A common mistake is including a timestamp—especially one precise to the second—at the beginning of the system prompt.”article, stated

Dynamic tool loading makes the model dumber.

Root cause: Large user-configurable action spaces and mid-iteration tool changes increase wrong actions, schema violations, and hallucinated actions.

Resolution: Keep tools serialized and use state-machine/logit masking to prevent or enforce actions.

“If you allow user-configurable tools, trust me: someone will inevitably plug hundreds of mysterious tools into your carefully curated action space. As a result, the model is more likely to select the wrong action”article, stated

Long contexts still create lost-goal behavior.

Root cause: A 50-tool-call loop is vulnerable to drifting off-topic or forgetting earlier goals.

Resolution: Use todo.md task recitation to keep the plan in the recent attention span.

“That's a long loop—and since Manus relies on LLMs for decision-making, it's vulnerable to drifting off-topic or forgetting earlier goals”article, stated

Uniform context can create repetitive action ruts.

Root cause: Models imitate repeated action-observation patterns even when they are no longer optimal.

Resolution: Add small controlled variation in serialization, phrasing, order, or formatting.

“If your context is full of similar past action-observation pairs, the model will tend to follow that pattern, even when it's no longer optimal.”article, stated

Evaluation

Manus judges context architecture through production metrics and behavior: KV-cache hit rate, latency/cost, action stability, task drift, recovery after errors, and repeated real-world rewrites.

Metrics: KV-cache hit rate · time-to-first-token · inference cost · input-to-output token ratio around 100:1 · typical 50 tool calls per task

“If I had to choose just one metric, I'd argue that the KV-cache hit rate is the single most important metric for a production-stage AI agent.”article, stated
“At Manus, we've learned these lessons through repeated rewrites, dead ends, and real-world testing across millions of users.”article, stated

Outcomes

The article cites millions of users, average 50 tool calls per task, 100:1 input/output token ratio, and a 10x cached-vs-uncached Claude Sonnet input-token price difference.

“we've rebuilt our agent framework four times, each time after discovering a better way to shape context.”article, stated
“How you shape the context ultimately defines how your agent behaves: how fast it runs, how well it recovers, and how far it scales.”article, stated