Context Engineering for Production AI Agents at Scale

manus · Tech · 2024 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: General-purpose production AI agent platform context management

Manus needed long-running autonomous agents to keep performing as tool calls, observations, and intermediate work accumulated. Typical tasks could require dozens of tool calls, while production agents may span hundreds of turns, so naive message-history accumulation creates context explosion, context rot, latency, cost, and tool-confusion problems.

Constraints:

Agents run autonomously in loops and append tool observations back into the message history.
Context reduction must avoid losing information that may become important many steps later.
Subagents can isolate work but may increase input-token and KV-cache costs.
Tool exposure must avoid context confusion and invalid calls when tools change.
The architecture must evolve as models improve and scaffolding becomes unnecessary.

Architecture

Manus uses a context-engineering stack with five interacting dimensions: offload bulky tool output to a file system/sandbox, compact reversible tool records before summarization, retrieve context back from files when needed, isolate some tasks with subagents, and preserve cache efficiency through a layered action space of function calls, sandbox utilities, and packages/APIs.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  autonomous-agent-loop["Autonomous agent loop"] --> message-context["Message context"]
  autonomous-agent-loop --> file-system-sandbox["File-system sandbox"]
  message-context --> context-reduction-engine["Context reduction engine"]
  context-reduction-engine --> context-retrieval["Context retrieval"]
  context-retrieval --> file-system-sandbox
  autonomous-agent-loop --> subagent-isolation["Subagent isolation"]
  autonomous-agent-loop --> layered-action-space["Layered action space"]
  layered-action-space --> model-kv-cache-layer["Model/KV cache layer"]
  layered-action-space --> file-system-sandbox

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Autonomous agent loop`	LLM bound to tools that repeatedly calls tools and receives observations while working on user tasks.	`Message context`, `File-system sandbox`, `Context reduction engine`, `Layered action space`
`Message context`	Chat/tool-call history that grows over time and must be managed to avoid context rot.	`Autonomous agent loop`, `Context reduction engine`, `Subagent isolation`
`File-system sandbox`	External state where bulky tool outputs, files, logs, and utilities live outside the model context but remain retrievable.	`Autonomous agent loop`, `Context retrieval`, `Layered action space`
`Context reduction engine`	Compacts reversible tool-call records and applies summarization only when compaction no longer frees enough context.	`Message context`, `File-system sandbox`, `Context retrieval`
`Context retrieval`	Mechanism for reconstructing or retrieving externalized details from paths, files, logs, or search when the agent needs them.	`File-system sandbox`, `Context reduction engine`, `Autonomous agent loop`
`Subagent isolation`	Task decomposition mechanism using either short instruction passing or shared-context subagents.	`Message context`, `Autonomous agent loop`
`Layered action space`	Three-level tool strategy: schema-safe function calls, sandbox utilities, and packages/APIs invoked through shell/files.	`Autonomous agent loop`, `File-system sandbox`, `Model/KV cache layer`
`Model/KV cache layer`	Inference/caching substrate whose cost and reuse depend on prompt size, tool definitions, action space, and provider implementation.	`Layered action space`, `Subagent isolation`

Data flow

The agent loops through tool calls, but bulky outputs are written to files or sandbox utilities instead of staying in messages. Context reduction compacts older tool calls and only summarizes after compaction stops buying enough room. Subagents either receive short instructions or share fuller context depending on the task, while layered actions keep the model-facing interface simple and cache-friendly.

“take the output of a tool message as an example, dump it to the file system, send back to your agent just some minimal piece of information necessary so it can reference the full context if it needs to”transcript, stated

“if you have these three layers from models point all three levels still go through the standard function calls. So the interface stays simple cache friendly and orthogonal across functions”transcript, stated

Why this architecture

Manus treats context management as a balancing problem. Offloading preserves details outside the context window; compaction is reversible where possible; summarization is reserved for later; subagent isolation is chosen by task shape; and layered actions reduce tool-context confusion without losing composability.

“compaction is reversible but summarization isn't both reduce context lengths but they behave very differently”transcript, stated

“offload and retrieve enables more efficient reduction and stable retrieve makes isolation safe. But isolation Oh yeah, isolation also slows down contacts and reduces the frequency of reduction.”transcript, stated

Models used: Frontier LLM providers · Manus agent models · Subagents · Open-source models for internal architecture testing

Scale

Manus cites typical tasks requiring around 50 tool calls.
“Manis talked about their piece that typical tasks require around 50 tool calls.”transcript, stated
The talk cites production agents spanning hundreds of turns.
“Anthropics mentioned similarly that production agents can engage in conversations spanning hundreds of turns.”transcript, stated
Manus had refactored its architecture five times between launch in March and the October talk.
“we have already um refactored Manis for five times and we've launched Manis in March and now it's October already five times.”transcript, stated
The context reduction trigger is tied to pre-rot thresholds around 128K to 200K tokens despite hard limits near 1M tokens.
“most models start degrading much earlier typically maybe around 200k and you'll begin to see what we call a context rot”transcript, stated
“it's typically 128K to to 200K and use it as the trigger for context reduction”transcript, stated

Alternatives considered (roads not taken)

Keep all tool outputs in the message history

Rejected because: Tool calls accumulate unbounded context, causing performance drops as context grows.

“These messages grow over time and so you can kind of get this unbounded explosion messages as agents run.”transcript, stated

“the observation simply is that performance drops as context grows.”transcript, stated

Summarize history first

Rejected because: Summarization is not reversible, so Manus starts with compaction and only summarizes when compaction no longer creates enough room.

“whenever like your context size approaches it you have to trigger context reduction but starting from compaction not summarization”transcript, stated

“when summarizing we always use the full version of the data not the compact one and we still like keep the last few tool calls and tool results in full detail”transcript, stated

Dynamically RAG-load tool descriptions into the model context

Rejected because: Changing front-of-context tool definitions resets KV cache and can leave past calls to removed tools in context, causing invalid calls.

“since tool definitions sit at the front of the context. Yeah, your KV resets every time. And most importantly the model's past calls to remove tools are still in the context.”transcript, stated

Use open-source models primarily to lower inference cost

Rejected because: At Manus scale with long inputs, distributed KV cache quality matters more than raw model cost.

“we often think that open source model can lower the cost but if you're at the scale of manis and and if you're building a real agent which the input is way longer than the output then KV cache is super important”transcript, stated

Patterns

`context-compaction` (context)

Manus strips reconstructable fields out of older tool-call records and keeps paths/references so details can be retrieved from external state.

“The compact version strips out any information that can be like reconstructed from the file system or external state.”transcript, stated

`tool-use-orchestration` (orchestration)

The platform layers action choices into function calls, sandbox utilities, and packages/APIs while keeping the model-facing interface simple.

“we can let manage to choose from three different levels of abstractions. Number one function calling, number two sandbox utilities and number three packages and API.”transcript, stated

`multi-agent-orchestration` (orchestration)

Manus chooses between short-instruction subagents and shared-context subagents depending on whether the task needs only a final output or the full previous context.

“if a task has a like short clear instruction and only the final output matters say like searching a codebase for a specific snippet then just use the communication pattern and keep it simple”transcript, stated

“for more complex scenarios in contrast by sharing memory means that the sub aent can see the entire previous context”transcript, stated

`agent-memory` (context)

The file-system sandbox acts as external working memory for offloaded tool outputs, logs, and files that can be retrieved later.

“before summarizing we might offload key parts of the context into files. And sometimes like we even more do more aggressively we can dump the entire pre-summary context as a text file or simply a log file into the file system”transcript, stated

`cost-aware-routing` (model-operations)

Model/provider decisions account for long-input KV-cache economics, not just nominal open-source model cost.

“if you're building a real agent which the input is way longer than the output then KV cache is super important and distributed KV cache is very hard to implement”transcript, stated

Tradeoffs

Context reduction order

Chose reversible compaction first, summarization later over early free-form summarization.

Rationale: Compaction preserves reconstructability through external state, while summarization loses detail that may matter many steps later.

“We think this kind of like reversibility is crucial because agents do like chain predictions based on previous actions and observations and you never know like which past action will suddenly become super important like 10 steps later.”transcript, stated

Subagent context pattern

Chose communication pattern for clear tasks and shared context for history-dependent tasks over always sharing full context with subagents.

Rationale: Full shared context avoids re-reading for complex work but wastes latency and tokens when only the final result matters.

“like for those uh scenario that requires a full history just use a share memory pattern but be aware that like sharing context is kind of expensive”transcript, stated

Tool exposure model

Chose layered action space through function calls, sandbox utilities, and packages/APIs over injecting all MCP/tool definitions directly into function-calling context.

Rationale: The layered approach avoids tool-context confusion and keeps the model interface cache-friendly while still enabling rich computation.

“We do not inject MCP tools to the function colony space. Instead, we do everything inside that sandbox through in the command line interface.”transcript, stated

Sandbox utility layer

Chose utilities for large outputs over low-latency front-end interactions through the same path.

Rationale: Sandbox utilities offload large outputs well, but they are awkward for front-end interactions that need low latency and visualization.

“it's super good for large outputs but it's also not that good for low latency back and forth interactions with the front end”transcript, stated

Gotchas (what bit them)

Tool-call accumulation creates context rot.

Root cause: Every tool call returns an observation appended to the message list, and performance drops as that context grows.

Resolution: Offload bulky outputs to external files, compact old tool records, and retrieve details only when needed.

“for every tool called you you get a tool observation back and that's appended to this chat list. These messages grow over time”transcript, stated

“performance drops as context grows.”transcript, stated

Compaction can make the model imitate compact formats.

Root cause: If all recent tool examples are compacted, the model may learn to output missing-field compact records.

Resolution: Keep the newest tool calls and results in full detail while compacting older history.

“we might compactate like the oldest 50% of tool calls while keeping the newer ones in full detail so the model still has fresh viewshot examples to of like how to use tools properly.”transcript, stated

Shared-context subagents are expensive.

Root cause: When a subagent sees the full previous context, it has a larger input prefill and cannot reuse the KV cache because prompts/action spaces differ.

Resolution: Use short-instruction communication when the final result is enough; reserve shared context for tasks that require full history.

“sharing context is kind of expensive because you know each sub aent has a larger input to prefill which is like you'll spend more on like input tokens”transcript, stated

“since the system prompt and the access space differs you cannot re reuse the KV cache so you have to pay the full price”transcript, stated

Context over-engineering can hurt the product.

Root cause: Extra context-management layers and retrieval tricks can make the model's job harder as models improve.

Resolution: Regularly simplify architecture and remove scaffolding when better models make it unnecessary.

“Every time we simplify the architecture, the system got faster, more stable, and smarter because we think context engineering should uh the goal of context engineering is to make the model's job simpler but not harder.”transcript, stated

Evaluation

Manus evaluates architecture by observing context-rot thresholds, testing weaker versus stronger models under a fixed agent architecture, and revisiting scaffolding every one or two months as model behavior changes.

Metrics: pre-rot context thresholds around 128K-200K tokens · free context gained after compaction · weaker-to-stronger model architecture gain · one- or two-month architecture review cadence

“it's typically 128K to to 200K and use it as the trigger for context reduction”transcript, stated

“we fix the AR agent architecture and we switch between models. If if like your architecture can gain a lot from switching from a weaker model to a stronger model then somehow your your architecture is more futurep proof”transcript, stated

Outcomes

Manus identifies context offload, reduction, retrieval, isolation, and caching as the interacting context-engineering dimensions for production agents.
The team refactored Manus five times after launch as models and architecture assumptions changed.
Simplification produced the biggest gains in speed, stability, and apparent intelligence.

The talk frames Manus as a production platform with typical tasks around 50 tool calls, long-running agents, and architecture reviews every one or two months.

“we have already um refactored Manis for five times and we've launched Manis in March and now it's October already five times.”transcript, stated

“Every time we simplify the architecture, the system got faster, more stable, and smarter”transcript, stated