Context Engineering and Agent Development at Scale: Building Open Deep Research

langchain · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: deep-research agents and context engineering

LangChain needed Open Deep Research to manage the context explosion created by long-running tool-calling agents, while continuing to adapt as model capabilities improved.

Constraints:

• Deep research agents can execute dozens to hundreds of tool calls.
• Naively appending tool outputs can hit context windows, increase cost, and trigger long-context degradation.
• Compaction and summarization risk losing information if raw context is discarded.
• Multi-agent designs work well for read-only parallel research but can conflict when subagents make independent write decisions.
• Agent scaffolding can become a bottleneck as models improve.

Architecture

Open Deep Research evolved from a structured research workflow into a more general tool-calling agent built with low-level orchestration primitives. Its context strategy combines offloading token-heavy tool observations to disk or state, high-recall summaries for retrieval decisions, compaction at tool-call boundaries, caching for repeated message history, parallel read-only subagents for context gathering, and a final one-shot writing step to avoid conflicting subagent-written sections.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  research-request["research request"] --> open-deep-research-agent["Open Deep Research agent"]
  open-deep-research-agent --> tool-calling-loop["tool-calling loop"]
  tool-calling-loop --> retrieval-and-tool-context["retrieval and tool context"]
  tool-calling-loop --> context-offload-store["context offload store"]
  context-offload-store --> summary-and-compaction-layer["summary and compaction layer"]
  summary-and-compaction-layer --> open-deep-research-agent
  open-deep-research-agent --> parallel-read-only-subagents["parallel read-only subagents"]
  parallel-read-only-subagents --> final-report-writer["final report writer"]
  open-deep-research-agent --> final-report-writer
  low-level-orchestration-runtime["low-level orchestration runtime"] --> open-deep-research-agent
  low-level-orchestration-runtime --> context-offload-store
  provider-prompt-cache["provider prompt cache"] --> open-deep-research-agent

Components

Data flow

A research request enters a LangGraph-style agent or workflow. The agent calls tools, offloads token-heavy observations to external storage/state, writes compact summaries back into the message history, optionally uses retrieval and caching, parallelizes read-only context gathering across subagents when appropriate, then performs final writing in one step from shared gathered context.

Why this architecture

The architecture focuses on context engineering rather than simply adding more prompt text. It keeps raw observations retrievable, compresses high-recall summaries into the active context, isolates parallel research work where it is safe, and avoids high-level agent abstractions so the system can be rebuilt as model capabilities improve.

Models used: LLM tool-calling agent · summarization model · fine-tuned summarization model as cited from other agent systems · LangGraph runtime

Scale

• Naive Open Deep Research runs used about 500,000 tokens.
  “my deep researcher, for example, with a naive tool calling loop was using 500,000 tokens.”transcript, stated
• The naive runs cost roughly one to two dollars per run.
  “It was like a dollar to $2 per run.”transcript, stated
• Relevant agent workloads can involve about 50 tool calls in Manis-like tasks.
  “they made the point that the typical Manis task is like 50 tool calls.”transcript, stated
• Anthropic-style production agents were described as reaching hundreds of tool calls.
  “the typical production agent and this is probably referring to Claude Code could be other agents that they've produced is like hundreds of tool calls.”transcript, stated
• Open Deep Research evolved over about a year.
  “building deep research over the course of a year.”transcript, stated
• The initial structured workflow dated to 2024-era model capabilities.
  “This was back in 2024. Decompose the problem into a set of sections and parallelize each one”transcript, stated

Alternatives considered (roads not taken)

naively append all tool outputs to the message history

Rejected because: It caused token-heavy runs, context-window pressure, and long-context performance degradation.

irreversible pruning/summarization without raw context

Rejected because: Summarization can lose information, so raw context should remain available through offloading.

multi-agent subagents that independently write final sections

Rejected because: Independent subagent writing produced disjoint reports because subagents did not communicate enough.

highly structured 2024-style research workflow

Rejected because: It initially improved reliability but later blocked tool-calling and MCP benefits as models improved.

Patterns

context-compaction (context-management)

Open Deep Research summarizes and compacts token-heavy tool results while preserving enough detail for later retrieval decisions.

agent-memory (context-management)

The design offloads raw tool observations to disk or runtime state so the agent can fetch them later instead of carrying everything in active context.

multi-agent-orchestration (agent-architecture)

Parallel subagents are used where the work is read-only context gathering and can be merged by a final writing step.

tool-use-orchestration (agent-architecture)

Open Deep Research moved toward a tool-calling agent that lets the model choose the research path.

semantic-caching (serving)

The discussion highlights caching repeated prior message history to reduce latency and cost in looped agents.

durable-execution (orchestration)

LangGraph is valued for low-level state management and checkpointing rather than opaque agent abstractions.

Tradeoffs

Offload raw tool outputs while summarizing active context.

Chose externalized raw context plus compact summaries over passing all tool outputs back into the message history.

Rationale: This keeps raw evidence retrievable while reducing token pressure in the active model context.

Use multi-agent parallelism only where task boundaries are clean.

Chose read-only parallel context gathering with one final write over subagents independently writing final solution components.

Rationale: Read-only research is easier to parallelize without conflicting decisions.

Move from a structured workflow to a more general agent as models improved.

Chose tool-calling agent deciding the research path over fixed section-decomposition workflow.

Rationale: The structured workflow was reliable for older models but blocked MCP/tool-calling improvements.

Use low-level orchestration rather than opaque agent abstractions.

Chose LangGraph-style nodes, edges, state, checkpointing, and state management over high-level import-agent abstractions.

Rationale: Low-level primitives are easier to tear down and recombine as assumptions change.

Cache repeated message history where possible but still reduce context.

Chose provider prompt caching plus explicit context management over treating caching as a substitute for context engineering.

Rationale: Caching helps latency and cost, but long-context failure modes remain.

Gotchas (what bit them)

Naive tool-call accumulation made deep research expensive and context-heavy.

Root cause: Each token-heavy tool result was pushed back into the model context at every turn.

Resolution: Offload raw observations, summarize what is available, and retrieve the raw context on demand.

Compaction and pruning can erase important evidence.

Root cause: Summaries are lossy, and irreversible pruning removes the raw context needed to recover.

Resolution: Keep raw tool outputs in an offload store while passing compact summaries back to the model.

Multi-agent designs can conflict when subagents make write decisions independently.

Root cause: Subagents implicitly make decisions without enough communication, producing coordination conflicts.

Resolution: Use subagents for parallel read-only context gathering and perform final writing in one step.

Useful structure can become a bottleneck as models improve.

Root cause: The early workflow encoded assumptions about research and tool reliability that stopped fitting newer models.

Resolution: Continuously reassess scaffolding and remove structure that blocks improved model capabilities.

Caching reduces cost but does not solve long-context quality degradation.

Root cause: Even cached tokens remain part of the model's long context, so context rot can still apply.

Resolution: Use caching for latency/cost, but still manage context length and relevance explicitly.

High-level agent abstractions can make the system hard to rebuild.

Root cause: Opaque abstractions hide what is under the hood, which is risky while agent designs are still immature.

Resolution: Prefer low-level orchestration building blocks such as nodes, edges, state, checkpointing, and state management.

Evaluation

The discussion references iterative development over a year, Deep Research Bench performance, and empirical costs/tokens from naive tool-calling runs, but does not describe a formal release gate or benchmark protocol in the transcript.

Metrics: 500,000 tokens in naive runs · $1 to $2 per naive run · typical agent workloads of 50 to hundreds of tool calls · best-performing open-source deep research assistant on Deep Research Bench

Outcomes

• Open Deep Research became a strong open-source deep research assistant on Deep Research Bench.
• The system evolved away from a structured 2024 workflow toward a more general tool-calling agent.
• The final architecture keeps subagent research read-only and performs final writing centrally.

The system was developed across roughly a year, after naive deep-research runs reached about 500,000 tokens and one to two dollars per run.