Building Production Multi-Agent Research Systems with Claude

anthropic · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production multi-agent research over web, Google Workspace, and integrated enterprise data sources

Anthropic needed Claude Research to answer open-ended research questions that require dynamic planning, parallel search, tool use, citation grounding, long-running state, evaluation, and reliable production operation across many external data sources.

Constraints:

• Research tasks are open-ended and path-dependent, so the required steps cannot be hardcoded in advance.
• Useful research often exceeds a single context window and requires exploring many directions at once.
• Multi-agent systems are expensive because they consume far more tokens than chat interactions.
• Agent behavior is non-deterministic, so evaluation cannot rely on checking a single prescribed path.
• Long-running agents maintain state across many tool calls, making restarts expensive and error compounding severe.
• Production deployments must avoid breaking already-running agents that may be in any part of the research process.

Architecture

Claude Research uses an orchestrator-worker architecture. A LeadResearcher agent analyzes the user query, thinks through a plan, saves it to external memory to survive context truncation, creates specialized subagents for parallel search, receives compressed findings, decides whether more research is needed, and then hands the result to a CitationAgent that grounds claims in source locations. Subagents independently search web and integrated sources, evaluate tool results with interleaved thinking, and return findings. Production reliability layers include detailed prompts for delegation and effort scaling, MCP/tool-description heuristics, small-sample and LLM-judge evaluation, human evaluation, full production tracing, retry logic, checkpoints, graceful tool-failure handling, and rainbow deployments.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  user-query[User query] --> leadresearcher-agent[LeadResearcher agent]
  leadresearcher-agent --> memory[Memory]
  leadresearcher-agent --> subagent-pool[Subagent pool]
  subagent-pool --> search-and-integration-tools[Search and integration tools]
  subagent-pool --> interleaved-thinking[Interleaved thinking]
  subagent-pool --> leadresearcher-agent
  leadresearcher-agent --> citationagent[CitationAgent]
  citationagent --> final-cited-answer[Final cited answer]
  evaluation-suite[Evaluation suite] --> leadresearcher-agent
  leadresearcher-agent --> production-tracing[Production tracing]
  production-tracing --> evaluation-suite
  production-tracing --> retry-and-checkpoint-layer[Retry and checkpoint layer]
  retry-and-checkpoint-layer --> leadresearcher-agent
  rainbow-deployment-system[Rainbow deployment system] --> leadresearcher-agent
  tool-description-testing[Tool-description testing] --> search-and-integration-tools

Components

Data flow

A user submits a research query. The LeadResearcher analyzes it, develops a strategy, writes the plan to Memory, and spawns specialized subagents. Each subagent uses search tools and other integrations to explore an assigned aspect, applies interleaved thinking to evaluate results, and returns compressed findings. The LeadResearcher synthesizes findings, may create more subagents or refine the strategy, and exits the loop once enough information is gathered. A CitationAgent then processes the documents and research report, identifies source locations for citations, and returns a cited answer to the user. Evaluation and production tracing run around this loop to judge outcomes, inspect decisions, and diagnose failures.

Why this architecture

Anthropic chose a multi-agent orchestrator-worker architecture because complex research benefits from breadth, parallelism, separate context windows, and dynamic search. Static RAG and single-agent sequential search are weaker fits when the path is unknown, the information exceeds one context window, and multiple independent lines of inquiry must be compressed into a final answer.

Models used: Claude Opus 4 · Claude Sonnet 4 · Claude Sonnet 3.7 · Claude 4 models · LLM judge · MCP servers · Google Workspace integrations · Web search tools

Scale

• The system supports search across the web, Google Workspace, and integrations.
  “allow it to search across the web, Google Workspace, and any integrations to accomplish complex tasks.”article, stated
• The system persists plans because context can exceed 200,000 tokens.
  “since if the context window exceeds 200,000 tokens it will be truncated and it is important to retain the plan.”article, stated
• The multi-agent research system outperformed a single-agent Claude Opus 4 baseline by 90.2% on internal research evals.
  “outperformed single-agent Claude Opus 4 by 90.2% on our internal research eval.”article, stated
• Token usage alone explained 80% of performance variance in Anthropic's BrowseComp analysis.
  “We found that token usage by itself explains 80% of the variance, with the number of tool calls and the model choice as the two other explanatory factors.”article, stated
• Agents use about 4x more tokens than chat interactions, while multi-agent systems use about 15x more tokens.
  “agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats.”article, stated
• Prompt effort guidelines range from one agent with 3-10 tool calls to more than 10 subagents for complex research.
  “Simple fact-finding requires just 1 agent with 3-10 tool calls, direct comparisons might need 2-4 subagents with 10-15 calls each”article, stated
• Parallelization uses 3-5 subagents and 3+ tools in parallel, cutting complex research time by up to 90%.
  “the lead agent spins up 3-5 subagents in parallel rather than serially; (2) the subagents use 3+ tools in parallel.”article, stated

  “These changes cut research time by up to 90% for complex queries”article, stated
• The early evaluation set started with about 20 queries.
  “We started with a set of about 20 queries representing real usage patterns.”article, stated
• The LLM judge scored outputs from 0.0 to 1.0 and produced a pass-fail grade.
  “a single prompt outputting scores from 0.0-1.0 and a pass-fail grade was the most consistent and aligned with human judgements.”article, stated
• Agent-assisted tool-description work reduced future task completion time by 40%.
  “This process for improving tool ergonomics resulted in a 40% decrease in task completion time for future agents using the new description”article, stated

Alternatives considered (roads not taken)

Static RAG over similar chunks

Rejected because: Anthropic contrasts static chunk retrieval with dynamic multi-step search that adapts to findings and analyzes results.

A single-agent Claude Opus 4 research system

Rejected because: The multi-agent system performed better on internal research evals and succeeded on breadth-first S&P 500 board-member research where the single agent failed.

Sequential search execution

Rejected because: Early agents that searched sequentially were too slow, so Anthropic added parallel subagents and parallel tool use.

Rigid scripted process evaluation

Rejected because: Multi-agent systems may take different valid paths, so evaluation must judge outcomes and reasonable process rather than exact step sequences.

Updating all running agents simultaneously

Rejected because: Agent systems are stateful and run continuously, so Anthropic uses rainbow deployments to keep old and new versions running while shifting traffic.

Restarting agents from the beginning after errors

Rejected because: Restarts are expensive and frustrating, so the production system resumes from where the agent failed.

Patterns

multi-agent-orchestration (agent-architecture)

The LeadResearcher coordinates specialized subagents that search in parallel, compress findings, and return them for synthesis.

tool-use-orchestration (agent-tools)

Agents use web, Google Workspace, MCP, and other tools, with explicit heuristics for matching tool choice to user intent and avoiding wrong-tool failures.

context-compaction (context-management)

Subagents operate with their own context windows and return compressed findings; the lead also saves plans to memory before context truncation.

agent-memory (state)

The LeadResearcher writes its plan to external Memory to preserve critical context across long research loops.

eval-driven-development (quality)

The team started evaluation early with small real-usage query sets, then used those tests to guide prompt and system changes.

llm-as-judge (evaluation)

Research outputs are graded by an LLM judge against a rubric for factual accuracy, citation accuracy, completeness, source quality, and tool efficiency.

observability-tracing (operations)

Full production tracing and high-level decision-pattern monitoring are used to diagnose why agents fail without inspecting individual conversation contents.

durable-execution (reliability)

Long-running agents can resume from failures using retry logic, checkpoints, and deterministic safeguards rather than restarting from scratch.

canary-rollout (deployment)

Rainbow deployments gradually shift traffic from old to new versions while both continue running so stateful agents are not disrupted.

latency-optimization (performance)

Parallel subagents and parallel tool calls replace slow sequential searches, reducing complex research time by up to 90%.

cost-aware-routing (economics)

The architecture is reserved for high-value tasks because multi-agent systems spend enough tokens to improve performance but cost roughly 15x standard chat token usage.

Tradeoffs

Use multi-agent search for Claude Research

Chose Lead agent plus specialized parallel subagents over Single-agent sequential search.

Rationale: The multi-agent system outperformed the single-agent baseline on internal evals and handled breadth-first research by decomposing independent directions.

Spend more tokens to buy research capacity

Chose High-token multi-agent execution over Lower-cost chat-like interaction.

Rationale: The data showed token usage strongly explained performance variance, but the system must be reserved for tasks valuable enough to pay the cost.

Use flexible outcome/process evaluation

Chose LLM judge, human review, and reasonable-process checks over Validating a prescribed sequence of steps.

Rationale: Agents can reach the same correct outcome through different paths, so the eval must judge result quality and process reasonableness.

Keep synchronous lead/subagent coordination for now

Chose Lead waits for subagent sets to complete over Fully asynchronous subagent coordination.

Rationale: Synchronous execution simplifies coordination today, while asynchronous execution promises more parallelism at the cost of state and error-propagation complexity.

Use rainbow deployments for stateful agent systems

Chose Gradually shift traffic while old and new versions both run over Updating every agent to the new version at once.

Rationale: Running agents may be anywhere in their process, so abrupt updates can break stateful workflows.

Use prompt heuristics instead of rigid rules

Chose Frameworks for collaboration, effort budgets, and search strategy over Strict step-by-step prompt scripts.

Rationale: The system needs flexible research behavior but still needs guardrails against spiraling, over-searching, and duplicate work.

Gotchas (what bit them)

Multi-agent research is only worth the cost for sufficiently valuable, parallelizable tasks.

Root cause: The architecture gains capability by spending many more tokens and tool calls than chat or single-agent approaches.

Resolution: Apply it where the task value, parallelism, context demand, and tool complexity justify the 15x token usage.

Poor delegation prompts cause duplicate work and gaps.

Root cause: Simple subagent instructions are too vague for coordinated research, so subagents may misinterpret tasks or repeat each other.

Resolution: Give each subagent an objective, output format, tool/source guidance, and clear task boundaries.

Agents over-invest in simple queries unless effort is explicitly scaled.

Root cause: Agents struggle to judge appropriate effort for different task complexity levels.

Resolution: Embed scaling rules in prompts for simple, comparative, and complex research tasks.

Bad tool descriptions send agents down wrong paths.

Root cause: With MCP and many external tools, agents encounter unseen tools with uneven description quality.

Resolution: Use explicit tool-choice heuristics and agent-assisted tool-description testing to improve future tool ergonomics.

Automated evals miss source-quality and edge-case failures.

Root cause: Research outputs are free-form, and agents can choose plausible-looking but low-quality sources.

Resolution: Keep human testing in the loop and add source-quality heuristics to prompts.

Small prompt or code changes can cascade into large behavior changes.

Root cause: Subagents interact with the lead agent in a non-deterministic, emergent system rather than a fixed execution path.

Resolution: Use tracing, interaction-pattern monitoring, careful prompting, and controlled deployments to diagnose and isolate changes.

Synchronous subagent execution bottlenecks coordination.

Root cause: The lead waits for each set of subagents before proceeding, preventing steering and coordination while work is underway.

Resolution: The team identifies asynchronous execution as a future path, while acknowledging result coordination, state consistency, and error propagation complexity.

Evaluation

Anthropic evaluates Claude Research with small real-usage query sets, LLM-as-judge rubrics, human evaluation, process/outcome checks suited to non-deterministic paths, production tracing, and end-state evaluation for state-mutating agents. The LLM judge grades factual accuracy, citation accuracy, completeness, source quality, and tool efficiency, while human testers catch edge cases and source-selection failures.

Metrics: Internal research eval uplift versus single-agent Claude Opus 4 · BrowseComp performance variance explained by token usage, tool calls, and model choice · Small initial set of about 20 real-usage queries · LLM judge rubric scores from 0.0 to 1.0 plus pass/fail · Factual accuracy · Citation accuracy · Completeness · Source quality · Tool efficiency · Human tester findings on hallucinations, failures, and source-selection bias

Outcomes

• The multi-agent system powers Claude Research across web, Google Workspace, and integrations.
• The multi-agent configuration with Claude Opus 4 lead and Claude Sonnet 4 subagents outperformed a single-agent Claude Opus 4 baseline by 90.2% on internal research evals.
• Token usage, tool calls, and model choice explained 95% of BrowseComp performance variance, with token usage alone explaining 80%.
• Parallel subagents and parallel tool calling cut complex research time by up to 90%.
• Tool-description testing with agents reduced future task completion time by 40%.
• Production tracing, checkpoints, graceful tool-failure handling, and rainbow deployments made the stateful agent system operable at scale.
• Users reported finding business opportunities, navigating healthcare options, resolving technical bugs, and saving up to days of work.

The article describes production Claude Research rather than a prototype: it searches web, Google Workspace, and integrations; persists plans around 200,000-token context limits; uses 3-5 parallel subagents and 3+ tools in parallel for some complex queries; consumes about 15x chat tokens; and reports top Research categories such as software systems (10%) and professional content (8%).