Building a Multi-Agent Research System for Complex Information Tasks

anthropic · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Multi-agent research systems

Anthropic needed Claude Research to answer complex open-ended questions that require dynamic search, parallel exploration, source attribution, and long-running coordination across web, Google Workspace, and integrations.

Constraints:

• research paths are dynamic and cannot be hardcoded in advance
• single agents have limited context windows and slow sequential search
• multi-agent systems add coordination, evaluation, reliability, and cost complexity
• agents can over-delegate, duplicate work, choose poor tools, or continue searching after enough evidence exists
• long-running stateful agents need durable execution, tracing, careful deployment, and privacy-preserving observability

Architecture

Claude Research uses an orchestrator-worker multi-agent architecture. A LeadResearcher plans the strategy, persists plans to memory, creates specialized subagents, and synthesizes findings. Subagents independently use web, Google Workspace, and integration tools with interleaved thinking, then return compressed findings. A CitationAgent locates supporting citations before the final answer is returned. The system is shaped by prompt heuristics for delegation, effort scaling, tool selection, broad-to-narrow search, and parallel tool use, plus LLM-as-judge evals, human review, full production tracing, checkpoints, retry logic, rainbow deployments, and privacy-preserving decision-pattern monitoring.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  user-query[User query] --> leadresearcher[LeadResearcher]
  leadresearcher --> memory[Memory]
  memory --> leadresearcher
  leadresearcher --> subagents[Subagents]
  subagents --> research-tools[Research tools]
  research-tools --> subagents
  subagents --> leadresearcher
  leadresearcher --> citationagent[CitationAgent]
  citationagent --> final-answer[Final answer]
  evaluation-harness[Evaluation harness] --> leadresearcher
  evaluation-harness --> subagents
  production-tracing[Production tracing] --> leadresearcher
  production-tracing --> subagents
  durable-execution-layer[Durable execution layer] --> leadresearcher
  durable-execution-layer --> subagents
  rainbow-deployments[Rainbow deployments] --> leadresearcher
  rainbow-deployments --> subagents

Components

Data flow

A user submits a research query. The lead agent analyzes complexity, thinks through a plan, stores that plan in memory, and spawns specialized subagents with objectives, output formats, tool/source guidance, and task boundaries. Subagents search independently, use tools in parallel, evaluate findings with interleaved thinking, and return compressed results. The lead agent synthesizes findings, decides whether more research is needed, and then passes documents and report content to a CitationAgent to attach citations before returning the final answer.

Why this architecture

Anthropic chose multi-agent research because open-ended research benefits from breadth-first parallelism, separate context windows, and independent exploration paths. Subagents let the system spend more useful tokens and compress findings for the lead agent, while memory, citations, evals, tracing, checkpoints, and deployments handle the production risks introduced by long-running stateful coordination.

Models used: Claude Opus 4 · Claude Sonnet 4 · Claude Sonnet 3.7 · Claude 4 models · LLM judge

Scale

• Claude Opus 4 lead plus Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2% on Anthropic's internal research eval.
  “a multi-agent system with Claude Opus 4 as the lead agent and Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2% on our internal research eval.”article, stated
• Three factors explained 95% of BrowseComp performance variance, with token usage alone explaining 80%.
  “three factors explained 95% of the performance variance in the BrowseComp evaluation”article, stated

  “We found that token usage by itself explains 80% of the variance, with the number of tool calls and the model choice as the two other explanatory factors.”article, stated
• Agents use about 4x more tokens than chat, and multi-agent systems use about 15x more tokens than chats.
  “agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats.”article, stated
• Prompt rules scale from one agent with 3-10 tool calls to more than 10 subagents for complex research.
  “Simple fact-finding requires just 1 agent with 3-10 tool calls, direct comparisons might need 2-4 subagents with 10-15 calls each, and complex research might use more than 10 subagents”article, stated
• Parallelization cut research time by up to 90% for complex queries.
  “These changes cut research time by up to 90% for complex queries, allowing Research to do more work in minutes instead of hours while covering more information than other systems.”article, stated
• The initial eval set had about 20 real-usage-pattern queries.
  “We started with a set of about 20 queries representing real usage patterns. Testing these queries often allowed us to clearly see the impact of changes.”article, stated
• Prompt tweaks could move success rates from 30% to 80% in early development.
  “A prompt tweak might boost success rates from 30% to 80%. With effect sizes this large, you can spot changes with just a few test cases. We started with a set of about 20 queries representing real usage patterns.”article, stated
• An improved tool description process reduced task completion time by 40%.
  “This process for improving tool ergonomics resulted in a 40% decrease in task completion time for future agents using the new description”article, stated
• The most common Research use categories include software systems at 10%, content at 8%, business growth at 8%, academic material at 7%, and people/place/org verification at 5%.
  “The top use case categories are developing software systems across specialized domains (10%), develop and optimize professional and technical content (8%), develop business growth and revenue generation strategies (8%)”article, stated

  “assist with academic research and educational material development (7%), and research and verify information about people, places, or organizations (5%).”article, stated

Alternatives considered (roads not taken)

Linear one-shot pipeline

Rejected because: Research requires path-dependent exploration and pivots based on intermediate findings.

Static RAG retrieval

Rejected because: Static chunk retrieval cannot dynamically search, adapt to discoveries, and analyze results across a complex research process.

Single-agent research

Rejected because: Single agents search sequentially and have fewer independent context windows, limiting breadth-first tasks.

Multi-agent for every task

Rejected because: Multi-agent systems are expensive and less suitable when tasks are not parallelizable or require tightly shared context.

Synchronous subagent execution forever

Rejected because: Synchronous execution is simpler but creates bottlenecks and blocks the system when one subagent is slow.

Patterns

multi-agent-orchestration (agent_architecture)

A lead agent decomposes research into specialized subagent tasks, coordinates parallel searches, and synthesizes results.

agent-memory (agent_architecture)

The LeadResearcher saves plans and essential context to memory so context-window truncation does not erase the research strategy.

tool-use-orchestration (agent_architecture)

Subagents are given tool/source guidance, and prompts teach tool selection heuristics so agents choose web, Workspace, or specialized tools appropriately.

eval-driven-development (quality_assurance)

Anthropic started with small real-usage eval sets, used rubrics, and iterated prompts/tools from failures.

llm-as-judge (quality_assurance)

Research outputs are scored by an LLM judge against factual accuracy, citation accuracy, completeness, source quality, and tool efficiency.

observability-tracing (operations)

Production tracing tracks agent decisions and interaction structures so failures can be diagnosed without reading conversation contents.

durable-execution (operations)

Long-running agents use retry logic, regular checkpoints, and resumability so failures do not require restarting from scratch.

canary-rollout (release_management)

Rainbow deployments keep old and new versions running while traffic shifts, protecting active long-running agents during deploys.

context-compaction (context_management)

Subagents compress findings for the lead agent, and long-horizon management stores summaries and essential context in memory before continuing.

Tradeoffs

Use multi-agent research for high-value broad tasks

Chose Lead agent plus parallel subagents over Single-agent sequential search.

Rationale: Parallel subagents increase effective token budget and coverage for breadth-first tasks, but cost much more and should be reserved for high-value parallelizable work.

Use prompt heuristics rather than rigid rules

Chose Broad-to-narrow search, delegation, tool-selection, and effort-budget heuristics over Strict fixed workflows.

Rationale: Research is too dynamic for fixed paths, so prompts encode expert research strategies and guardrails.

Use a single LLM judge prompt for evaluation

Chose Single LLM call with scores and pass/fail over Multiple judges for each component.

Rationale: Anthropic found the single-prompt judge more consistent and aligned with human judgments.

Accept synchronous subagent coordination initially

Chose Lead waits for each set of subagents over Fully asynchronous subagent execution.

Rationale: Synchronous execution simplifies coordination today, while asynchronous execution could unlock more parallelism later at the cost of result coordination and state consistency complexity.

Gotchas (what bit them)

Agents over-delegate or duplicate work without explicit delegation guidance.

Root cause: Simple subagent instructions leave objectives, boundaries, output format, and source choices ambiguous.

Resolution: Teach the lead agent to give detailed task descriptions with objectives, output format, tool/source guidance, and task boundaries.

Tool descriptions can send agents down the wrong path.

Root cause: Agents encounter many tools from MCP servers and depend heavily on tool descriptions to choose correctly.

Resolution: Use explicit tool-selection heuristics and improve descriptions through tool-testing agents.

Automated evals miss source-quality failures.

Root cause: Agents may choose SEO-optimized content farms over authoritative sources even if outputs look plausible.

Resolution: Keep human evaluation in the loop and add source-quality heuristics to prompts.

Minor failures compound in long-running agents.

Root cause: Agents maintain state across many tool calls and one failed step can change the trajectory.

Resolution: Use resumable execution, retry logic, checkpoints, and model-mediated graceful error handling.

Deploying changes can break active stateful agents.

Root cause: Agents may be anywhere in a long-running prompt/tool/execution process when code changes ship.

Resolution: Use rainbow deployments to keep old and new versions running while traffic shifts gradually.

Evaluation

Anthropic evaluates Research with internal research evals, BrowseComp analysis, small real-usage query sets, LLM-as-judge rubrics, and human testing. Evaluation focuses on outcome quality rather than fixed paths because multi-agent systems can take different valid routes.

Metrics: 90.2% multi-agent improvement over single-agent Claude Opus 4 on internal research eval · BrowseComp performance variance explained by token usage, tool calls, and model choice · factual accuracy · citation accuracy · completeness · source quality · tool efficiency · 0.0-1.0 judge scores and pass/fail grade · human-discovered edge cases and source-selection biases

Outcomes

• The multi-agent Research system outperformed single-agent Claude Opus 4 by 90.2% on Anthropic's internal research eval.
• Parallel subagents and tool calls reduced complex-query research time by up to 90%.
• Tool-description improvement reduced future task completion time by 40%.
• Research reached production with tracing, checkpoints, retry logic, rainbow deployments, citations, and eval/human-review loops.
• Users report saved time and discovery of business, healthcare, technical, and research connections.

The source reports a 90.2% internal-eval gain, 4x token use for agents versus chat, 15x token use for multi-agent systems versus chats, up to 90% research-time reduction for complex queries, a 40% task-completion-time reduction from better tool descriptions, and top Research use categories from 10% down to 5%.