Scaling Deep Research Agents through Architecture Optimization and Context Management

tavily_/_nebius · Tech · 2026 · source Agentic: yes · promotional bias: med · summary-only (no fetched source) · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: deep research agent scaling

Tavily needed to scale long-running deep research agents while controlling context growth, quality drift, latency, and token cost.

Constraints:

• Research tasks run for one to ten minutes and accumulate context across many steps.
• The system serves enterprise customers and processes very large weekly token volumes.
• The source cache is unavailable, so this extraction is based on the ZenML summary only.

Architecture

Tavily replaced a ReAct-style single-agent loop with a supervisor/sub-agent architecture and reflection tools. The supervisor decomposes research tasks, dispatches focused sub-agents with isolated context windows, runs parallel searches and extraction, and synthesizes final reports. Reflection distills raw page content at step boundaries so only essential knowledge carries forward.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  A["Research task interface"] --> B["Supervisor agent"]
  B --> C["Research sub-agents"]
  C --> D["Search and extraction tools"]
  C --> E["Reflection tool"]
  E --> F["Report synthesizer"]
  B --> F
  F --> G["Deep Research Bench evaluator"]

Components

Data flow

A user or invoking agent submits a research task with model, schema, and output preferences. The supervisor decomposes it into sub-tasks, activates focused research sub-agents, each sub-agent searches and extracts internet information, reflection tools distill findings into compact state, and a report synthesizer returns a long structured report with citations.

Why this architecture

The supervisor/sub-agent pattern limits context contamination and enables parallelism; reflection tools prevent raw content from accumulating unboundedly; benchmark evaluation checks both final-report quality and source-grounded retrieval understanding.

Models used: user-selected models · LLM evaluators for Deep Research Bench

Scale

• Tavily served over 180 enterprise customers.
  “Tavily, recently acquired by Nebius, developed a production-scale deep research agent serving over 180 enterprise customers and processing 30 billion tokens weekly.”summary, stated
• The system processed 30 billion tokens weekly.
  “Tavily, recently acquired by Nebius, developed a production-scale deep research agent serving over 180 enterprise customers and processing 30 billion tokens weekly.”summary, stated
• Over 200 agents can be running concurrently.
  “At any given time, over 200 agents are running concurrently in the air.”summary, stated
• Each invocation triggers approximately 55 tool calls.
  “Each invocation triggers approximately 55 different tool calls.”summary, stated
• Typical research-task execution time ranges from one to ten minutes.
  “The typical execution time for a research task ranges from one to ten minutes”summary, stated
• The evaluated benchmark had 100 research cases across 22 topics and two languages.
  “100 different research test cases • Coverage across 22 different topics • Support for two languages: English and Chinese”summary, stated

Alternatives considered (roads not taken)

ReAct single-agent architecture

Rejected because: It accumulated more context with every step and reached overflow problems.

Carry all prior raw knowledge forward

Rejected because: It creates a polynomial dependency on the number of steps and increases token consumption.

Single end-to-end report score

Rejected because: The team also needed to verify retrieval quality and whether cited pages truly supported report claims.

Patterns

multi-agent-orchestration (agent-architecture)

A supervisor decomposes research work and activates focused sub-agents for specific tasks.

context-engineering (context-management)

Tavily deliberately separates sub-agent contexts and controls what information moves between steps.

context-compaction (context-management)

Reflection distills raw page information into shorter summaries before passing state to the next step.

eval-driven-development (evaluation)

The team validates architecture changes against Deep Research Bench, including report and retrieval-quality checks.

Tradeoffs

Agent architecture

Chose supervisor/sub-agent pattern over ReAct single-agent loop.

Rationale: Sub-agents isolate context, enable parallel execution, and reduce context conflicts.

Context transfer

Chose distilled reflection summaries over carrying all raw page content forward.

Rationale: Reflection reduces token growth and cost, though it can risk losing some information.

Benchmark strategy

Chose Deep Research Bench over unmeasured subjective evaluation alone.

Rationale: The benchmark gives external validation across research topics and checks both final reports and retrieval understanding.

Gotchas (what bit them)

Single-agent ReAct accumulated too much context.

Root cause: Each step carried forward prior research context, eventually creating overflow and quality/cost problems.

Resolution: Switch to supervisor/sub-agent architecture with isolated contexts.

Long research workflows create quality drift and hallucination risk.

Root cause: Large contexts across many steps increase the chance that errors or hallucinations enter later synthesis.

Resolution: Use context separation and reflection to keep research threads focused and distilled.

Stateful learning introduces governance concerns.

Root cause: Learning customer-specific domains and preferences across runs raises privacy and cross-contamination questions.

Resolution: The case identifies stateful learning as future work, not a solved production detail.

Evaluation

Tavily used Deep Research Bench, including end-to-end report scoring and retrieval/source-support verification over statement-URL pairs.

Metrics: Deep Research Bench score · comprehensiveness · insight and depth · instruction following · retrieval quality · statement-URL support · token consumption

Outcomes

• Tavily achieved a 52.44 Deep Research Bench score.
• The system consumed significantly fewer tokens than the LangChain Open Deep Research baseline.
• The case claims first-place performance on Deep Research Bench during the period discussed.

The extraction is summary-only because the YouTube source was not fetched; all evidence is therefore from the ZenML summary rather than the transcript.