Building Deep Research: A Production AI Research Assistant Agent

google_deepmind · Tech · 2024 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Production AI research assistants

Google DeepMind built Deep Research for multi-faceted research tasks where ordinary search forces users to open dozens of tabs, compare noisy sources, and synthesize a report manually. The system needed to spend multiple minutes browsing and reasoning, let users steer the research plan, maintain state across long-running work, and evaluate long-form reports whose quality is hard to capture with simple benchmarks.

Constraints:

• The product targets queries with multiple facets where users might otherwise spend a weekend opening 50-60 tabs.
• A 5-10 minute research run must spend compute on the user's actual intent, so the plan needs transparency and steering before work starts.
• The web is fragmented and noisy, requiring iterative search, page reading, inconsistency handling, and synthesis.
• Long-running agent jobs need async state, retries, and continuation after users close their browser or computer.
• Evaluation is difficult because output reports have high entropy and require groundedness, completeness, and comprehensiveness judgments.

Architecture

Deep Research wraps a specialized, post-trained Gemini 1.5 Pro agent in a product flow that first drafts an editable research plan. After the user starts or edits the plan, the agent parallelizes obvious substeps, uses search and page-browsing tools, reads prior outputs to decide what to double-click, and iterates until it decides the plan is complete. The browsed material remains in long context for follow-up questions when possible, with retrieval available past context limits. An async execution platform maintains state and retries for multi-minute jobs, while the product evaluates behavior with dev-set metrics, human review, groundedness/completeness attributes, and an ontology of research behavior types.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  user[User] --> gemini-deep-research-ui[Gemini Deep Research UI]
  gemini-deep-research-ui --> editable-research-plan[Editable research plan]
  gemini-deep-research-ui --> report-chat-workspace[Report chat workspace]
  editable-research-plan --> gemini-deep-research-agent[Gemini Deep Research agent]
  gemini-deep-research-agent --> search-tool[Search tool]
  gemini-deep-research-agent --> web-page-browser[Web page browser]
  gemini-deep-research-agent --> source-context-store[Source context store]
  gemini-deep-research-agent --> analysis-and-drafting-stage[Analysis and drafting stage]
  gemini-deep-research-agent --> async-execution-platform[Async execution platform]
  search-tool --> source-context-store
  web-page-browser --> source-context-store
  source-context-store --> analysis-and-drafting-stage
  analysis-and-drafting-stage --> report-chat-workspace
  report-chat-workspace --> user
  evaluation-harness[Evaluation harness] --> gemini-deep-research-agent

Components

Data flow

The user submits a broad research query in Gemini. Deep Research generates a proposed plan, lets the user review or edit it, then uses that accepted plan as a contract for execution. The agent launches parallel search and page-browsing steps, stores the material it reads in context, reasons over previous outputs to refine later searches, enters an analysis/drafting stage, self-critiques the draft, and presents a report with citations in a side-by-side chat/report workspace. Follow-up questions can either use the existing context, trigger more deep research, or edit the report.

Why this architecture

The architecture trades instant response for agentic depth. A visible plan helps users steer a long-running job before compute is spent. Search and deep browsing are separated so the agent can first cover breadth and then double-click when previous findings expose partial or inconsistent information. Long context keeps fresh research material available for follow-up questions, while async execution protects multi-minute runs from browser closures and failures.

Models used: Gemini 1.5 Pro · Gemini · Gemma

Scale

• Deep Research runs normally take about five minutes and may take five to ten minutes.
  “browses the web for about 5 minutes, and then outputs a research report for you to review”transcript, stated

  “something takes about 5-6 minutes trying to perform your research.”transcript, stated
• The target user pain was replacing research sessions that could involve 50-60 browser tabs.
  “you spend like a weekend, you know, just opening like 50-60 tabs”transcript, stated
• The shipped version had a 10-minute hard stop, while an internal hardcore version took about 15 minutes.
  “We had like a hardcore mode that takes like 15 minutes and then what we actually shipped is a thing that takes 5 minutes”transcript, stated

  “Can never take more than 10 minutes.”transcript, stated
• The system uses Gemini's one-to-two-million-token context window before falling back to retrieval.
  “you have 1 million to 2 million token context has it ever exceeded 2 million?”transcript, stated

  “we have retrieval mechanisms if required so uh we natively try to use the context as much as it's available”transcript, stated
• The async execution platform is already used across Android and iOS surfaces.
  “we wrapped on all Android phones and then iOS is this week.”transcript, stated
• The async platform was designed for hundreds of LLM calls and future hour or multi-day jobs.
  “you can imagine once you start doing hour or multi-day jobs”transcript, stated

  “in short like it's stable, you know, for for like hundreds of LLM calls.”transcript, stated

Alternatives considered (roads not taken)

Ask users direct follow-up questions before research.

Rejected because: The team believed users often do not know what questions or facets to ask about, so showing an editable proposed plan both teaches the topic and invites steering.

Use retrieval for all accumulated research material.

Rejected because: The team prefers long context for recent research tasks because RAG similarity can struggle when queries combine multiple attributes and users need complex comparisons.

Render pages and use vision for web understanding.

Rejected because: The team judged rendering latency as not worth the value for head queries, while leaving vision for tail cases where it matters.

Use static workflow engines for orchestration.

Rejected because: Agentic research requires dynamic adaptation to the agent's decisions, while Airflow and Step Functions are framed as static execution graphs.

Optimize for external benchmarks as the main product target.

Rejected because: The team considered benchmarks useful but not directly representative of Deep Research's desired product experience.

Patterns

human-in-the-loop-gate (ux)

Deep Research shows a proposed plan before execution and lets users edit or conversationally steer it before the agent spends minutes researching.

tool-use-orchestration (agent-architecture)

The agent uses search and deep page browsing tools, starts parallel substeps, and uses findings from earlier turns to choose later tool calls.

prompt-chaining (agent-architecture)

Deep Research decomposes a query into plan, browse, analysis, draft, and self-critique stages instead of relying on a single answer-generation call.

durable-execution (infrastructure)

Google built an async execution platform so multi-minute research jobs can survive user departure, failures, and retries without losing progress.

rag-retrieval (context)

Deep Research keeps recent research material in long context, but falls back to retrieval when context limits are exceeded or older material is less central.

eval-driven-development (quality)

The team evaluates Deep Research with auto-computed behavior distributions, human review, groundedness/completeness/comprehensiveness criteria, and a use-case ontology.

Tradeoffs

Show a proposed plan instead of asking direct clarification questions.

Chose editable research plan over direct follow-up-question interview.

Rationale: A proposed plan teaches users the facets of an unfamiliar topic while giving them a steering surface before compute-heavy research begins.

Spend minutes researching rather than optimizing for instant latency.

Chose multi-minute research runs over fast synchronous answer generation.

Rationale: The team found that users valued the visible work when it produced deeper exploration and verification, despite this contradicting normal Google latency orthodoxy.

Keep fresh research in long context and move older material to retrieval.

Chose long context first with RAG fallback over retrieval-only session memory.

Rationale: Recent research tasks are more likely to need complex follow-up comparisons, while older tasks can be retrieved when needed.

Use markdown conversion where useful instead of forcing raw HTML into the model.

Chose hybrid HTML and markdown processing over raw HTML only.

Rationale: Markdown reduces markup noise, while raw or embedded HTML remains available when the access path or content requires it.

Evaluate by product use-case behavior rather than optimizing mainly for public benchmarks.

Chose human and ontology-based product evaluation over benchmark-first optimization.

Rationale: The team considered benchmarks useful but not directly predictive of whether Deep Research creates the desired user value.

Gotchas (what bit them)

Users may not edit the plan even when the steering surface exists.

Root cause: Many users accept the model's first plan, treating it like an 'I'm Feeling Lucky' flow.

Resolution: The team added explicit UI affordances and still treats the plan as useful transparency even when users do not edit it.

Iterative planning is hard to make generalizable.

Root cause: Training separate planning traces per domain or per ontology type would be too brittle and expensive.

Resolution: The team relied on post-training, data augmentation, and careful tuning to learn the behavior without overfitting away useful pretraining knowledge.

Raw web content is noisy and sometimes not worth rendering visually.

Root cause: HTML includes JavaScript, CSS, snippets, and visual edge cases; rendering pages adds latency.

Resolution: The system uses both raw/HTML-compatible representations and markdown conversion, and postpones vision to cases where value outweighs latency.

Long-running research jobs are failure-prone if treated like synchronous chat.

Root cause: Five-to-six-minute agent runs can outlive a browser session and may fail mid-journey.

Resolution: Deep Research uses an async platform with state retention and retry logic.

Auto-evaluating long-form research reports is intrinsically difficult.

Root cause: The output space is high entropy and the quality target includes completeness, comprehensiveness, and groundedness, not just a single answer.

Resolution: The team combines auto-computed distribution metrics, human review, and ontology-based evaluation sets.

Directly relying on model memory can undermine source-grounded research.

Root cause: Thinking models and model memory may know facts, but users still need sourced claims for trust.

Resolution: The team treats balancing model memory with external grounding as an explicit research-agent challenge.

Evaluation

Deep Research is evaluated with a mixture of automatic behavioral statistics, human review, product-defined quality attributes, and an ontology of research behavior types. Automatic metrics detect distribution changes such as planning time, plan length, and iterative-step counts across a dev set. Human evaluation judges comprehensiveness, completeness, and groundedness across use cases ranging from broad/shallow option exploration to specific/deep research and compound planning tasks.

Metrics: time spent on planning · number of iterative browsing/planning steps · research plan length · number of research plan steps · distribution shifts across a dev set · comprehensiveness · completeness · groundedness · coverage across research behavior ontology types · human PM review quality

Outcomes

• Deep Research can browse the web for about five minutes and generate a research report with citations
• users can review and edit the research plan before execution
• the agent can search and browse pages in parallel, then iteratively refine later searches from prior findings
• all browsed sites can remain in context for follow-up questions until retrieval becomes necessary
• users can close their computer and receive a phone notification when the async research job completes
• the product runs on Android and was rolling out to iOS at the time of the discussion
• the team found that users value longer research time when it produces a deeper result

The case describes 5-6 minute research jobs, a 5-minute shipped mode, a 10-minute hard cap, an internal 15-minute hardcore mode, 50-60-tab user research journeys, one-to-two-million-token context, Android rollout and iOS launch timing, and async infrastructure designed for hundreds of LLM calls and future hour or multi-day jobs.