Building a Universal Search Product with RAG and AI Agents

dropbox · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Enterprise search and knowledge management

Dropbox needed Dash to answer business-user questions over fragmented, diverse, multi-modal enterprise data while preserving access controls, latency, quality, and safe multi-step agent execution.

Constraints:

• Information is scattered across applications and formats, creating collaboration, productivity, and security problems.
• Business data includes text, images, audio, video, documents, meeting notes, tasks, and other modalities.
• Search responses needed high relevance while staying under tight latency budgets.
• Multi-step AI agents needed code execution with static analysis, type enforcement, dry runs, and a minimal secure runtime.

Architecture

Dropbox Dash combines RAG for common retrieval and summarization tasks with AI agents for complex multi-step workflows. Retrieval uses traditional IR, on-the-fly chunking, and embedding-based reranking to keep quality high under latency constraints. Agents express plans in a Python-like DSL, then a custom minimal interpreter validates code with static analysis, dry runs, and runtime type enforcement before execution.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  dash-user[Dash User] --> dash-product[Dash Product]
  dash-product --> access-control[Access Control]
  enterprise-content-sources[Enterprise Content Sources] --> access-control
  access-control --> retrieval-system[Retrieval System]
  retrieval-system --> rag-pipeline[RAG Pipeline]
  rag-pipeline --> llm[LLM]
  dash-product --> ai-agent[AI Agent]
  llm --> ai-agent
  ai-agent --> dsl-plan[DSL Plan]
  dsl-plan --> custom-interpreter[Custom Interpreter]
  custom-interpreter --> enterprise-content-sources

Components

Data flow

A Dash query first goes through access-controlled retrieval over fragmented enterprise content. For common questions, RAG retrieves relevant chunks through lexical IR, query-time chunking, and reranking, then an LLM generates an answer. For multi-step tasks, an LLM writes a simple DSL plan, Dropbox validates it with static analysis and type enforcement, fills missing helper functionality when needed, executes steps, and returns the final typed result.

Why this architecture

Dropbox split simple retrieval from multi-step agents because RAG is fast and reliable for common information retrieval, while business tasks like finding meeting notes need planning, tool/helper execution, and code-level validation for safety and debuggability.

Models used: large language models · embedding models · LLM judge

Scale

• Dropbox targeted under 1-2 seconds for more than 95% of queries.
  “Specifically, we stayed under 1–2 seconds for over 95% of our queries, which allows us some latency budget for the rest of the pipeline so that our users don’t click away because the response time is too long.”article, stated
• The retrieval design uses periodic data syncs and webhooks when appropriate.
  “cases, we prioritized reasonable latency but also high-quality and reasonable data freshness, with both periodic data syncs and the implementation of webhooks whenever appropriate. Specifically, we stayed under 1–2 seconds for over 95% of our queries, which allows us some latency budget for the”article, stated
• Evaluation covered multiple public datasets, including Natural Questions, MuSiQue, and Microsoft Machine Reading Comprehension.
  “including Google’s Natural Questions (featuring real user queries with large documents); MuSiQue (with multi-hop questions requiring information linking across different passages)”article, stated

Alternatives considered (roads not taken)

Pure vector/semantic search over pre-chunked embeddings

Rejected because: Advanced semantic methods can add latency and may not satisfy Dash's response-time target.

Direct API access to data sources for freshness

Rejected because: On-the-fly API calls can push latency above a few seconds.

Use RAG for complex multi-step business tasks

Rejected because: RAG cannot plan and execute multi-step workflows such as resolving dates, finding meetings, and retrieving attached documents.

Full-featured CPython-compatible interpreter

Rejected because: Dropbox only implemented minimal required functionality to reduce security risk.

Patterns

rag-retrieval (retrieval)

Dash uses RAG for search, summarization, answers, insights, and drafts over business content.

hybrid-search (retrieval)

Dropbox combines lexical IR with embedding-feature rerankers rather than relying on only lexical or only semantic search.

retrieval-ranking (retrieval)

A larger but efficient embedding model reranks initial results to put relevant chunks first.

tool-use-orchestration (agents)

Dash agents execute planned helper calls through a Python-like DSL and custom interpreter.

structured-output-contract (agents)

The custom interpreter uses runtime type enforcement so generated logic returns expected typed values.

eval-driven-development (quality)

Dropbox evaluated retrieval/model options using public datasets, custom judges, source precision/recall/F1, correctness, and completeness metrics.

Tradeoffs

Retrieval architecture

Chose Traditional IR with on-the-fly chunking and embedding reranking over Pure vector search or pure lexical search.

Rationale: The combination met Dropbox's speed and relevance goals while keeping costs in check.

Task architecture

Chose RAG for common information retrieval and agents for multi-step tasks over One universal LLM workflow.

Rationale: RAG handles simple retrieval well, but agents are required when a query needs planning and execution.

Agent plan representation

Chose Python-like DSL with static analysis and type enforcement over Opaque natural-language reasoning only.

Rationale: Code plans expose where logic fails and make tests more deterministic.

Interpreter scope

Chose Minimal custom interpreter over Feature parity with CPython.

Rationale: Minimal runtime functionality turns full-interpreter security risks into non-issues.

Gotchas (what bit them)

Retrieval quality sets the ceiling for answer quality.

Root cause: The LLM can only answer from the context retrieved at inference time.

Resolution: Invest in IR, chunking, reranking, and end-to-end evaluation of retrieval plus generation.

Freshness, latency, quality, and cost pull against each other.

Root cause: Frequent re-indexing, third-party API calls, larger embeddings, and reranking all add latency or compute.

Resolution: Use periodic syncs/webhooks and a traditional IR plus on-the-fly chunking/reranking balance.

Model variations make prompt reuse unreliable.

Root cause: Different LLMs respond differently to the same prompt.

Resolution: Carefully select models and optimize prompts per task while preserving model-agnostic architecture.

Executing LLM-generated code creates security and correctness risk.

Root cause: Generated code may contain missing functionality, incorrect logic, or unsafe operations.

Resolution: Use a custom minimal interpreter with static analysis, dry runs, runtime type enforcement, and security controls.

Evaluation

Dropbox evaluated RAG and agent behavior with public datasets, custom LLM judges for correctness and completeness, source precision/recall/F1, static analysis, dry runs, runtime type enforcement, and type-level checks of agent outputs.

Metrics: Answer correctness · Answer completeness · Source precision · Source recall · Source F1 · Latency under 1-2 seconds for >95% of queries · Runtime type correctness · Step-level logic failures

Outcomes

• Dash achieved high-quality retrieval results under 2 seconds for over 95% of queries.
• The model-agnostic RAG architecture allowed customer/provider flexibility.
• The DSL/interpreter design made agent failures more inspectable and tests more deterministic.

The article reports production-target latency for Dash queries and detailed engineering choices, but not user-count or traffic-volume metrics.