Large-Scale Semantic Search Platform for Food Delivery

uber · E-commerce · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Food-delivery semantic search and retrieval infrastructure

Uber Eats needed semantic search across stores, dishes, grocery, and retail items because lexical matching could not handle synonyms, typos, shorthand, multilingual queries, or ambiguous terms. The system had to serve a multi-billion document corpus with low latency, fresh data, global vertical coverage, strong recall, model/index version control, validation gates, rollback, and cost discipline.

Constraints:

• search is a primary discovery funnel for Uber Eats orders
• queries include synonyms, typos, shorthand, mixed language, and ambiguous terms
• query embeddings must be computed online while document embeddings can be generated offline
• the document corpus includes billions of candidates
• the same embedding model supports Uber Eats verticals and markets globally
• indexes must refresh on a biweekly cadence without disrupting live reads
• cost, latency, recall, storage, version consistency, and rollback safety must be balanced

Architecture

Uber Eats built a two-tower semantic-search platform using Qwen as the embedding backbone, fine-tuned on in-house Uber Eats data. Query embeddings are calculated online, while document embeddings are generated offline at feature granularity, stored in feature-store tables, joined back to billions of candidates, and indexed in Lucene Plus with HNSW graphs. Matryoshka Representation Learning provides multiple embedding dimensions from one model, quantization reduces serving cost, boolean pre-filters shrink candidate sets before ANN search, and a biweekly blue/green column refresh pipeline validates completeness, backward compatibility, correctness, and serving-time model/index compatibility.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  train["Training infrastructure"] --> model["Two-tower embedding model"]
  train --> registry["Versioned artifact registry"]
  model --> online["Online query embedding service"]
  model --> offline["Offline document embedding service"]
  offline --> store["Feature store"]
  store --> index["Lucene Plus index"]
  online --> ann["ANN retrieval service"]
  index --> ann
  ann --> rankers["Downstream rankers"]
  refresh["Biweekly refresh workflow"] --> registry
  refresh --> offline
  refresh --> index
  refresh --> gates["Validation gates"]
  gates --> rollback["Rollback controller"]
  compat["Serving-time compatibility check"] --> rollback
  online --> compat
  index --> compat

Components

Data flow

The system trains a two-tower embedding model with Qwen, MRL-based infoNCE loss, PyTorch, Ray, DDP, and DeepSpeed ZeRO-3. Offline inference embeds stores or items at feature level, writes entity-keyed vectors to feature-store tables, joins them to the full catalog, and builds Lucene Plus HNSW indexes with float and quantized vectors. At serving time, an online service computes query embeddings, structured filters reduce the search space, ANN retrieves candidates, optional micro-reranking refines results, and downstream rankers consume the retrieval cohort. Refresh jobs build new index snapshots, validate active and inactive columns, deploy non-prod correctness checks, and sample production requests for model/index ID compatibility.

Why this architecture

Uber chose a two-tower architecture to decouple low-latency online query embedding from expensive offline document embedding. Qwen provided world knowledge and cross-lingual ability, while fine-tuning adapted the embedding space to Uber Eats. MRL, quantization, pre-filters, and micro-reranking let the system trade off recall, latency, and storage without retraining separate models for each use case. Blue/green columns and validation gates keep the biweekly refresh process safe without doubling index storage.

Models used: Qwen · two-tower deep neural network · Matryoshka Representation Learning · compact neural network micro-reranker · HNSW vector index

Scale

• The same embedding model supports all Uber Eats verticals and markets.
  “Currently, this single embedding model supports the content embedding task for all Uber Eats verticals and markets.”article, stated
• Training processes hundreds of millions of data points.
  “This setup, combined with mixed-precision training and gradient accumulation, allows us to train on hundreds of millions of data points efficiently.”article, stated
• Offline inference joins feature-level embeddings back to billions of candidates.
  “then join these embeddings back to the full catalog of billions of candidates before indexing.”article, stated
• Lowering shard-level k from 1,200 to about 200 reduced latency by 34% and CPU by 17%.
  “Lowering shard-level k from 1,200 to around 200 yielded a 34% latency reduction and 17% CPU savings in our experiments”article, stated
• Int7 quantization maintained recall above 0.95 while cutting latency by more than half versus fp32.
  “applying scalar quantization (int7) further cut compute costs, reducing latency by more than half compared to fp32 while maintaining recall above 0.95.”article, stated
• MRL enabled 256-dimensional embeddings with less than 0.3% degradation for English and Spanish and nearly 50% storage reduction.
  “256 dimensions) with almost no loss (< 0.3% for English and Spanish) in retrieval quality compared to full-size embeddings”article, stated

  “while also reducing storage costs by nearly 50%.”article, stated
• The semantic-search refresh workflow runs every two weeks.
  “runs a scheduled end-to-end workflow that updates the embedding model and serving index in lockstep on a biweekly cadence.”article, stated

Alternatives considered (roads not taken)

Rely on lexical search alone

Rejected because: Lexical matching fails on synonyms, typos, shorthand, language mixing, and ambiguous terms because it sees strings rather than meaning.

Compute all embeddings online

Rejected because: The architecture decouples query and document computation so only query embeddings are online while document embeddings are batch-generated.

Embed every catalog document individually

Rejected because: Uber embeds at feature level and joins vectors back to the catalog to reduce redundant work at corpus scale.

Maintain two separate blue and green indexes

Rejected because: Separate indexes would simplify reliability but would significantly increase storage and maintenance costs.

Patterns

hybrid-search (retrieval)

Uber Eats combines semantic vector retrieval with locale-aware lexical fields and boolean pre-filters for production search across verticals and languages.

fine-tuning-adaptation (model-adaptation)

Uber fine-tuned the Qwen embedding backbone with proprietary Uber Eats data to adapt semantic space to delivery-search scenarios.

latency-optimization (serving)

MRL enables smaller embedding cuts from one trained model so serving can trade quality for latency, storage, and retrieval cost.

cost-aware-routing (operations)

Uber tunes shard k, quantization, embedding dimension, pre-filters, and feature-level embedding to reduce latency, CPU, and storage without materially degrading recall.

canary-rollout (deployment)

Uber refreshes blue/green index columns and rolls out model changes gradually, with rollback triggered by serving-time compatibility errors.

eval-driven-development (evaluation)

The refresh pipeline blocks deployment through completeness, backward-compatibility, and correctness checks before serving a new index.

observability-tracing (operations)

Uber samples production requests, emits compatibility-error metrics, logs context, alerts, and rolls back on sustained version mismatches.

Tradeoffs

Embedding architecture

Chose two-tower online/offline embedding separation over coupling query and document embedding calculation into one serving path.

Rationale: The split lets Uber compute query embeddings in real time while document embeddings are scheduled offline at corpus scale.

Index-refresh topology

Chose one index with blue and green embedding columns over two fully separate blue and green indexes.

Rationale: The single-index approach avoids higher storage and maintenance costs but requires stronger validation to offset reliability risk.

Embedding representation size

Chose MRL cut-down embeddings such as 256 dimensions over serving full-size embeddings for every use case.

Rationale: MRL lets downstream tasks trade quality for latency and storage while preserving strong retrieval quality in evaluated languages.

ANN cost tuning

Chose lower shard-level k and quantized vectors over higher k and fp32 vectors everywhere.

Rationale: Experiments showed large latency and CPU savings with negligible recall impact and recall above 0.95 under int7 quantization.

Gotchas (what bit them)

Pure lexical search fails on real marketplace query language.

Root cause: User queries contain synonyms, typos, shorthand, language mixing, and ambiguous words that string matching cannot resolve.

Resolution: Encode queries and documents as vectors in a shared semantic space while retaining lexical and boolean filters for production constraints.

Embedding every document independently would waste expensive inference.

Root cause: The document corpus is large enough that redundant embedding computation would dominate cost.

Resolution: Compute embeddings at feature level, store them by entity ID, and join them back to billions of candidates before indexing.

Refreshing a shared blue/green-column index can corrupt the currently serving column.

Root cause: The refresh build re-materializes both columns even when only the inactive column is intended to change.

Resolution: Block deployments with completeness, backward-compatibility, and correctness validations before switching traffic.

Model/index version mismatches can still happen after build-time checks.

Root cause: Human error or configuration bugs can map the wrong model to the active index column or build an index against an unexpected model.

Resolution: Store model IDs in index metadata, sample requests for compatibility, emit metrics, alert, and auto-rollback the model deployment.

Evaluation

Uber evaluates the platform through controlled cost/quality experiments, offline retrieval-quality comparisons, index-build validation gates, non-production query replay, production compatibility metrics, and rollback alerts.

Metrics: recall · latency reduction · CPU savings · storage reduction · document-count completeness · byte-for-byte backward compatibility · production compatibility-error metric

Outcomes

• Uber reports a scalable multilingual search system across restaurants, grocery, and retail
• k tuning reduced latency by 34% and CPU by 17%
• int7 scalar quantization reduced latency by more than half while maintaining recall above 0.95
• 256-dimensional MRL embeddings had less than 0.3% quality loss for English and Spanish while reducing storage by nearly 50%
• the production system refreshes embedding models and indexes every two weeks with validation and rollback controls

The system targets all Uber Eats verticals and markets, trains on hundreds of millions of data points, indexes a multi-billion-document corpus, and refreshes model/index state biweekly.