Mercury: Agentic AI Platform for LLM-Powered Recommendation Systems

ebay · E-commerce · 2025 · source Agentic: yes · promotional bias: high · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: E-commerce recommendation systems

eBay needed to use LLMs for personalized recommendations across a marketplace with billions of live listings and hundreds of millions of customers, while keeping recommendations current, low-latency, safe, reusable, and grounded in actual inventory.

Constraints:

• LLMs produce text, but recommendations must resolve to live marketplace listings.
• eBay inventory and trends change continuously, beyond static model training data.
• The platform must serve hundreds of millions of customers across billions of items.
• Cost and latency targets require near-real-time processing, caching, and preprocessing.
• Recommendation systems need safety, content moderation, prompt-injection defenses, observability, and reusable engineering contracts.

Architecture

Mercury is eBay's internal agentic framework and execution platform for LLM-powered recommendation experiences. It packages LLM work as contract-driven Agents, composes agents into Chains, supports online and near-real-time queue-based execution, connects to multiple model providers and internal fine-tuned models, integrates RAG sources, and uses a Listing Matching Engine to translate LLM-generated product ideas into live listings through query expansion, exact retrieval, vector search, anomaly filtering, and personalized ranking.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  UseCase["Recommendation use case"] --> Mercury["Mercury agent framework"]
  Mercury --> Models["Model provider layer"]
  Mercury --> RAG["RAG source layer"]
  Mercury --> NRT["NRT execution platform"]
  Mercury --> LME["Listing Matching Engine"]
  Mercury --> Obs["Observability and metrics"]
  Mercury --> Safety["Safety and moderation layer"]
  RAG --> LME
  LME --> Exact["Exact match retrieval"]
  LME --> Semantic["Semantic vector retrieval"]
  LME --> Filter["Anomaly filtering"]
  Filter --> Ranking["Personalized ranking"]

Components

Data flow

A recommendation use case invokes Mercury agents that transform inputs, call generative models, and optionally compose Chains or agent graphs. RAG sources provide current user, listing, web, and partner context. The LLM produces product names or queries; the Listing Matching Engine expands those queries, runs exact search and embedding/vector retrieval, filters anomalies by category, price, title, and hard constraints, then ranks remaining listings with a personalized model and caches optimized queries when useful.

Why this architecture

Mercury separates agent contracts from internal implementation so teams can reuse and swap components, then relies on queue-based execution to smooth eBay traffic and GPU utilization. RAG handles current knowledge that LLMs lack, while the Listing Matching Engine solves the core mismatch between textual LLM recommendations and eBay's live inventory.

Models used: LLMs · Llama · Mistral · Lillium · OpenAI models · BERT-based semantic search models · custom deep learning retrieval models

Scale

• eBay's marketplace has over two billion active listings.
  “eBay, a global e-commerce platform with over two billion active listings, enables users to sell and buy virtually anything.”article, stated
• The platform targets hundreds of millions of customers across billions of listings and experiences.
  “scaling these applications to serve hundreds of millions of customers across billions of item listings across numerous experiences is both challenging and costly.”article, stated
• NRT processing has latency requirements as low as hundreds of milliseconds.
  “Mercury’s NRT execution platform is where things get interesting due to cost and latency requirements at eBay (as little as hundreds of ms).”article, stated
• The execution layer is intended to scale local single-instance work to industrial scale in days rather than months.
  “If it works on a developer’s local environment for a single instance, the platform is able to scale to eBay's industrial scale needs.”article, stated
• RAG incorporates periodically updated Common Crawl data and real-time Google searches for trends and recent releases.
  “vast amounts of scraped publicly available content from the internet that is updated periodically via Common Crawl”article, stated

Alternatives considered (roads not taken)

Let each team build one-off LLM recommendation systems

Rejected because: Mercury emphasizes reusable contracts, plug-and-play components, and a growing component library across eBay use cases.

Tie inference directly to user traffic peaks and valleys

Rejected because: That would underutilize GPUs and reserved provider capacity, so Mercury uses queue-based NRT processing to smooth demand.

Use only LLM text output as recommendations

Rejected because: LLM output must be mapped to current inventory, so eBay built a Listing Matching Engine.

Use exact keyword search alone

Rejected because: eBay combines exact search with semantic embedding retrieval for broad or exploratory user intent.

Patterns

multi-agent-orchestration (orchestration)

Mercury treats reusable Agents and Chains as composable units and supports trees or graphs of multiple agents for more autonomous operations.

rag-retrieval (knowledge_grounding)

Mercury uses RAG over current listing, user, web, Common Crawl, Google Search, and partner data to keep LLM recommendations timely.

tool-use-orchestration (tooling)

Agents expose defined inputs, outputs, transformations, model calls, data sources, retrieval components, and safety tools through a common framework.

eval-driven-development (evaluation)

Mercury treats prompts as code and requires prompt changes to pass pull requests, reviews, unit tests, and functional tests.

observability-tracing (operations)

Mercury includes monitoring and maintenance features for reliability, debugging, and performance monitoring.

safety-policy-enforcement (control)

Mercury provides content moderation, LLM safety mechanisms, and internal prompt-injection detection selected according to use-case scale and policy standards.

Tradeoffs

Use contract-based Agents and Chains

Chose Reusable agent components with stable input/output contracts over Bespoke LLM implementations per recommendation use case.

Rationale: Contracts allow teams to swap models and implementations while preserving inter-agent compatibility and reuse.

Smooth work through NRT queues

Chose Distributed queue-based message passing with controllable throughput over Directly coupling processing to live user-traffic spikes.

Rationale: Queueing improves utilization of GPUs and external model capacity while allowing priorities and parallelism to be adjusted.

Combine exact match and semantic retrieval

Chose Search-engine exact retrieval plus embedding/KNN vector retrieval over Relying on either lexical matching or semantic retrieval alone.

Rationale: Exact search handles precise product requests, while semantic retrieval handles exploratory or differently worded intent.

Cache optimized queries when useful

Chose Reuse optimized queries for future needs over Always regenerate and rerank from scratch.

Rationale: Caching can reduce cost and latency, but it must be balanced against freshness in a dynamic inventory marketplace.

Gotchas (what bit them)

LLM product ideas are not directly actionable marketplace recommendations.

Root cause: LLMs output text and lack access to eBay's current dynamic inventory.

Resolution: eBay built the Listing Matching Engine to map generated queries and product ideas to live listings through exact and semantic retrieval.

User traffic patterns can waste expensive GPU and model-provider capacity.

Root cause: Shopping activity has peaks and valleys, but GPU allocations and external reserved capacity are costly when underutilized.

Resolution: Mercury uses distributed queue-based NRT execution to smooth demand, control throughput, retry failures, and resume after disruptions.

RAG quality depends on current, relevant data sources with different latency/cost profiles.

Root cause: Current inventory, trends, user context, web data, and partner signals are heterogeneous and stale at different rates.

Resolution: Mercury integrates many structured and unstructured sources and lets each use case choose latency, relevance, and cost tradeoffs.

LLM-generated retrieval results need post-generation validation.

Root cause: Generated product queries can yield listings that mismatch category, price, title, or policy constraints.

Resolution: The Listing Matching Engine applies anomaly detection, category/price validation, title alignment, hard filters, and personalized ranking.

Evaluation

The article describes production engineering gates and operational monitoring more than offline model metrics: prompts go through pull requests, code reviews, unit tests, and functional tests; Mercury includes observability and metrics; retrieval results are filtered and ranked; and the NRT layer supports retries and auto-resume.

Metrics: hundreds-of-milliseconds latency target · pull requests for prompt changes · unit testing and functional testing · reliability and debugging monitoring · throughput controls per use case · automated retries and auto-resume

Outcomes

• Mercury provides eBay teams with reusable agentic infrastructure for LLM-powered recommendation experiences.
• The platform links LLM/RAG-generated product ideas to live inventory through the Listing Matching Engine.
• Distributed NRT processing addresses cost, latency, retry, priority, and throughput needs at eBay scale.
• The recommendation pipeline combines query expansion, exact search, semantic vector retrieval, anomaly filtering, and personalized ranking.
• The system incorporates prompt-as-code practices, observability, content moderation, and prompt-injection defenses.

The source cites over two billion active listings, hundreds of millions of customers, numerous experiences, latency requirements as low as hundreds of milliseconds, Common Crawl and real-time Google data sources, and industrial-scale execution.